Skip to content

feat: add batch computer action endpoint (POST /computer/batch)#144

Merged
rgarcia merged 3 commits intomainfrom
raf/kernel-969-batch-computer-use-api
Feb 5, 2026
Merged

feat: add batch computer action endpoint (POST /computer/batch)#144
rgarcia merged 3 commits intomainfrom
raf/kernel-969-batch-computer-use-api

Conversation

@rgarcia
Copy link
Contributor

@rgarcia rgarcia commented Feb 5, 2026

Checklist

  • A link to a related issue in our repository
  • A description of the changes proposed in the pull request.
  • @mentions of the person or team responsible for reviewing proposed changes.

Summary

Adds a POST /computer/batch endpoint that accepts an array of computer actions and executes them sequentially while holding a single input lock, eliminating per-action network round-trip latency.

Key changes:

  • New endpoint POST /computer/batch — accepts a BatchComputerActionRequest with an ordered actions array. Supported action types: click_mouse, move_mouse, type_text, press_key, scroll, drag_mouse, set_cursor, and a new sleep action (with duration_ms for configurable delays between actions)
  • Action schemas are identical to the existing individual endpoint request bodies (via $ref), so there's nothing new for users to learn per-action
  • Execution semantics: actions run sequentially under a single inputMu lock acquisition; stops on first error and reports the failing action index in the error message (e.g. "actions[2] (click_mouse): coordinates must be non-negative")
  • Refactored existing handlers — extracted core validation+execution logic from 7 computer control handlers into private do* methods (e.g. doClickMouse, doMoveMouse), making the public handlers thin wrappers. This avoids mutex re-entrancy issues and enables clean reuse from the batch handler
  • Error classification — introduced validationError (→ 400) and executionError (→ 500) sentinel types with isValidationErr() helper for consistent HTTP status mapping in both individual and batch handlers

Test plan

  • All existing unit tests pass (go test ./...)
  • go vet ./... clean
  • New unit tests for error sentinel types (TestValidationError, TestExecutionError, TestIsValidationErr_Nil)
  • Manual testing of batch endpoint with a sequence of actions (e.g. move_mouse → sleep → click_mouse)

Made with Cursor


Note

Medium Risk
Touches multiple input-control handlers and changes how errors are classified and surfaced, so regressions could affect automation reliability and client-visible status codes. The new batch execution path also increases the impact of validation/execution bugs because it holds the input lock across a whole sequence.

Overview
Adds POST /computer/batch, allowing clients to submit an ordered list of computer actions (mouse, keyboard, cursor, plus a bounded sleep) that are executed sequentially while holding the existing inputMu lock; execution stops on the first failing action and returns an error message annotated with the action index/type.

Refactors existing computer-control endpoints (MoveMouse, ClickMouse, TypeText, PressKey, Scroll, DragMouse, SetCursor) to delegate to shared do* helpers and introduces validationError vs executionError to consistently map failures to HTTP 400 vs 500 across both single-action and batch flows; updates generated oapi types/clients, Swagger spec, and adds unit tests for the new error classification helpers.

Written by Cursor Bugbot for commit 8fdeda8. This will update automatically on new commits. Configure here.

Adds a new endpoint that accepts an array of computer actions and
executes them sequentially while holding a single input lock, avoiding
per-action network round-trip latency. Includes a new "sleep" action
type for configurable delays between actions.

Refactors existing computer control handlers to extract core logic into
private do* methods, enabling reuse from both individual endpoints and
the batch endpoint. Introduces validationError/executionError sentinel
types for clean 400/500 classification.

Co-authored-by: Cursor <cursoragent@cursor.com>
Address bugbot review comments:
- Validate duration_ms against bounds (0-30000ms) before sleeping
- Use select with timer + ctx.Done instead of time.Sleep so the sleep
  is interrupted on context cancellation and doesn't hold inputMu
  indefinitely

Co-authored-by: Cursor <cursoragent@cursor.com>
@rgarcia rgarcia requested a review from masnwilliams February 5, 2026 19:41
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Remove the extra actionIndex parameter from doSleep so it matches the
(ctx, body) signature of all other do* methods.

Co-authored-by: Cursor <cursoragent@cursor.com>
Copy link
Contributor

@masnwilliams masnwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - clean refactoring to enable batch actions. the do* pattern is well-structured and error handling is solid.

@rgarcia rgarcia merged commit a90f3b3 into main Feb 5, 2026
5 checks passed
@rgarcia rgarcia deleted the raf/kernel-969-batch-computer-use-api branch February 5, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants