Browser and Computer Use

Studio has two different ways to operate a graphical interface:

• Browser tools run a managed headless browser session for websites and web consoles.
• Computer Use observes and, with approval, controls applications on your local macOS desktop.

Use the browser whenever the task can stay inside a website. Use Computer Use for a native desktop application or an operating-system surface that has no safer API, connector, command-line, or browser path.

​

Choose the right surface

Studio routes to structured tools before pixel control when one is available. Structured operations are easier to review, retry, audit, and constrain.

​

Browser sessions

Ask Copilot to open or use a website. The active headless-browser session appears in the activity area and opens a live browser tab. The live view shows the current URL, connection state, interactive element strip, and the next proposed intent. Browser tools can navigate, inspect content, fill forms, click elements, capture screenshots, collect downloads, and gather network or trace evidence. Actions that could submit data or change external state pause for approval unless browser autopilot is enabled.

​

Take control

Select Take over when you need to interact with the browser yourself. Studio pauses AI control and sends your pointer, keyboard, and scroll input to the headless session. Select Release to return control to Copilot. Click the live preview while Copilot controls the session to open a picture-in-picture view. If an approval is needed, Studio closes picture-in-picture and focuses the conversation so the request is not hidden.

​

Browser autopilot

The lightning control in the browser toolbar enables autopilot for browser actions in that session. While enabled, browser actions are auto-approved.

​

Browser AI Grounding

The composer option Browser AI Grounding gives Copilot additional visual grounding while it operates browser pages. This can improve performance on unfamiliar or visually complex sites, but consumes additional tokens per browser action. Turn it on for difficult interfaces and leave it off when semantic element data is sufficient.

​

Computer Use on macOS

Computer Use is available in the Studio desktop app on macOS. It can list applications and windows, capture the current desktop, read visible text, and operate the pointer and keyboard through the local Studio helper. Observation operations are read-only. State-changing operations—clicking, typing, scrolling, opening an application, or activating a control—require approval unless your current autonomy settings explicitly bypass approvals.

​

Grant permissions

Before first use:

1. Open Settings → Computer Use.
2. Grant Screen Recording so Studio can see the desktop before acting.
3. Grant Accessibility so approved actions can move the pointer and type.
4. Return to Studio and select Re-check.

If macOS still reports a permission as blocked, quit and restart Studio. macOS can retain the previous permission state until the app process restarts.

​

Secondary cursor

Studio uses a secondary-cursor helper for background desktop actions. Its status appears while Computer Use runs. The helper checks the active window and display before acting so a moved or changed window does not silently receive input intended for the old target.

​

Review an approval

Before approving a browser or Computer Use action, check:

• The website URL, application, window title, and signed-in account.
• The proposed target element and input text.
• Whether the action submits, purchases, deletes, publishes, or changes access.
• Whether a structured tool could perform the same action with a clearer boundary.

Take over or reject when the visual state does not match Copilot’s description.

​

Troubleshooting

​

Related