feat(desktop): run a chat's desktop tools in the background executor by waleedlatif1 · Pull Request #8668 · simstudioai/sim

waleedlatif1 · 2026-10-06T06:06:49Z

Summary

Phase 2a of the desktop background executor. The Sim desktop app now picks up and runs the desktop tool calls of turns bound to it from Electron main, so a chat keeps working after the user switches chats, switches workspace, or reloads the window. It uses the device protocol from #8644 and #8650 as is, and stays dormant until Sim enables the executor for the user.

Based on staging, now that #8650 is squashed.

What the desktop app does

Identity. It keeps a stable install id in userData. It registers that id with the app partition's own session on startup, on sign-in and session change, and when the browser or terminal is switched on or off. Signing out stops every action, clears the outbox and retires the id. A 409 (the id belongs to another account) mints a new one.
Pickup. An SSE doorbell, plus the reconciling inbox read. The inbox is read on connect, on every ring, on wake (powerMonitor resume), when the network returns, and on Sim's reconcile timer. A missed doorbell is normal: the timer catches it inside the pickup window. Reconnects use backoffWithJitter, and a silent stream is replaced after two missed heartbeats.
Claim, then queue. A call is claimed as soon as it is offered, then queued per chat and per surface. Its lease is renewed from the claim until Sim acknowledges the result, so a backlog never misses the pickup window. Calls in one chat run in the order they were offered, and different chats run side by side.
Execution. Each call runs from Sim's record of it, in that chat's own browser scope, terminals or granted folders. This covers browser_*, terminal, read_local_file and user-local read/grep/glob. A surface switched off on the machine answers "not run".
Outbox. An encrypted journal (claiming → claimed → started → result), written before the step it guards, using safeStorage like the other userData stores. After a restart:
- a call that never started is reported as not started;
- one that started is reported as outcome unknown;
- a finished one is reported with its real result.
Nothing runs twice. Results are retried until Sim answers. Any answer acknowledges the result: recorded, duplicate or superseded. A refused token (404/410) is final and is dropped.
Stop. An inbox cancel, or a 410 on lease renewal, stops the action through the browser's own cancel. Terminal runs get a real cancel: Ctrl-C, then SIGTERM, then SIGKILL to that command's own process group, with the shell left running. It also reaches input, kill and pane close, between keystrokes. A tmux run gets Ctrl-C in its own window, and the window is closed if that does not end it.
Agent commands end with the session. Sign-out, an account change, and switching Terminal off each stop every command the agent started, including a run whose wait window has passed and any tmux run window, before the shells go. A command the user started is never touched: only commands a run started count as the agent's.
Approvals. A chat in the background that waits on the user's approval raises a native notification. The notification only opens the chat at its approval card. It closes once the call is decided, and it never shows the command, since it can appear on a locked screen.
Turn binding. The preload exposes desktopExecutor.getDevice(), so the composer from feat(desktop): bind turns to a desktop and enforce its deadlines from the row #8650 binds turns. It returns null while Sim reports the executor off.

Shared projections. The browser, terminal and local-filesystem result projections move from the web app into @sim/desktop-bridge (tool-results, local-filesystem-tools). The chat view and the executor now build identical model-facing results. The renderer behaviour is unchanged; its existing tests pass as they are. The one code path removed, a REJECTED terminal code mapped to cancelled, was dead: nothing produces that code.

Not in this PR

import_local_files in the background is answered as "not run" here; feat(desktop): import local files from the background executor #8672 adds the upload route and the runner.
The web chat view going display-only for bound runs, sidebar status and turn-complete notifications are in the companion web PR. That PR must land before the flag is turned on.
Prevent-sleep and browser takeover are in feat(desktop): keep the machine awake for background chats and yield the page to the user #8671.

Test plan

Unit tests, written first and red before the code existed:
- the executor state machine: claim, queue order, renewal while queued and running, delivery retry, 413 compaction, superseded and duplicate results, Stop of a queued call and of a running call, 410 revocation, crash recovery, 401 re-registration, the claim cap, pause on suspend, and write-ahead of started;
- the encrypted journal;
- the doorbell: rotation, reconnect, staleness and unregistered;
- approval notifications;
- terminal cancel and its escalation;
- Stop of input and kill;
- sign-out stopping the agent's running command and an in-flight tmux run, and leaving a user-started command alone;
- the agent's command staying the agent's after an interactive detach.
Guards were checked by mutation: write-ahead off fails its test, and terminal cancel or journal recovery off fails E2E scenarios E and C.
Packaged-app E2E (apps/desktop/e2e/background-executor.spec.ts), run with Playwright _electron against a fixture Sim that speaks the device protocol. The JSON report goes to BACKGROUND_EXECUTOR_REPORT_PATH, wired in desktop-e2e.yml.
- A. Chat A drives the browser (open, then 10 clicks) while chat B runs three terminal commands and a read_local_file. Meanwhile the window navigates to chat C, then to another workspace, then fully reloads. Every call completes exactly once with its own token. The page sees exactly 10 clicks, and the commands run once each, in order. Chat B cannot see chat A's tabs. Nothing touches /api/copilot/confirm or /authorize. p95 pickup is under 1.5 s.
- B. A result produced while Sim is unreachable is delivered once after reconnecting.
- C. A crash in the middle of a command is reported as outcome unknown after relaunch, and the command runs once.
- E. Stop from elsewhere ends a running browser_wait_for and a sleep command. The process is gone, and both results are acknowledged as superseded.
- F. An approval in a background chat raises one notification without the command. Nothing is claimed before approval, and the call is claimed within 1.5 s of approval.
- G. Of two installs, only the bound device claims the call.
- H. A device Sim has not enabled returns no device to the composer; an enabled one returns its device.
All 7 scenarios pass. With the executor not started, all 7 fail.
Full gate on the rebased branch: lint:check, type-check, check:audits, test, docs-manifest:check and the block registry check. Under heavy machine load, a few unrelated timing-bound tests (the route inventory, markdown perf) time out; each passes when run on its own.

waleedlatif1 · 2026-10-06T06:06:53Z

@greptile

waleedlatif1 · 2026-10-06T06:06:54Z

@cubic-dev-ai review this PR

vercel · 2026-10-06T06:06:55Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Oct 6, 2026 7:31pm UTC

cubic-dev-ai · 2026-10-06T06:06:59Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

greptile-apps · 2026-10-06T06:13:49Z

Confidence Score: 5/5

[High risk] Adds background execution system for desktop chat tools.

The PR appears safe to merge based on this review; no new actionable issue or outstanding previous finding remains.

Summary

The PR adds an Electron-main background executor for device-bound desktop tool calls.

It registers a device, reconciles offered calls, runs them in chat-scoped surfaces, and journals results for recovery and delivery.
It adds cancellation and approval notifications, shares tool-result projections with the web app, and adds unit and packaged-app coverage.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Sim["Sim device protocol"] --> Inbox["Doorbell and inbox reconciliation"]
  Inbox --> Executor["Claim and queue per chat/surface"]
  Executor --> Tools["Browser, terminal, and local-file tools"]
  Executor --> Journal["Encrypted journal"]
  Tools --> Journal
  Journal --> Sim

_{Reviews (8) · Last reviewed commit: "test(desktop): drive the doorbell backof..."}

cubic-dev-ai

All reported issues were addressed across 38 files

_{Re-trigger cubic}

waleedlatif1 · 2026-10-06T06:35:27Z

@greptile

waleedlatif1 · 2026-10-06T06:35:28Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T06:35:33Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 99 files

_{Re-trigger cubic}

waleedlatif1 · 2026-10-06T15:40:19Z

@greptile

waleedlatif1 · 2026-10-06T15:40:21Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T15:40:27Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

waleedlatif1 · 2026-10-06T15:44:06Z

@greptile

waleedlatif1 · 2026-10-06T15:44:08Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T15:44:14Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 45 files

_{Turn on auto-fix | Re-trigger cubic}

…or its successor dispose() now reports idle once. A recovered delivery that settles after sign-out no longer reports through the shared busy callback, where it could release the sleep blocker the next session's executor was holding.

…en Terminal is switched off - A tmux run is tracked as soon as its window exists, not once its wait ends. Sign-out now reaches a command the chat view started that is still inside its wait window. Reaping skips runs whose call is still reading their files. - Switching Terminal off stops the agent's commands before the shells go, as sign-out does. - The runner's deadline is built on the shared interruptible sleep. - A hostname longer than Sim's 128-character limit is shortened, so registration does not fail on it.

waleedlatif1 · 2026-10-06T18:05:58Z

@greptile

waleedlatif1 · 2026-10-06T18:05:59Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T18:06:05Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 47 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

…ach other - Sign-out waits for a restart's journal walk before clearing the journal, and a walk or delivery that starts after sign-out sends nothing, so none of the previous session's results outlive it. - An unrecognized device's stopped loops stay stopped: the reconcile timer no longer re-arms itself after them. - Switching Terminal off disposes the shells only if it is still off once the agent's commands have stopped. - A tmux run whose terminal closes mid-wait keeps its output files until its call has read them. - A terminal operation that outlives its deadline is reported as outcome unknown and not to be retried, as a browser action is.

waleedlatif1 · 2026-10-06T19:01:16Z

@greptile

waleedlatif1 · 2026-10-06T19:01:18Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T19:01:23Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

1 issue found across 47 files

Confidence score: 3/5

In apps/desktop/src/main/terminal/index.ts, a run stopped through escalation is reported as completed with no exit code, so the agent can mistake a force-killed command for success. Preserve a non-success status for these runs.

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/desktop/src/main/terminal/index.ts">

<violation number="1" location="apps/desktop/src/main/terminal/index.ts:1292">
P2: A `run` stopped through the escalation path is reported to the agent as `status: 'completed'` with `exitCode: null`, which makes a force-killed command indistinguishable from a successful completion. When `interruptTmuxRun` has to `closeRunWindow` (the command ignored Ctrl-C), the run's status file is never written, so `pollRun` returns `done: false` and the stop path forces `done: true`, which `runInTmux` turns into `status: 'completed'`. That result is what the executor records and the model reasons from, so the agent can conclude an escalated kill finished normally. Track that the outcome came from the stop and report it distinctly (for example `status: 'stopped'` with `exitCode: null`) instead of completing the run's normal-done semantics.</violation>
</file>

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Turn on auto-fix | Re-trigger cubic}

… assertions in tests When the app has no usable account storage (signing out, switching account, storage unavailable), a local file call now says so. It no longer tells the model a setting is off and to ask the user to switch it on.

waleedlatif1 · 2026-10-06T19:24:43Z

@greptile

waleedlatif1 · 2026-10-06T19:24:45Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T19:24:51Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 47 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 · 2026-10-06T19:31:58Z

@greptile

waleedlatif1 · 2026-10-06T19:31:59Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-06T19:32:07Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 47 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 requested a review from a team as a code owner October 6, 2026 06:06

vercel Bot deployed to Preview October 6, 2026 06:08 View deployment

waleedlatif1 force-pushed the feat/desktop-executor-binding branch from a31aaa8 to 72a5ba3 Compare October 6, 2026 06:11

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview October 6, 2026 06:33 Inactive

This was referenced Oct 6, 2026

feat(desktop): show background desktop chats and keep the chat view display-only for them #8670

Open

feat(desktop): keep the machine awake for background chats and yield the page to the user #8671

Open

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/desktop-executor/journal.ts Outdated

Comment thread apps/desktop/src/main/desktop-executor/approval-notifier.ts

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

waleedlatif1 force-pushed the feat/desktop-executor-main branch from 8c2b3fd to de04c40 Compare October 6, 2026 15:39

waleedlatif1 mentioned this pull request Oct 6, 2026

feat(desktop): import local files from the background executor #8672

Open

3 tasks done

vercel Bot deployed to Preview October 6, 2026 15:41 View deployment

waleedlatif1 force-pushed the feat/desktop-executor-binding branch from 48920e2 to 0a5f255 Compare October 6, 2026 15:52

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/local-filesystem.ts Outdated

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

waleedlatif1 added 2 commits October 6, 2026 10:48

waleedlatif1 changed the base branch from feat/desktop-executor-binding to staging October 6, 2026 18:05

waleedlatif1 force-pushed the feat/desktop-executor-main branch from f6d9127 to 6cf16e3 Compare October 6, 2026 18:05

vercel Bot deployed to Preview October 6, 2026 18:08 View deployment

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/index.ts Outdated

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview October 6, 2026 19:00 Inactive

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/desktop-executor/executor.test.ts Outdated

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/terminal/index.ts

Comment thread apps/desktop/src/main/desktop-executor/runner.ts Outdated

vercel Bot temporarily deployed to Preview October 6, 2026 19:24 Inactive

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/desktop-executor/doorbell.test.ts Outdated

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

test(desktop): drive the doorbell backoff test with fake timers

93df205

vercel Bot temporarily deployed to Preview October 6, 2026 19:31 Inactive

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

Conversation

waleedlatif1 commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

vercel Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot commented Oct 6, 2026

Uh oh!

greptile-apps Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 6, 2026

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 6, 2026

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

waleedlatif1 commented Oct 6, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 6, 2026

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Oct 6, 2026 •

edited

Loading

vercel Bot commented Oct 6, 2026 •

edited

Loading

greptile-apps Bot commented Oct 6, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading