test(promotion): pin the exact files promoting an assistant writes to the target

[scott-lowe-vapi]

Value

V.A.L.U.E. tier: small — a tests-only baseline under the remaining promotion changes for TEST-141 (TEST-141). It changes no behaviour. It's small rather than micro because it guards a blast-radius path: promotion into production.

• Problem: the code that moves assistants from a lower org into production is the riskiest path in gitops. The existing tests check pieces of it (canonicalization, bindings, deletions) but never the full set of files a promotion writes. So a refactor could change what lands in prod without any test failing.

• Who it affects: teams that promote resources between orgs (dev → staging → prod). It's a baseline for future changes to promotion's plan/apply core. The stacked PRs don't touch that core: fix(promotion): commit the files of transitions that applied when a later one fails #65's commit behaviour is covered by its own promote-cmd and workflow-commit tests, and feat(promotion): gate promotion out of an org on a passing check #66 only adds the check field.

• What changes: new tests/promotion-golden.test.ts. It runs one staging → prod promotion through promotionPlanBuild and promotionPlanApply, then asserts the plan and every file left in the target org:

  • source UUIDs canonicalized to names (toolIds, a handoff destination);
  • referenced tools and assistants pulled in as dependencies, and an unreferenced staging-only tool left behind;
  • a credential bound by name, and a phone number bound to the target's;
  • a target file the source no longer has deleted;
  • an unrelated target file left untouched;
  • the markdown prompt body preserved;
  • the source org never written.

  A second test re-plans right after the apply and expects no changes.

Evidence of value

Mutation check: each deliberate break of src/promotion.ts turns the test red, and it's green again once the break is reverted.

Break	Golden test
Phone binding returns the source phone	❌ fails
Credential binding returns the source UUID	❌ fails
UUID → name canonicalization disabled	❌ fails
None (main)	✅ passes

The expected output was read line by line against promotion's documented behaviour before it was pinned.

Tests: npm test goes from 474 to 476 passing.

Testing plan

• tests/promotion-golden.test.ts: the golden promotion, and idempotence on re-plan.
• Not tested:
  • the child pull/apply processes (this exercises the file-writing core, not the CLI);
  • a real two-org promotion (manual QA before fix(promotion): commit the files of transitions that applied when a later one fails #65 and feat(promotion): gate promotion out of an org on a passing check #66 merge).

Base of the promotion stack: #68 ← #65 ← #66.

Refs TEST-141

After review

Chris's mutation-tested suggestions, each verified to fail the golden when the code it guards is broken:

• .vapi-ignored prod files are left alone (an ignored, prod-only assistants/hotline.yml).
• The whole repository is snapshotted, so a stray write to .env.* or a state file fails it.
• Billing arrives only as a dependency of a dependency (intake → handoff-to-billing → billing).
• A file promotion doesn't rewrite is copied byte for byte (it keeps a comment).
• The prompt body has a heading, blank lines and a --- separator.

Not taken: the omit phone case (promotion.test.ts already covers it), the unused state entries, and the input/output formatting pass, which is a follow-up.

🤖 Generated with Claude Code

[scott-lowe-vapi]

• feat: identify every gitops API request with a User-Agent #78
• feat(validate): catch broken references and show findings on the PR #77
• ci: validate every org's resources on every pull request #76
• ci: lint workflows and test the PR check's key-sharing decision #75
• test(docs): check that doc links, commands and flags exist #74
• fix(cleanup): never delete resources matched by .vapi-ignore #73
• docs(agents): one compact guide for Claude, Codex and Cursor #72
• docs: edit the guides for new readers and fix inaccuracies #71
• docs: restructure the README into a short landing page and topic guides #70
• docs: prepare for public release: scrub internal names, fix examples, add community files #69
• feat(promotion): gate promotion out of an org on a passing check #66
• fix(promotion): commit the files of transitions that applied when a later one fails #65
• test(promotion): pin the exact files promoting an assistant writes to the target #68 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[chris-garber-vapi]

This seems like a fine test and very low risk. If we wanted to expand it much further it might make sense to break the file fixtures into real files -- but I think this is fine for now

[dhruva-vapi]

lgtm, nice to have the whole written output of a promotion pinned in one place

[chris-garber-vapi]

Mutation-tested review of the golden. Both tests pass on main. I then applied realistic breaks to src/promotion.ts and checked whether anything catches them:

Break in src/promotion.ts	Caught by this golden?	Caught by any promotion test?
Delete .vapi-ignored target files	❌ no	❌ no
Always re-render, never copy verbatim	❌ no	❌ no
Don't collect dependencies of dependencies	❌ no	❌ no
Don't treat assistantId as a dependency	❌ no	❌ no
promotionPlanApply also writes .env.staging / .env.prod / a state file	❌ no (test 1)	—
Unbound staging phones bound or passed through	❌ no	✅ yes

Each inline comment has a fix that I checked against main. With all of them applied, every break above fails the golden, and the changes list it expects doesn't change.

[scott-lowe-vapi]

Merge activity

• Oct 7, 5:47 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Oct 7, 5:47 PM UTC: @scott-lowe-vapi merged this pull request with Graphite.