feat(projects): enforce Project membership and retire the connector by mzxchandra · Pull Request #8590 · simstudioai/sim

mzxchandra · 2026-10-02T22:32:41Z

Summary

Complete the two-release migration to workspace.project_id. feat(projects): move Project membership to the workspace column #8830 supplies compatible column-based readers/writers; this PR enforces membership and retires project_workspace.
Prepare legacy data with apps/sim/scripts/backfill-projects.ts: read-only plan, reviewed archive repair, bounded apply, verify, and status. Persist private manifests/reports for resume; preserve existing assignments and stop on ambiguous or changed families.
Start at 50 independent workspaces per transaction, keeping each fork family atomic. Use nonblocking writer-compatible locks, NOWAIT row locks, transaction/statement limits, adaptive batch reduction, bounded retries, committed progress, and graceful interruption.
Repair archived environments through the shared workflow/archive cascade and require external provider cleanup to succeed. Replays retain the original workflow identities so cleanup can finish after a committed archive. Busy locks and transient database failures defer that environment while unrelated repairs continue; permanent/provider failures still fail the run.
Support PG16 statement/idle-transaction limits and PG17+ total transaction limits; CI runs preparation and enforcement on both versions.
Prevent db:push from contracting existing Project databases; retained data must complete the reviewed migration path. Fresh schema push installs the same permanent lifecycle enforcement as SQL migrations.
Require verification artifacts and a conflict-free reviewed plan, report fully assigned split families as conflicts, wait for every bounded cleanup worker before reporting a strict failure, and invalidate MCP caches/notifications on failure. Validate artifact byte limits before publishing files. Every database-backed command coordinates with enforcement through the same nonblocking maintenance lock. Read-only commands honor cancellation; offline status stays available and its exit code reflects the saved outcome.
SQL 0405 trigger locks reject contention with retryable 55P03 before waiting with a row lock; application operations retain their existing mutex order.
SQL 0405 validates prepared data, enforces membership/lifecycle constraints, and atomically removes the temporary bridge and connector. Deployment preflight rejects incomplete preparation; the CLI and SQL migration exclude simultaneous execution.

This is release 2 of 2. Fully deploy #8830 to production and verify that pre-#8830 application tasks and relevant background jobs have drained before merging this PR into staging. Run the reviewed preparation tool separately for each target database before its #8590 migration. Run verify with both the manifest and report to check pending external cleanup as well as database invariants. Keep the artifacts in durable private job storage. #8830 is the oldest supported application rollback release after enforcement and connector removal; retain the contracted schema during rollback.

Type of Change

New feature

Testing

All 170 PostgreSQL integration checks passed on PG17, plus 69 Project preparation/contract/bridge and schema-push checks on PG16. These cover real CLI interruption/resume, lost checkpoints, batch rollback, stale plans, contention/timeouts, migration refusal/replay, and SQL/CLI exclusion.
All 31 Project/application repair checks passed on the migrated schema; all 27 Project application checks passed on the schema-push database. The full-schema repair test uses real Redis and held HTTP notifications to prove all cleanup workers settle before failure, cache invalidation still occurs, and replay completes the committed archive. Three additional cases hold workspace, Project and row locks, prove unrelated repairs complete, and resume the deferred target after release.
New regressions fail on their prior implementations: direct-write lock contention, artifact-free verification, split-family discovery, strict cleanup completion, oversized plan publication, read-only scan cancellation and maintenance-lock exclusion, status exit codes, and verification of previously conflicted plans. Removing the initial SQL archive precheck also makes its regression fail. All guards were restored before the passing runs.
19 live HTTP checks / 31 requests passed with the compatible app running before and after connector removal. A held workflow-create transaction made backfill defer while an unrelated workflow creation proceeded. A large synthetic full-schema benchmark measured about 43 ms local p95 for batches of 50; this is not a production latency guarantee.
Fresh schema-push provisioning installs lifecycle enforcement; populated legacy databases refuse unsafe schema push before DDL. The inherited billing concurrency regression passed 36 checks on both provisioning paths, and 36 deadline-related billing unit checks passed with stable clocks.
Full root tests, 26 workspace type checks, lint, 58 audits, workflow lint, generated-artifact checks, migration safety and Drizzle schema consistency passed. Block-registry validation passed against staging. Hosted validation and the separate desktop end-to-end run passed every applicable job on 4025e5a4b8.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Focused tests added/updated and passing (new tests pass the test-audit authoring gate)
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-10-02T22:32:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Oct 9, 2026 10:16am UTC

…ty-enforcement # Conflicts: # apps/sim/lib/projects/__integration__/foundation.integration.ts

…ty-enforcement

mzxchandra · 2026-10-09T08:59:35Z

@greptile

mzxchandra · 2026-10-09T08:59:36Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-09T08:59:42Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 150 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

mzxchandra · 2026-10-09T09:16:19Z

@greptile

mzxchandra · 2026-10-09T09:16:20Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-09T09:16:26Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 150 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

mzxchandra · 2026-10-09T09:30:09Z

@greptile

mzxchandra · 2026-10-09T09:30:10Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-09T09:30:18Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 150 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

mzxchandra · 2026-10-09T09:47:12Z

@greptile

mzxchandra · 2026-10-09T09:47:13Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-09T09:47:20Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 150 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

mzxchandra · 2026-10-09T10:10:17Z

@greptile

mzxchandra · 2026-10-09T10:10:19Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-09T10:10:25Z

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 150 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

mzxchandra added 7 commits October 2, 2026 00:31

feat(projects): add project identity and lifecycle foundation

00a83d0

feat(projects): create projects with their initial environment

aa9b415

docs(projects): record project files follow-up

91cba23

docs(projects): explain project and workspace creation flows

0980ad1

fix(projects): stage activation after compatible writers deploy

a14b448

feat(projects): enforce membership after the staged backfill

5121347

fix(projects): retry writes after concurrent membership changes

523338a

mzxchandra mentioned this pull request Oct 2, 2026

feat(projects): add project identity and lifecycle foundation #8580

Merged

6 tasks done

vercel Bot deployed to Preview October 2, 2026 22:34 View deployment

fix(projects): align integration fixtures with membership constraints

dc5e301

vercel Bot deployed to Preview October 2, 2026 23:01 View deployment

mzxchandra added 5 commits October 2, 2026 16:34

refactor(projects): prepare compatible writers for the SQL backfill

c1c8e5d

Merge branch 'feat/project-entity-foundation' into codex/project-enti…

2e94a13

…ty-enforcement # Conflicts: # apps/sim/lib/projects/__integration__/foundation.integration.ts

refactor(projects): backfill through bounded SQL migration batches

5e11d7d

fix(projects): clean up automatically created fixture Projects

99189f2

Merge branch 'feat/project-entity-foundation' into codex/project-enti…

78f773d

…ty-enforcement

vercel Bot deployed to Preview October 2, 2026 23:54 View deployment

fix(projects): backfill only families missing membership

2ecbfdf

vercel Bot deployed to Preview October 3, 2026 00:00 View deployment

mzxchandra changed the title ~~feat(projects): enforce membership after the staged backfill~~ feat(projects): backfill and enforce membership in SQL Oct 3, 2026

mzxchandra added 2 commits October 2, 2026 17:19

chore(projects): merge staging into foundation

30bc9d8

chore(projects): merge updated foundation into enforcement

b5cc3c4

vercel Bot deployed to Preview October 3, 2026 00:24 View deployment

fix(projects): bound the trigger installation lock window

2afc652

vercel Bot deployed to Preview October 3, 2026 01:06 View deployment

mzxchandra added 4 commits October 2, 2026 18:21

fix(workflows): guard restore against concurrent workspace archive

6dfbc4f

fix(projects): keep workflow lifecycle guards at workspace scope

8965210

fix(workflows): guard restore against concurrent workspace archive

552ce68

Merge branch 'feat/project-entity-foundation' into codex/project-enti…

8be18e5

…ty-enforcement

fix(projects): await cleanup and bound backfill artifacts

8c3c334