Skip to content

fix(db): bootstrap Projects for development push databases that predate them - #9611

Merged
waleedlatif1 merged 3 commits into
stagingfrom
fix/dev-push-project-bootstrap
Nov 3, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
fix/dev-push-project-bootstrap

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Dev schema push (SIM_DEV_DB_PUSH=1 db:push) refused any database that has workspaces but no Project schema, so a push-managed database from before Projects could never sync: push can't backfill workspace.project_id and the guard refuses every unenforced Project state
  • In push mode, reconcile-project-membership.ts --prepare now expands those databases: it replays 0394_project_foundation + 0406_workspace_project_column verbatim (minus 0406's leading migrator batch COMMIT, so the foundation commits atomically with the rollout marker)
  • push.ts then sees the rollout marker on its fence session and runs one Drizzle push with workspace.project_id still nullable (SIM_DB_PUSH_PROJECT_BOOTSTRAP=1, read only by schema.ts for that column), so an older database gains every column 0032 reads (e.g. the fork parent). reconcile-project-membership.ts --bootstrap then runs the registered 0032_project_membership through runScriptMigrations on its single session (recorded exactly as migrate.ts records it), and the ordinary push and reconcilers follow. drizzle.config.ts keeps the connector and rollout marker out of Drizzle's diff until 0032 retires them
  • Runs inside the existing push fence: the fence's transaction-level advisory lock on the migration key keeps migrate.ts and other pushes out; the fence holds no table locks or snapshot, so 0406's concurrent index build and 0032's workspace barrier don't wait on it
  • Resume is narrow: a rollout marker that the drizzle journal doesn't account for (push never creates it, versioned 0406 journals it) with an empty connector. Crash after the expansion, Project conflicts reported by discovery, or a failed enforcement phase all resume on the next push
  • A fresh bootstrap requires no Project table, connector, marker, or project_id column on any public table. Every other state behaves as today: connector with memberships, connector or Projects without the column expansion, a nullable column without Projects, a versioned expansion awaiting 0032, and any push without SIM_DEV_DB_PUSH=1 still refuse with the same message. migrate.ts is untouched
  • drizzle-kit patch: push added PostgreSQL unique constraints after the foreign keys of the same diff, so the composite folder FK on its new (id, project_id, resource_type) unique always failed on an existing table. Unique constraints now come right after new columns. In the noninteractive push policy, the "truncate before adding a unique constraint?" prompt (which aborts without a TTY) is answered with its non-destructive choice; duplicates still fail the push

Type of Change

  • Bug fix

Testing

  • New packages/db/scripts/project-push-bootstrap.integration.ts: pushes the real schema, strips every Project object to build a pre-Project template, then per test runs the real scripts/push.ts --force (fence included). It covers bootstrap + push + replay, a database missing a column 0032 reads (fork parent dropped), (memberships, fork family, archived Project, validated/deferrable constraints, connector and marker retired, 0032 recorded), resume after a discovery conflict, atomic rollback when the workspace barrier is busy, a migration holding the lock, no-op without SIM_DEV_DB_PUSH, and seven refusal states (incl. orphaned folder Project ownership) that must keep schema, workspaces and connector byte-identical
  • Red/green: the bootstrap tests fail on the pre-fix reconciler with the refusal message. Reverting each guard separately (journal check, connector-populated check, batch-commit atomicity, push-mode gate, strict fresh shape, strict resume shape, the nullable first push) turns its named test red
  • New push.integration.ts case for a composite FK on a new unique constraint of a populated table; it goes red with either drizzle hunk removed
  • Also checked locally: databases pushed from the last pre-Project schema.ts and from the last pre-forking schema.ts, seeded with workspaces, then pushed with the current code: both bootstrap and replay. The pre-forking one first hits an unrelated Drizzle diff error dropping a webhook FK that never existed (also on an empty copy, before this change); the rerun resumes and completes
  • scripts/push.integration.ts, project-contract, project-authority, workspace-project-column integration suites (120 tests) pass on PostgreSQL 17; push.test.ts; packages/db type-check; lint; check:utils, check:comment-hygiene, check:test-patterns, check:file-names

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…te them

A push-managed database with workspaces but no Project schema could never pass
the schema-push guard: push cannot backfill workspace.project_id, and the
guard refuses every unenforced Project state. Under SIM_DEV_DB_PUSH=1 the
pre-push reconciler now replays migrations 0394 and 0406 (the foundation
commits atomically with the rollout marker) and runs the registered
0032_project_membership script migration on its single session, recording it
as the migration runner does. An interrupted bootstrap resumes; every other
unenforced state still refuses with the same message.

The pinned drizzle-kit patch also adds unique constraints before foreign keys,
so a composite foreign key on a new unique constraint of an existing table no
longer fails, and in the noninteractive push policy it answers the
unique-constraint truncate prompt with its non-destructive choice.
@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 3, 2026 4:56am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Enables development schema pushes for databases that predate Projects.

  • Development pushes can bring pre-Project databases up to the current schema.
  • Drizzle pushes add unique constraints before the foreign keys that rely on them.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Acquire migration fence] --> B[Check Project state]
  B --> C[Expand eligible development database]
  C --> D[Push schema with nullable Project column]
  D --> E[Run Project membership migration]
  E --> F[Push required schema]
  F --> G[Run reconcilers]
Loading

Reviews (3) · Last reviewed commit: "fix(db): pin the Project bootstrap to th..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files

Confidence score: 3/5

  • In packages/db/scripts/reconcile-project-membership.ts, classifying the database as fresh while the Project-specific columns remain can let bootstrap reach a Drizzle foreign-key push that fails on orphaned values. Account for those columns before treating the database as fresh.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/db/scripts/reconcile-project-membership.ts">

<violation number="1" location="packages/db/scripts/reconcile-project-membership.ts:49">
P2: This classifies a database as `fresh` even when Project-specific `folder.project_id` or `workspace_files.project_id` columns remain. Orphaned values can make Drizzle’s later foreign-key push fail after bootstrap has committed the Project migration. Include these columns in the schema-state check and refuse this shape.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

The bootstrap ran 0032_project_membership before Drizzle pushed anything, so a
development database from before a column the migration reads (such as the
fork parent) failed after committing the expansion, on every retry.

--prepare now only replays the expansion. push.ts then runs one Drizzle push
with workspace.project_id still nullable (SIM_DB_PUSH_PROJECT_BOOTSTRAP=1),
runs the migration through reconcile-project-membership.ts --bootstrap, and
finishes with the ordinary push. The connector and rollout marker are kept out
of Drizzle's diff until the migration retires them. A fresh bootstrap also
requires that no public table carries a project_id column.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

The replayed migration SQL and the script runner name tables unqualified, so a
connection search path that lists another schema first created the Project
foundation outside public. The refusal snapshot now also covers folders and
Projects.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit e137d82 into staging Nov 3, 2026
81 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/dev-push-project-bootstrap branch November 3, 2026 05:52

This branch was previously deployed

1 inactive deployment
Preview — f51be95c Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant