Repository navigation
fix(migrations): wait for a 0305 index build to commit before judging it invalid - #16981
Conversation
… it invalid Postgres ends a concurrent build's pg_stat_progress_create_index row just before the build's last transaction (the one that sets indisvalid) commits. In that window 0305 saw no build in progress, read indisvalid as false and dropped the index, either rebuilding it or failing with "tuple concurrently updated". A build now counts as in progress while its pg_index row has an uncommitted update (xmax held by an open transaction in pg_locks). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143hYTDGiy48mntt7T9N2uT
Adversarial review: full tier, OpusTier: CI read, not re-run: every check green, including Rest Framework Unit Tests (both), User Interface Tests, k8s deploy, Migration Graph Check, ruff, Unit Tests Complete. The CI unit-test database user is the compose Regression test, failing first (Django test runner in the Pro dojo container, against a private PostgreSQL 16.16 container with a superuser so the new test does not skip):
Read: the docstring's claim about the order of events checks out against the observed catalog state (below): the progress row is gone, Break attempts
Performance: one No screenshots: no user-visible surface. Findings
Recommendationchanges required: 1 Not checked: a production-scale table (the window is independent of size, the waits were measured on 200k rows); PostgreSQL 17 (16 and 18 behave identically here). 🤖 Generated with Claude Code |
|
Resolution of the adversarial review (verdict 1, head ab2895a):
|
Description
AddIndexConcurrentlyIfMissingin migration0305_archive_batch_indexeswaits whilepg_stat_progress_create_indexshows an earlier build of its index, then readspg_index.indisvalidand drops and rebuilds the index when it is not valid. Postgres ends a concurrent build's progress row just before the build's last transaction commits, and that transaction is the one that setsindisvalid. In that window the wait sees no build,indisvalidstill reads false, and the migration drops an index that was about to become valid. Two outcomes were measured:dojo_finding);DROP INDEX CONCURRENTLYcollides with the build's uncommittedpg_indexupdate and fails withtuple concurrently updated, so the migration errors out.Fix: a build now counts as in progress until its transaction commits. Besides the progress row,
_build_in_progressalso checks whether the index'spg_indexrow has an uncommitted update: itsxmaxnames a transaction that still holds the lock on its own id inpg_locks(every transaction holds that lock until it ends, andpg_locksis readable without extra privileges). The existing wait loop then covers the window. Migration state is unchanged.Test results
New test
test_build_ending_is_waited_for_until_it_commitsinunittests/test_migration_0305_archive_batch_indexes.py. It holds a realCREATE INDEX CONCURRENTLYin the window with no timing: addl_command_endevent trigger (it runs after the progress row ends and before the commit) waits on an advisory lock the test holds. The test checks that state directly (no progress row,indisvalidfalse), then runs the operation and releases the lock from the operation's first sleep. ADROP INDEXis refused with an assertion instead of being run, so a regression fails fast instead of hanging. Creating an event trigger needs a superuser, which the unit-test compose database is; the test skips otherwise, astest_bulk_risk_acceptance_apialready does.Measured on PostgreSQL 16.9. The adversarial review also held the build in the commit window on PostgreSQL 18.6 (the compose image) and 16.16: the old check reports no build, the new one reports it in progress.
0305fromdevdropped an index whose build had not committed yet: DROP INDEX CONCURRENTLY IF EXISTS "dojo_test_archive_batch"tuple concurrently updatedtest_migration_0305_archive_batch_indexesmodule as a superuser, 10 runsmakemigrations dojo --check --dry-run: No changes detected.0305with this change, as a superuser and as a non-superuser: OK (pg_locksneeds no extra privilege).ruff check(0.16.10) on the test file: clean (dojo/db_migrationsis excluded from ruff).Related: migration
0307_event_history_lookup_indexes, already ondev, has its own copy of this wait (_build_in_progresschecks onlypg_stat_progress_create_index), so it can drop an index whose build has not committed yet in the same way; it is tracked separately and is not changed here. #16971 also copies this class into other concurrent-index migrations; those copies carry the same wait and would need the same check.Documentation
None needed: no user-facing change.
🤖 Generated with Claude Code
https://claude.ai/code/session_0143hYTDGiy48mntt7T9N2uT