Repository navigation
Conversation
An update made of operators alone, such as a counter increment, read the row with SELECT ... FOR UPDATE and wrote every column back. Concurrent increments of a hot row queued behind the whole of each other's transaction: the locked read, validation, the write, the re-read of the operators' results and the cache purge. Such an update now reads the row without the lock, writes only its operator columns and $updatedAt (so nothing read without the lock is written back), and re-reads the results after commit. The UPDATE itself still takes the row lock, so increments stay atomic. It applies only when every changed attribute is an operator on a non-relationship attribute, $id and $permissions are absent or unchanged, no request timestamp is set, and the adapter locks reads for update. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
A bound such as increment(1, max) is validated against the stored value, which the fast path reads without the lock, so a concurrent change could slip between the check and the write. Only unbounded increments and decrements now skip the locked read. They only apply to numeric attributes, so the relationship check goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
updateDocument()reads the row withSELECT … FOR UPDATE, merges the change into it and writes every column back. An increment of a hot row therefore waits for the whole of the previous increment's transaction: the locked read, validation, the write, the re-read of the operators' results and the cache purge.On Appwrite Cloud a project counting SendGrid opens sent ~430
Operator.increment()PATCHes to two rows in 25 s. They took 7–12 s each, and the lock waiters slowed other tenants on the same MySQL host.Change
When an update is only unbounded increments or decrements,
updateDocument()now:$updatedAt, through the adapter's sparseupdateDocuments()(SET opens = opens + 1, _updatedAt = ? WHERE _id = ?), so nothing read without the lock is written back;The
UPDATEstill takes the row lock, so increments stay atomic and none are lost.It applies only when all of these hold; otherwise the existing locked path runs unchanged:
incrementordecrementwith nomax/minbound (a bound is checked against the stored value, so bounded and all other operators keep the locked read);$idand$permissionsare absent or unchanged (Appwrite always sends both; permissions are compared after the read);Capability::UpdateLock, so Mongo, Redis and Memory are untouched;Behaviour changes
increment(1, max),decrement(1, min)) stay on the locked path, because their bound is checked against the stored value.$updatedAtcomes from the unlocked read, so under contention it isn't strictly monotonic across concurrent increments.Evidence
New E2E test
MySQLTest::testOperatorOnlyUpdateLeavesConcurrentWriteToAnotherColumn: a second connection writes another column between the update's read and its write.main: fails with1205 Lock wait timeout exceeded, because the locked read blocks the other writer.MySQLTest::testBoundedIncrementKeepsTheRowLockedFromReadToWrite: the same interleaving withincrement(1, 10)must still block the other writer (1205), and the storedstatusstaysbefore. This test has not run yet: the local Docker daemon broke before it could. CI is its first run, and nobody has seen it go red.Hot-row benchmark: 16 processes × 50
increment(1)on one row, plus 2 processes writing another column of the same row. Fresh MySQL 8.4 and Redis for each run, alternatingmainand this branch. Run on a laptop, so the numbers are noisy.mainincrements/smain→ branchEvery run ended with
opensexactly equal to the number of increments and the other column holding the last value written, so no updates were lost.Tests
MySQLTestpasses (1270 tests, 13 skipped) andSharedTables/MySQLTestpasses (1269 tests, 12 skipped).MariaDBTestwas cut short when the local Docker daemon died, so CI has to cover it. Postgres and SQLite weren't run locally.EvictionProtocolTestcases that need the composeredishost. CI should run those, because this change touches the cache-purge ordering.Draft: a prototype for discussion.
🤖 Generated with Claude Code