Skip to content

Update a document of operators only without a locked read - #1073

Draft
loks0n wants to merge 2 commits into
mainfrom
feat/operator-only-update
Draft

loks0n wants to merge 2 commits into
mainfrom
feat/operator-only-update

Conversation

@loks0n

@loks0n loks0n commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem

updateDocument() reads the row with SELECT … FOR UPDATE, merges the change into it and writes every column back. An increment of a hot row therefore waits for the whole of the previous increment's transaction: the locked read, validation, the write, the re-read of the operators' results and the cache purge.

On Appwrite Cloud a project counting SendGrid opens sent ~430 Operator.increment() PATCHes to two rows in 25 s. They took 7–12 s each, and the lock waiters slowed other tenants on the same MySQL host.

Change

When an update is only unbounded increments or decrements, updateDocument() now:

  1. reads the row without the lock;
  2. writes only the operator columns and $updatedAt, through the adapter's sparse updateDocuments() (SET opens = opens + 1, _updatedAt = ? WHERE _id = ?), so nothing read without the lock is written back;
  3. re-reads the operators' results after commit, outside the lock.

The UPDATE still takes the row lock, so increments stay atomic and none are lost.

It applies only when all of these hold; otherwise the existing locked path runs unchanged:

  • every changed attribute is increment or decrement with no max/min bound (a bound is checked against the stored value, so bounded and all other operators keep the locked read);
  • $id and $permissions are absent or unchanged (Appwrite always sends both; permissions are compared after the read);
  • no request timestamp is set (that conflict check needs the locked read);
  • the adapter supports Capability::UpdateLock, so Mongo, Redis and Memory are untouched;
  • the collection isn't metadata.

Behaviour changes

  • The response is read after commit, so a returned counter can include increments that committed just after this one. It never misses this update's own.
  • Bounded operators (increment(1, max), decrement(1, min)) stay on the locked path, because their bound is checked against the stored value.
  • $updatedAt comes from the unlocked read, so under contention it isn't strictly monotonic across concurrent increments.

Evidence

New E2E test MySQLTest::testOperatorOnlyUpdateLeavesConcurrentWriteToAnotherColumn: a second connection writes another column between the update's read and its write.

  • main: fails with 1205 Lock wait timeout exceeded, because the locked read blocks the other writer.
  • This branch: passes; the concurrent write lands and survives, and the counter is incremented.

MySQLTest::testBoundedIncrementKeepsTheRowLockedFromReadToWrite: the same interleaving with increment(1, 10) must still block the other writer (1205), and the stored status stays before. This test has not run yet: the local Docker daemon broke before it could. CI is its first run, and nobody has seen it go red.

Hot-row benchmark: 16 processes × 50 increment(1) on one row, plus 2 processes writing another column of the same row. Fresh MySQL 8.4 and Redis for each run, alternating main and this branch. Run on a laptop, so the numbers are noisy.

Network main increments/s this branch increments/s Lock wait per waiter, main → branch
local (~0.1 ms) 183 / 241 / 198 256 / 201 / 283 59–78 ms → 39–57 ms
+netem, ~3 ms MySQL RTT 54 / 46 / 31 92 / 94 / 49 263–467 ms → 112–232 ms

Every run ended with opens exactly equal to the number of increments and the other column holding the last value written, so no updates were lost.

Tests

  • Full E2E: MySQLTest passes (1270 tests, 13 skipped) and SharedTables/MySQLTest passes (1269 tests, 12 skipped).
  • MariaDBTest was cut short when the local Docker daemon died, so CI has to cover it. Postgres and SQLite weren't run locally.
  • Unit suite: the only failures are the 4 EvictionProtocolTest cases that need the compose redis host. CI should run those, because this change touches the cache-purge ordering.
  • Pint and PHPStan pass on the changed files.

Draft: a prototype for discussion.

🤖 Generated with Claude Code

An update made of operators alone, such as a counter increment, read the
row with SELECT ... FOR UPDATE and wrote every column back. Concurrent
increments of a hot row queued behind the whole of each other's
transaction: the locked read, validation, the write, the re-read of the
operators' results and the cache purge.

Such an update now reads the row without the lock, writes only its
operator columns and $updatedAt (so nothing read without the lock is
written back), and re-reads the results after commit. The UPDATE itself
still takes the row lock, so increments stay atomic.

It applies only when every changed attribute is an operator on a
non-relationship attribute, $id and $permissions are absent or unchanged,
no request timestamp is set, and the adapter locks reads for update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A bound such as increment(1, max) is validated against the stored value,
which the fast path reads without the lock, so a concurrent change could
slip between the check and the write. Only unbounded increments and
decrements now skip the locked read. They only apply to numeric
attributes, so the relationship check goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant