fix(stack-encrypt): check that a json_tree read returns the entry asked for by coderdan · Pull Request #1358 · cipherstash/stack

coderdan · 2026-10-24T08:06:51Z

Summary

EQL v4's json_tree column stores an encrypted JSON document as one entry per JSON node. When the database returns entries for a read (doc -> selector, or a subtree walk), the client decrypted each one under the selector and position the database sent back with it, and never checked that it was the entry it had asked for. Every entry of a row opens that way, so a dishonest database could answer a read of $.salary with the same row's $.bonus, or with the sealed root (the whole document), or splice other entries into a walk, and the client would hand back the wrong value. Now the client checks every entry against what it asked for and refuses anything else with a new error, Error::JsonWrongEntry.

Changes

Stack Encrypt (packages/stack-encrypt)

JsonSelector::open(node) -> JsonExtracted is now the only way to open the entry -> returned. It refuses an entry whose selector is not the one asked for, before any key is retrieved. The sealed root opens only when $ itself was asked for.
JsonSubtreeKey::open(rows) -> JsonSubtree now checks every row when it is opened. A node a gated walk reaches must carry the path's selector, at depth 0, and the position the path ends in. Every other row must carry the selector of its parent's path plus its own decrypted label. Each subtree's rows must be consecutive in the document, so a dropped, moved or orphaned row is refused. A reported depth d that disagrees is refused too.
JsonSubtreeKey now holds what it needs to derive those selectors: the keyset's index-key PRF (shared with the keyset handle) and the column's context. Neither is printed or serialized.
JsonNode no longer opens on its own: it is the raw row the database returned.
A new error variant, Error::JsonWrongEntry { expected }. It is not Aead (every entry is genuine) and not OpenedType (the entry is not of a wrong shape; it answers a different question).
JsonDocument, the stored form, is unchanged. So are the SQL and the wire format.

EQL bindings (packages/eql/crates/eql-bindings)

JsonTreeExtracted::from_selector(&selector, entry) is how an entry -> returned opens. JsonTreeEntry no longer opens on its own.
JsonTreeSubtree::from_key(&key, rows) is unchanged, and now checks the rows through the key.

Docs

The leakage page (packages/eql/docs/reference/json-tree-leakage.md, "What is authenticated") says what is checked now and what is not.
The design record's Phase 4 bullet is marked taken.
CHANGELOGs, plus a patch changeset for @cipherstash/eql (eql-bindings ships in lockstep with it).

Not covered, by design

Which row an entry comes from. Another row's $.salary, returned with that row's header, still opens. Nothing in an entry names its row, and two values sealed under the same context are interchangeable between rows (ADR 0010, packages/stack-encrypt/docs/adr/0010-what-a-stored-value-authenticates.md, and the "Known non-findings" in AGENTS.md).
An array position before the last step of a path. Every element of an array shares one selector, so a walk for $.a[1].b can be answered with $.a[0].b of the same row. -> already reads a position as the wildcard.
A walk with no gates through a chained array position ($.l[3], whose children share every other element's selectors). The walk does not return the node it starts from yet. json_tree: a subtree read of {}, [], a scalar or a missing field all return no rows, so the client cannot tell them apart #1356 is the issue; feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363, stacked on this PR through docs(plans): design json_tree tokens derivable from the path alone #1361, fixes it, and with it this case.

Verification

Run from the worktree root unless noted. PostgreSQL 17 was a disposable container on port 55437, stopped afterwards.

mise x --env test -- cargo nextest run -p stack-encrypt --all-features: 829 tests run: 829 passed, 1 skipped at the first head; 833 tests run: 833 passed, 1 skipped after the review fixes (9ec5c59f1, aa591f240, efc6bba23; see the review comments)
cargo test -p stack-encrypt --doc --all-features: test result: ok. 50 passed; 0 failed
cargo clippy -p stack-encrypt --all-features --all-targets -- -D warnings: clean
cargo fmt --all --check at the root and in packages/eql: clean
RUSTDOCFLAGS=-D warnings cargo doc -p stack-encrypt --all-features --no-deps: clean
packages/eql: cargo test -p eql-bindings --all-features: all pass (18 passed, 16 passed; 2 ignored, …)
packages/eql: mise run types:check: passes
packages/eql: mise run test:encryption: exit 0
packages/eql: mise run test:encryption:postgres: exit 0. The json_tree suite: test result: ok. 7 passed. That includes the new postgres_hostile_answers_are_refused_by_the_client, which gets real rows back from PostgreSQL. It answers -> with another path's entry and with the sealed root, and a gated walk with another position's subtree. It also tampers with a stored row twice: once by rewriting a top-level field's parent tag so the walk takes it for a member, and once by swapping two entries in sv. The client refuses each wrong entry as JsonWrongEntry and each moved entry as Aead.
New engine tests: packages/stack-encrypt/tests/json_read_checks.rs (a new file, to stay clear of test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345's rewrite of tests/json_index.rs). It has 23 tests: honest rows open, and each refusal (another path's entry, the sealed root, a relabelled entry, another position, another field of the same shape, a spliced row, a dropped row, a wrong depth, an orphan, a gap) is checked. It also checks what the design leaves open (a position read as the wildcard, another row's entry). There are unit tests in src/json/derive.rs for the selector derivation and the label check.
Mutation testing, mise x --env test -- cargo mutants -p stack-encrypt --in-diff <(git diff origin/next...HEAD): 88 mutants, 4 missed. All 4 were in rebuild_walked's depth counter. I merged that counter into the depth the rows are checked against and added a test at the deepest document the index seals. A re-run of the affected functions found 29 mutants: 24 caught, 5 unviable, 0 missed.

Review notes

Start at place and rebuild_walked in packages/stack-encrypt/src/json/derive.rs. Then read JsonSelector::open and JsonSubtreeKey::open in query.rs.
Conflict with test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345. That PR rewrites packages/stack-encrypt/tests/json_index.rs and adds an oracle test that opens -> results through JsonTreeEntry. This PR changes a few call sites in json_index.rs (JsonNode no longer opens on its own, and the hostile line-up tests now expect JsonWrongEntry). Whichever merges second needs those lines carried over. New engine tests went into a new file to keep the overlap small.
There are no Go, TypeScript or protect-ffi readers of json_tree (checked with grep), so the change is Rust only. No skill file mentions json_tree, so there is no skill update.

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…ed for Every entry of a json_tree document is sealed under its own selector and position, and nothing in its AAD says which path a query asked for. The client opened what `->` and the subtree walks returned under the `s` and `j` that came back with each row, so a database could answer `$.salary` with the row's `$.bonus`, or with the sealed root (the whole document), or splice other entries of the row into a walk, and the client would rebuild whatever it was handed (#1355). A row now opens only through what the client derived for the read. `JsonSelector::open(node)` refuses an entry that does not carry the selector, before any key is retrieved. `JsonSubtreeKey::open(rows)` checks every row: a node a gated walk reaches carries the path's selector and the position the path ends in; every other row carries the selector of its parent's path extended by its decrypted label, which the key now derives through the keyset's index-key PRF it holds; and each subtree's rows are consecutive in the document, so a dropped, moved or orphaned row is refused. The refusal is a new `Error::JsonWrongEntry`: each entry is genuine, so it is not `Aead`, and it is not of a wrong shape, so it is not `OpenedType`. `JsonNode` no longer opens on its own. `eql-bindings` follows: an entry `->` returned opens through `JsonTreeExtracted::from_selector(&selector, entry)`, and `JsonTreeEntry` no longer opens on its own. Which row an entry comes from stays the database's word, by design (ADR 0010); the leakage page and the design record say so. Closes #1355 Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

changeset-bot · 2026-10-24T08:06:56Z

🦋 Changeset detected

Latest commit: fe55bf1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@cipherstash/eql	Patch
stash	Patch
@cipherstash/stack-prisma	Patch
@cipherstash/basic-example	Patch
@cipherstash/e2e	Patch
@cipherstash/prisma-example	Patch
@cipherstash/stack-drizzle	Patch
@cipherstash/stack-supabase	Patch
@cipherstash/stack	Patch
@cipherstash/wizard	Patch
@cipherstash/bench	Patch
@cipherstash/test-kit	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

coderabbitai · 2026-10-24T08:06:59Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI; Enforced overrides: Organization
Review profile: CHILL
Plan: Advanced
Run ID: 01b21ea3-847d-48cb-aafb-5ee7553ca169

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

The design record marks the Phase 4 bullet on extracted entries taken; it now names the pull request that took it, #1358. Refs #1355 Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

github-actions · 2026-10-24T08:07:19Z

Mutation testing (cargo-mutants, `--in-diff`, stack-auth + stack-encrypt)

caught	missed	unviable	timeout
54	0	31	0

Every mutant in the changed lines was caught by a test.

A walk's rows that stopped before a node's authenticated child count was met were refused as `OpenedType` ("a complete JSON subtree"), while the same row left out between two others was `JsonWrongEntry`. Which error the client raised for a dropped row depended on where the database dropped it. The SQL walk never returns part of a subtree: it returns the node's whole subtree, or past its budget the sealed root alone, and an open root has no budget. So rows that run out early are always the database leaving a row out, and `rebuild_walked` now refuses them the same way it refuses a gap. `open_document` keeps `OpenedType` for a stored document cut short, which is a stored container's shape (ADR 0010), not a read's answer. Tests: the last row of a gated walk's subtree and of a walk without gates, left out; both failed with `OpenedType` before this change. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…s first element `json_tree_subtree` (a key with no gates) returns the descendants of the token's node but not the node, so nothing before the first row bounds where the children begin. The consecutive-row check caught a child left out between two others, but not the first: for `{"arr": [10, 20, 30]}` a database that dropped element 0's row had `$.arr` open as `[20, 30]`, every element moved down one position, with no error. An array's children are its elements from 0, so `place` now requires the nth row at depth 1 that is an element to be element n. A member's place is a keyed rank the client does not hold, so a member left out, and the last children of either kind, are still undetected; the rustdoc on `JsonSubtreeKey::open` says so. Both close once the walk returns the node it starts from (#1356), whose child count bounds its children. Tests: the first element left out, opened as a value and as a list (both opened before this change), and a pin of the trailing case that stays open. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

The leakage page's "What is authenticated" said a dropped row is refused and that each decrypted value is at the path asked for. For a walk with no gates neither held in full: the walk does not return the node it starts from, so children left out after the last row (and members left out anywhere) are not detected, and for a token chained through array positions (`$.l[3]`) the children can be another element's. The second was in the rustdoc and the design record but not on the page customers read. The page now lists both under what the checks do not cover, with the issue that closes them (#1356), and says what is checked for that walk: no gap between children, an array starting at element 0, and no row left out of a node's subtree. The design record's bullet says the same. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

coderdan · 2026-10-25T05:08:06Z

Review of `c3d7fee74`

I traced JsonSelector::open, place and rebuild_walked against each refusal the PR claims, built a counterexample for each gap I suspected, and confirmed it against the code before writing it down. Line numbers are at c3d7fee74. Severity is for this PR. "Inherited by #1363" means the same code survives in feat/json-tree-derivable-tokens at 617a15687.

Findings

[Medium] A walk with no gates accepts rows whose first array element was left out, and the array opens shifted down one position. packages/stack-encrypt/src/json/derive.rs:1399-1408 (the ungated branch of place) and :1293-1301 (assembling the elements). The walk does not return the node it starts from, so nothing bounds where its children begin. The consecutive check starts at the second child (next is None for the first). Concretely: {"arr": [10, 20, 30]}, key for $.arr (no gates), honest rows j = 2, 3, 4. The database returns j = 3, 4, and DecryptInto<Value> gives Ok([20, 30]), so the element that was at index 1 now reads as index 0. Vec<Value> gives the same. Children left out after the last row also open, as a shorter array, and so does a member left out of an object anywhere ({"o": {"a":1,"b":2,"c":3}} opened as two members). Proposed fix: in the ungated branch, require the nth top row that is an element to carry Label::Index(n). That closes the leading drop and the shift. A trailing child and a member cannot be checked without the node's child count, so document them as not covered until the walk returns its node (json_tree: a subtree read of {}, [], a scalar or a missing field all return no rows, so the client cannot tell them apart #1356). Not inherited by feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363: every read there returns the named node at depth 0 with its child count, so this goes away with the redesign. It is cheap to fix here, and fix(stack-encrypt): check that a json_tree read returns the entry asked for #1358 should be correct on its own.
[Medium, docs] The leakage page says more than the code checks. packages/eql/docs/reference/json-tree-leakage.md:45 says "a dropped or reordered row … [is] refused", and :47 says "each decrypted value is genuine and at the path asked for". Finding 1 contradicts both for a walk with no gates. The page also leaves out the chained-position gap ($.l[3] answered with another element's children). The rustdoc on JsonSubtreeKey::open and the design record both state that gap. Proposed fix: list both under "What the checks do not cover", with json_tree: a subtree read of {}, [], a scalar or a missing field all return no rows, so the client cannot tell them apart #1356, and state what the ungated walk does check.
[Low] The error for a dropped row depends on where the row was dropped. derive.rs:1495-1497: when the rows run out before a node's authenticated child count is met, the closure returns shape("a complete JSON subtree"), which is OpenedType. Dropping a row from the middle gives JsonWrongEntry. Concretely: a gated walk of $.nested.list[1] on the fixture returns j = 7, 8. Drop 8 and the result is OpenedType. Drop a middle row of a longer subtree and the result is JsonWrongEntry. The SQL never returns part of a subtree: past its budget it falls back to the sealed root, and an open root has no budget. So running out early is always the database leaving a row out. Proposed fix: return JsonWrongEntry ("the next node of the walked subtree"). open_document should keep OpenedType for a stored document that is cut short, because that is a stored container's shape (ADR 0010). Inherited by feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363: same closure, and feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363's leakage page already lists "truncated" among the JsonWrongEntry refusals.
[Low, not proposed for this PR] Repeated rows and rows of two documents are still OpenedType. derive.rs:1336-1345 ("rows of distinct entries", "rows of one document"). This predates the PR, and tests/json_index.rs:654-664 pins it. Like finding 3, it is the database's answer rather than a stored shape. Inherited by feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363, whose leakage page says "a dropped, truncated or repeated row … [is] refused" in the JsonWrongEntry paragraph. Changing it means editing tests/json_index.rs, the file that already conflicts with test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345. For the owner: either reword feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363's page, or move these two refusals to JsonWrongEntry once test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345 has merged.
[Low, test] the_sealed_root_among_walked_rows_is_refused never reaches the check it is named for. The test is in tests/json_read_checks.rs:520. The root sorts first (j = 0), so the top-of-walk check refuses it (depth 1 is not 0, and its selector is not the path's). rebuild_walked's node.whole.is_some() is never reached. As far as I can tell nothing reaches it: the root sits at j = 0, so it can only be a top row. A gated top must carry the selector of a path with at least one key or wildcard step, and an ungated top cannot carry Label::Root. Keeping it as defence in depth is fine. The test's comment already says which check refuses it. No change proposed. Inherited by feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363.
[Info]
- The PR body calls json_tree: a subtree read of {}, [], a scalar or a missing field all return no rows, so the client cannot tell them apart #1356 "the stacked PR". It is an open issue; the stacked PRs are docs(plans): design json_tree tokens derivable from the path alone #1361, feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363 and test(stack-encrypt): check the json_tree leakage page against the scheme #1362.
- languages/golang/encrypt/guest/src/status.rs:122-145 maps JsonWrongEntry to STATUS_INTERNAL through the catch-all. That is unreachable today, because there is no Go json_tree reader, but Go SDK: decrypt json_tree extracted entries and subtree walks #1352 will need it classified.

Checked, no finding

JsonSelector::open. The check runs before any key is retrieved. The sealed root opens only when $ was asked for. A position reads as the wildcard, as documented. A relabelled s fails as Aead.
Gated tops. Each must carry the path's selector, its last-step label or position (reaches), and depth 0 when a depth is reported.
Rows beneath a top. Below a top, the consecutive j check plus the AAD-bound j and the shared header pin every row to the document's own subtree. The selector and label checks below the top are therefore defence in depth. MAX_DEPTH bounds the recursion.
What JsonSubtreeKey holds.
- Debug prints the gate count only. Serialize writes t and g only. Selectors has no Debug and no Serialize.
- The PRF is shared through Arc<KeysetState>, so no key material is copied. It does live as long as any JsonSubtreeKey or JsonSubtree does, which I am noting, not reporting.
- Selector bytes built from decrypted keys stay in Protected up to the PRF (prf16_of).
JsonWrongEntry. expected is always a static description, never plaintext. The error payload and codes.rs cover it.
Mutation testing. cargo mutants --in-diff over the PR's diff (-p stack-encrypt): 84 mutants tested in 19m: 53 caught, 31 unviable, 0 missed.
Known non-findings (ADR 0010, the three accepted deviations, json_tree: the stored entry order reveals each row's shape and the alphabetical order of sibling keys #1357) are left out.

Conflict with #1345

git merge-tree origin/test/json-tree-containment-properties origin/fix/json-tree-extracted-entry-check:

Textual conflict: packages/stack-encrypt/tests/json_index.rs only. docs/plans/2026-10-17-json-tree-index.md and packages/eql/tests/encryption/tests/json_tree.rs merge cleanly.
The merged tree will not compile as it stands. test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345's new code opens JsonTreeEntry directly (decrypt_as::<Value, _>(entry, column()) in packages/eql/tests/encryption/tests/json_tree.rs) and JsonNode directly in tests/json_index.rs. The rebase has to route both through JsonTreeExtracted::from_selector and JsonSelector::open.

I'll push fixes for 1, 2 and 3 to this branch and restack the dependents.

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…e-derivable-tokens Brings in the review fixes on #1358: a walk cut short is JsonWrongEntry, a walk with no gates refuses an array that does not start at element 0, and the leakage page says what that walk still does not check. The leakage page conflicted. This branch's "What is authenticated" describes the derivable-token design, in which every read returns the node it names first, so the ungated-walk paragraph #1358 added does not apply here; this branch's text is kept. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…rivable-tokens Brings in the review fixes on #1358 through #1361. Conflicts in `json/derive.rs`, `json/query.rs` and `tests/json_read_checks.rs`, all where this branch replaced the gated and ungated walks with token reads. This branch's `place` and `JsonSubtreeKey::open` are kept: the leading-element check #1358 added belongs to the walk with no gates, which no longer exists here, and every read now returns the node it names with the child count that bounds its children. The truncation fix is carried over by hand: `rebuild_walked` here still refused rows that ran out before a child count was met as `OpenedType`, while the leakage page lists a truncated row among the `JsonWrongEntry` refusals. It is now `JsonWrongEntry`, like a gap. `a_truncated_read_is_refused` now expects that. #1358's ungated-walk tests are replaced by one that pins what this design closes: an array read missing its first or its last element is refused. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…e-leakage-profile Brings in the review fixes on #1358: a walk cut short is JsonWrongEntry, a walk with no gates refuses an array that does not start at element 0, and the leakage page says what that walk still does not check. No conflicts; the leakage profile's tests pass unchanged. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

coderdan · 2026-10-25T05:32:49Z

Fixes pushed (`c3d7fee74..efc6bba23`; new commits only, no history rewrite)

Finding 3 (a walk cut short) is fixed in 9ec5c59f1. When rows stop before a node's child count is met, rebuild_walked now refuses them as JsonWrongEntry ("the next node of the walked subtree"), the same refusal as a gap. open_document still returns OpenedType for a stored document that is cut short. Two new tests, one for a gated walk and one for a walk with no gates. Both failed with OpenedType before the fix.
Finding 1 (an array missing its first element) is fixed in aa591f240. In a walk with no gates, the nth child that is an element must now be element n. {"arr": [10, 20, 30]} answered without element 0 is refused, where before it opened as [20, 30]. One new test opens those rows both as a value and as a list; it failed before the fix. A second test pins what stays open until json_tree: a subtree read of {}, [], a scalar or a missing field all return no rows, so the client cannot tell them apart #1356: the last child left out (the array opens as [10, 20]), and a member left out anywhere. The rustdoc on JsonSubtreeKey::open now says so.
Finding 2 (docs) is fixed in efc6bba23. The leakage page's "What is authenticated" now says what a walk with no gates checks and what it does not: children left out after the last one returned, members left out, and the chained-position case, all with json_tree: a subtree read of {}, [], a scalar or a missing field all return no rows, so the client cannot tell them apart #1356. The design record's bullet matches.
Left as they are:
- 4 (repeated rows and mixed headers are OpenedType): changing it means editing tests/json_index.rs, which already conflicts with test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345. feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363's leakage page lists "repeated" among the JsonWrongEntry refusals, so the owner should decide whether to reword the page or move these refusals after test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345 merges.
- 5 (a test that never reaches its branch): defence in depth, no change.
- 6 (informational): no change.

Verification on efc6bba23:

cargo nextest run -p stack-encrypt --all-features: 833 tests run: 833 passed (1 slow), 1 skipped
doctests: 50 passed
clippy -D warnings: clean
cargo fmt --check, at the root and in packages/eql: clean
rustdoc with -D warnings: clean
cargo test --locked -p eql-bindings --all-features: all pass
mise run types:check: passes
mise run test:encryption: exit 0
mise run test:encryption:postgres, on a fresh database on PostgreSQL 17: exit 0. The json_tree suite gave 7 passed.
Biome: clean
cargo mutants --in-diff: 85 mutants tested: 54 caught, 31 unviable, 0 missed

Dependents restacked with signed merge commits:

docs(plans): design json_tree tokens derivable from the path alone #1361 is at 790d82782. The leakage page conflicted; I kept docs(plans): design json_tree tokens derivable from the path alone #1361's text, which describes the token-read design.
feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363 is at f42496ccb. It was built on 617a15687. I kept feat(stack-encrypt, eql): json_tree tokens derivable from the path, a self tag, client-side wildcards #1363's place and open, since the ungated walk is gone there. I carried the truncation fix into its rebuild_walked by hand, changed a_truncated_read_is_refused to expect JsonWrongEntry, and added a test that an array read missing its first or last element is refused. nextest: 855 passed, 1 skipped. clippy: clean.
test(stack-encrypt): check the json_tree leakage page against the scheme #1362 is at a0261c80c. It merged with no conflicts. json_leakage: 7 passed.

The bases on GitHub are unchanged.

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

next now carries #1345, which rewrote packages/stack-encrypt/tests/json_index.rs into given_<scenario> modules with an assertions module and proptest cases, and added the generated EQL oracle (packages/eql/tests/encryption/tests/json_tree/generated.rs). Conflict: packages/stack-encrypt/tests/json_index.rs only. Resolved by taking #1345's file as the base and re-applying this branch's changes to it, test by test: - imports JsonExtracted and JsonSelector; adds the selector_of and extracted helpers (now over the file's Cipher alias), and an assertions::refused_as_wrong_entry beside the existing refusal helpers. - given_an_extracted_node::opens_to_its_scalar_and_a_walked_subtree_to_its_value: each scalar opens through the selector of its path ($.hello, $.nested.list[0].x, $ for the sealed root, $.nested for the container), and the list opens from the gated walk to $.nested.list. - given_an_extracted_node::round_trips_with_the_envelope_the_sql_grafts and given_a_tampered_document's lifted node open through JsonSelector::open. - given_gated_walk_rows::open_to_the_reached_nodes_value: the ungated walk's rows under the gated key for $.nested.list are refused as JsonWrongEntry, where they used to assemble. - given_a_derived_record: the node field is an extracted entry and the subtree comes from the gated walk, five rows with the list itself. - given_a_fallback_root::is_only_the_sealed_root_alone_at_depth_zero: the root at depth 1 is refused as JsonWrongEntry. - given_rows_the_server_chose: both lines of containers are JsonWrongEntry; the repeated row stays OpenedType, as on this branch. No assertion of either side was dropped, and every #1345 test is kept. Compile fix outside the conflict: #1345's generated arrow test opened the entry -> returned as a bare JsonTreeEntry, which this branch no longer allows. It now keeps each derived JsonSelector and opens through JsonTreeExtracted::from_selector. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…e-leakage-profile Brings in next after #1345, through #1358's merge of it. Conflict: packages/stack-encrypt/Cargo.toml only. Both sides added the same proptest dev-dependency line (#1345 for the generated containment property, this branch for tests/json_leakage) with different comments. Kept the one line and one comment naming both uses. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

coderdan · 2026-10-25T06:55:08Z

`next` merged in after #1345 (`efc6bba23..fe55bf1a4`, a merge, no history rewrite)

What conflicted. Only packages/stack-encrypt/tests/json_index.rs, which #1345 rewrote into given_<scenario> modules with an assertions module and proptest cases. Everything else merged on its own.

How it was resolved. I took #1345's file as the base and put this PR's changes back into it, one test at a time:

The imports gain JsonExtracted and JsonSelector. The selector_of and extracted helpers come back, and there is a new assertions::refused_as_wrong_entry next to the other refusal helpers.
given_an_extracted_node: each scalar opens through the selector of its path ($.hello, $.nested.list[0].x, $ for the sealed root, $.nested for the container). The list opens from the gated walk to $.nested.list. The round-trip test opens through JsonSelector::open.
given_gated_walk_rows: the rows an ungated walk returns, opened under the gated key for $.nested.list, are refused as JsonWrongEntry. Before this PR they assembled into the list.
given_a_derived_record: the node field is an extracted entry, and the subtree comes from the gated walk (five rows, the list included).
given_a_fallback_root: the root at depth 1 is refused as JsonWrongEntry.
given_rows_the_server_chose: both lines of containers are refused as JsonWrongEntry. The repeated row is still OpenedType, as before (finding 4 of the review, still open).
given_a_tampered_document: the lifted node opens through JsonSelector::open and still fails as Aead.

No assertion from either side was dropped, and every #1345 test is still there.

Compile fix outside the conflict. #1345's generated -> oracle (arrow_opens_a_leaf_to_the_value_jsonb_holds, packages/eql/tests/encryption/tests/json_tree/generated.rs:795, :843) opened each returned entry as a bare JsonTreeEntry, and this PR no longer allows that. It now keeps each derived JsonSelector and opens the entry through JsonTreeExtracted::from_selector.

Verification on fe55bf1a4 (local PostgreSQL 17 on port 5400, in a throwaway database that I dropped afterwards):

mise x --env test -- cargo nextest run -p stack-encrypt --all-features: 852 tests run: 852 passed (1 slow), 1 skipped
cargo test -p stack-encrypt --doc --all-features: test result: ok. 50 passed; 0 failed
cargo clippy -p stack-encrypt --all-features --all-targets -- -D warnings: clean
cargo fmt --all --check, at the root and in packages/eql: exit 0 both
RUSTDOCFLAGS=-D warnings cargo doc -p stack-encrypt --all-features --no-deps: clean
packages/eql: cargo test --locked -p eql-bindings --all-features: all pass (136 passed, …, 16 passed; 2 ignored)
packages/eql: mise run types:check: exit 0
packages/eql: mise run test:encryption: exit 0
packages/eql: mise run build, then mise run test:encryption:postgres: exit 0. The json_tree suite gave test result: ok. 10 passed (this PR's 7 plus test: hold json_tree containment to jsonb with proptest, and restructure the JSON index tests #1345's three generated properties: -> opened 254 leaves, and <@ and @> passed at seed 1336, 64 cases)
Biome 2.5.9, check --diagnostic-level=error: no errors. The new proptest-regressions/*.txt files are not a type Biome checks.

Stack. #1362 (test/json-tree-leakage-profile) now has this branch merged in, head 9a7ca372c. The only conflict was in packages/stack-encrypt/Cargo.toml: both branches added the same proptest dev-dependency line, each with its own comment, so I kept the line once with one comment naming both uses. There, cargo nextest run -p stack-encrypt --all-features gives 859 tests run: 859 passed (1 slow), 1 skipped, with all 7 json_leakage tests, and cargo fmt --all --check exits 0. #1361 and #1363 will be merged up separately.

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

…e-derivable-tokens Brings in next after #1345, through #1358's merge of it. No conflict: the only line of these two documents that #1345 touched is the design record's "Where it is" paragraph (the generated oracle and the proptest properties), which this branch does not change. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

chatgpt-codex-connector · 2026-10-25T07:11:04Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-25T07:13:18.174088Z	`fe55bf1`	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-25T07:15:50.884721Z	`fe55bf1`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…rivable-tokens Brings in next after #1345 (through #1358 and #1361). Nothing in this branch's walk, derivations, SQL, EQL types or leakage page changes: the merge touches only tests, the proptest regressions and the oracle. Conflicts: - packages/stack-encrypt/Cargo.toml: both sides added the same proptest dev-dependency line with different comments. One line, one comment naming both uses. - packages/stack-encrypt/tests/json_index.rs: #1345 rewrote it into given_<scenario> modules. I took that file (with #1358's changes already in it) as the base and re-applied this branch's changes to the tests they now live in: - helpers: gated_walk becomes read_rows (the rows json_tree_subtree returns for key.tokens()); subtree_ordinals takes the token list; child_token and subtree_nodes are gone, with nothing left to use them. - given_the_fixture_document: stores_the_wire_form_the_sql_domain_checks expects the self tag `x`; places_the_chain_fields_where_the_design_says checks one distinct self tag per entry; stores_no_token follows the token list and checks self tags as well as parent tags. - given_a_subtree_key: #1345 split this branch's old token test three ways, and each part is ported to the token list. unlocks_exactly_its_subtree_from_the_path_alone is renamed finds_exactly_its_node_from_the_path_alone (one token, no probe, the node itself at depth 0). gates_each_key_step_below_its_head is renamed derives_every_token_below_its_head (the second element's own token, the server deriving the token the client sends, find). Its wildcard gates assertion is dropped, because a wildcard key carries no gates; the wildcard's tokens and probe are pinned in given_wildcard_rows. has_no_token_at_a_sealed_root_and_derives_from_an_open_one is renamed reads_the_root_under_either_option: `$` is now a read under the sealed root, so its TermError::JsonSealedRoot refusals (`$`, `$[*].x`) and the open-root gate count are gone, as on this branch. - given_a_position: reaches_no_object_member takes this branch's form, over `$.a[0]` and `$.h.a[0]`. That covers gates_on_the_elements_selector_below_a_key_step (a gates() assertion), which is removed. - given_an_extracted_node: the walk reads by token, and the $.nested read opens as a list to the one node at the path. - given_gated_walk_rows::open_to_the_reached_nodes_value becomes given_wildcard_rows::open_to_one_value_per_element_or_fail_with_the_bound, this branch's test. With it goes #1358's "ungated rows under a gated key" refusal: no walk is gated now. - given_a_derived_record: rows from read_rows. - given_a_fallback_root::opens_to_what_the_walk_opens_to takes this branch's form (paths compared against JsonPath::select, as a list and as an optional value). - given_rows_the_server_chose: the repeated row is JsonWrongEntry, as febc4b2 decided. #1345's generated oracle (json_tree/generated.rs) already opens through JsonTreeExtracted::from_selector and builds against this branch's types unchanged. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

cipherstash-agent

Automated review attestation (Claude Code, `/code-review medium` + `cross-check-review-findings`)

Head reviewed: fe55bf1a4. Result: no new findings. No critical or blocking issues.

What was read at head

place, rebuild_walked, open_subtree and child_selector_bytes in packages/stack-encrypt/src/json/derive.rs.
JsonSelector::open and JsonSubtreeKey::open in query.rs.
JsonExtracted and JsonSubtree in json/mod.rs.
The eql-bindings JsonTreeExtracted change.
The leakage doc and the changeset.

Checks

The -> selector check runs before key retrieval. The sealed root opens only for $.
A gated top must carry the path's selector, depth 0, and the last-step position (reaches).
Below a top, the checks cover consecutive j, depth, parent-path selector plus decrypted label, and authenticated child counts. The ungated leading-element fix is present (out_of_place).
Key material stays in Protected. JsonSubtreeKey Debug and Serialize expose no PRF or selectors.
The leakage page now matches the code, including the gaps it lists as not covered (#1356, ADR 0010).
CI is green at head: the nextest, doctest, rustdoc, mutants (0 missed), CRAP, fuzz-regression, Postgres-17 encryption and Go jobs.

Cross-check against existing feedback
The earlier review by @coderdan on this PR (c3d7fee74) raised 6 findings. I checked that findings 1–3 are fixed at head (9ec5c59f1, aa591f240, efc6bba23). Findings 4–6 are accepted or informational on that thread. My own candidates matched those and added nothing material, so I'm not repeating them. Bot comments (changeset-bot, CodeRabbit, mutants) carry no findings. The Codex review was still running when I read the PR.

Non-blocking, already known

JsonWrongEntry falls through to STATUS_INTERNAL in languages/golang/encrypt/guest/src/status.rs. This is unreachable until a Go json_tree reader exists (#1352). Please classify it there.
Repeated or mixed-header rows still return OpenedType, not JsonWrongEntry. The author deferred this, and it needs a decision on the #1363 wording.

Generated by Claude Code

docs(plans): name the pull request that took the extracted-entry check

c3d7fee

The design record marks the Phase 4 bullet on extracted entries taken; it now names the pull request that took it, #1358. Refs #1355 Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

coderdan added 3 commits October 25, 2026 15:51

coderdan mentioned this pull request Oct 25, 2026

json_tree: the stored entry order reveals each row's shape and the alphabetical order of sibling keys #1357

Open

coderdan marked this pull request as ready for review October 25, 2026 07:10

coderdan requested a review from a team as a code owner October 25, 2026 07:10

cipherstash-agent reviewed Oct 25, 2026

View reviewed changes

coderdan merged commit ae27e40 into next Oct 25, 2026
54 checks passed

coderdan deleted the fix/json-tree-extracted-entry-check branch October 25, 2026 08:06

This was referenced Oct 25, 2026

json_tree: a value read with -> or a subtree walk is not checked to be the path asked for #1355

Closed

stack-encrypt has no searchable JSON — add a Json index that seals a document's entries under one data key #1060

Closed

Navigation Menu

Repository navigation

fix(stack-encrypt): check that a json_tree read returns the entry asked for - #1358

fix(stack-encrypt): check that a json_tree read returns the entry asked for#1358
coderdan merged 6 commits into
nextfrom
fix/json-tree-extracted-entry-check

coderdan commented Oct 24, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Oct 24, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 24, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented Oct 24, 2026 •

edited

Loading

Uh oh!

coderdan commented Oct 25, 2026

Uh oh!

coderdan commented Oct 25, 2026

Uh oh!

coderdan commented Oct 25, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 25, 2026 •

edited

Loading

Uh oh!

cipherstash-agent left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

coderdan commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Not covered, by design

Verification

Related

Review notes

Uh oh!

changeset-bot Bot commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

coderabbitai Bot commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

Uh oh!

coderdan commented Oct 25, 2026

Review of c3d7fee74

Findings

Checked, no finding

Conflict with #1345

Uh oh!

coderdan commented Oct 25, 2026

Fixes pushed (c3d7fee74..efc6bba23; new commits only, no history rewrite)

Uh oh!

coderdan commented Oct 25, 2026

next merged in after #1345 (efc6bba23..fe55bf1a4, a merge, no history rewrite)

Uh oh!

chatgpt-codex-connector Bot commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

cipherstash-agent left a comment

Choose a reason for hiding this comment

Automated review attestation (Claude Code, /code-review medium + cross-check-review-findings)

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderdan commented Oct 24, 2026 •

edited

Loading

changeset-bot Bot commented Oct 24, 2026 •

edited

Loading

coderabbitai Bot commented Oct 24, 2026 •

edited

Loading

github-actions Bot commented Oct 24, 2026 •

edited

Loading

Mutation testing (cargo-mutants, `--in-diff`, stack-auth + stack-encrypt)

Review of `c3d7fee74`

Fixes pushed (`c3d7fee74..efc6bba23`; new commits only, no history rewrite)

`next` merged in after #1345 (`efc6bba23..fe55bf1a4`, a merge, no history rewrite)

chatgpt-codex-connector Bot commented Oct 25, 2026 •

edited

Loading

Automated review attestation (Claude Code, `/code-review medium` + `cross-check-review-findings`)