Repository navigation
Conversation
An object member's token mixes in its ordinal, a keyed rank among its siblings that the client cannot know for a row it has not read. So the client can derive a token only for a top-level field and the array positions after it, and walks the rest through gates. A read of $.profile.address.city therefore sends $.profile's token, a capability over that whole field in every row, and the fix proposed for #1356 widens that further by sending an array's token to read one element. Record the agreed change: member tokens drop the ordinal, every entry gains an eight-byte self tag so a read can find a node from its own token and return it at depth 0, and the client expands wildcards into concrete paths with a bound and a probe, so gates and json_tree_subtree_at go away. The section gives the derivations, the SQL read, the adapted client check, the wire change, the leakage, the implementation and measurement plan, and the inconsistencies found while checking the design against the code, as open questions for the owner. Short dated pointers mark where sections 2, 4 and 6 and the Phase 3b notes state what the change reverses. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu
The leakage page told customers that a subtree read sends the token of the top-level field the path starts with. Under the design change in the json_tree design record, a read sends the token of exactly the node asked for, a wildcard sends a bounded list of concrete tokens and one probe per level, and every entry carries a self tag. Revise the subtree read bullet, add the self tag to the at-rest table and the unauthenticated fields, adapt the walk check, the comparison row and the sealed_root option, so the page describes the format the implementation stacked on this branch will build. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu
|
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
This was referenced Oct 24, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A write-up only: no code changes. EQL v4's
json_treecolumn stores an encrypted JSON document as one entry per JSON node. To read part of one, the client sends the database a token, a secret derived from the path that lets the database map the structure under that node in every row. Today an object member's token mixes in its place among its siblings, which the client cannot know. So a read of$.profile.address.cityhas to send the token of the whole$.profilefield. The same cause is behind #1356, and the fix proposed in #1359 widens the leak further. This PR records the agreed change that removes it (tracked in #1360):It also revises the leakage page to say what the database learns after the change. It lists eight open questions, found while checking the design against the code, for the owner to answer before implementation starts.
Changes
docs/plans/2026-10-17-json-tree-index.md: a new dated section, "Design change (2026-10-24): derivable tokens, the self tag, and wildcard expansion on the client". It covers why, then the token rules, the self tag, wildcard expansion and the new SQL read, the client's checks, the wire change, the leakage, the plan with its test list and benchmark step, and the open questions. One-line dated pointers sit where sections 2, 4 and 6, the Phase 1/2 deviations and the Phase 3b notes state what the change reverses. Nothing earlier is rewritten.packages/eql/docs/reference/json-tree-leakage.md: the subtree-read bullet (exact tokens, wildcard expansion and its probe, the root), a new row for the self tagxin the at-rest table,xin the list of fields that are not authenticated, the walk check, the comparison row, and thesealed_rootoption.Do not merge this ahead of the implementation. The leakage page will then describe a format that is not built yet. The plan is to stack the implementation on this branch so the two land together.
No changeset: this changes documentation only, in the repository's design record and the EQL reference page, and no published package's surface.
Verification
bash tasks/test/doc-anchors.shinpackages/eql(withEQL_ROOTset to the subtree):doc-anchor gate OK — 123 intra-document link(s) resolve.docs:validate:sourcewas not run: it checks the--!comments in the SQL sources only, and this PR changes no SQL.grep eql_v2on the leakage page: no match (whattest:docs_v3_grepchecks).child_token,parent_tag,rank/shuffled_ordinals,entry_aad, the walk's head derivation, andsubtree_key_forinpackages/stack-encrypt/src/json/derive.rs;selector_bytes/token_bytesinpath.rs; andjson_tree_child_token,json_tree_subtree_atand the walk's "bound" comment inpackages/eql/src/v3/json_tree/functions.sql.Related
Refs #1360 (the issue this designs).
Refs #1356 (solved as a side effect), #1359 (superseded in mechanism; its behaviour and oracle test carry over), #1358 (base branch; its checks are adapted), #1357 (unchanged by this), #1345 (should merge before #1358).
https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu