Repository navigation
Conversation
Contributor
|
Important Review skippedToo many files! This PR contains 712 files, which is 612 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files.
 |
🦋 Changeset detectedLatest commit: 7d63656 The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Mutation testing (cargo-mutants,
|
| caught | missed | unviable | timeout |
|---|---|---|---|
| 543 | 0 | 404 | 0 |
Every mutant in the changed lines was caught by a test.
coderdan
added this pull request to stack #1230
October 16, 2026 11:24
A Bloom filter can hold every bit of a query without the value holding
the needle, so a `Matches` query returns false positives. ADR-0009
decision 4 makes such a term `Probabilistic`, with `verify` as the
operator's own meaning run on a decrypted value.
- `op::Probabilistic { type Plaintext; fn verify(&self, &Plaintext) -> bool }`.
Infallible: every outcome other than a match is `false`.
- `MatchQuery` keeps the needle's tokens as the index cut them, through a
new `Supports::carry` taken before the value is consumed. `verify` cuts
the decrypted value the same way and checks every needle token is among
its tokens. The tokens are zeroized on drop and never shown by `Debug`.
- `Supports::Query` is bounded by `op::TermFor<Op>`: any query term answers
`Eq`, only a probabilistic one answers `Matches`, so an index cannot
claim a text match without saying how its answer is checked
(tests/ui/op_inexact_operator_needs_probabilistic.rs).
- `verify` on an exact term does not compile (tests/ui/op_verify_exact_term.rs).
The tests plant a real Bloom false positive under a 32-bit filter and
check `verify` drops it.
Closes #1183
Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
- `Supports::query` takes the sealed `QueryToken` (from the #1190 fix), so no caller can pair a needle with another value's filter and have `verify` drop the real rows. `Needle`'s doc says how one is joined. - `TermFor`'s doc and ADR-0009 say the "only a probabilistic term answers `Matches`" rule binds this crate's terms; a term in another crate can opt in by hand. - The tokenizer zeroizes its own copies of the text: the folded string, its characters, and each n-gram, built at its final size so it does not reallocate. `verify` runs it on every decrypted row. - `MatchQuery`'s `PartialEq` says two needles can share a filter and still verify differently. - ADR-0009 decision 4's code shows the trait as this PR ships it; `target_limit` joins it with #1184. - Tests: `verify` is a token test, not a substring test ("abcd" verifies "abc bcd"), and the standard tokenizer verifies whole words in any order. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
- A match query under a configuration with an n-gram length of zero is refused, not a panic. The query chain tokenizes the value for `verify` before the term is derived, so it reached `slice::windows(0)` ahead of `MatchOptions::validate`. `tokenize` now yields no tokens for a zero length, and the derivation refuses the options as `TermError::InvalidOptions`, as it always did. - `tokenize` sizes its character buffer from the character count. Built with `collect`, it started from `Chars::size_hint`, about a quarter of the text, and each buffer it outgrew was freed holding plaintext. Its doc now names the one copy it cannot control, inside `str::to_lowercase`. - `match_term` says where its tokens are wiped: the PRF takes them by value and moves each buffer into a zeroizing `Protected`.
…erifier `Supports::Carry` and `Supports::carry` become `Supports::Verifier` and `Supports::verifier`. "Carry" said only that something was carried; the type is what a query term keeps of its plaintext so it can check a decrypted row later: `()` for an exact term, the query's tokens (`QueryTokens`) for a match query, which `verify` tests each row against. The trait's rustdoc and ADR-0009 decision 2 now explain the verifier and show the match index's impl, with the tokens `"alice"` is cut into.
ADR-0009 decision 5: a caller who wants `want` real matches from a probabilistic query fetches `want + Poisson_quantile(λ = p·N, confidence)` rows, where `p` is the mean over stored filters of `f^b`, `f` a filter's fill and `b` the bits the query sets. - `Probabilistic::target_limit(&self, want, &Stats) -> Limit`. - `op::Stats::declared(rows)` uses a new `MatchConfig::expected_tokens()` prior, a default method returning `None`, so existing configurations compile unchanged; it is not part of the index's identity. - `op::Stats::measured(rows, histogram)` takes `(set bits, rows)` buckets from one sampled aggregate over the column, and takes the mean of the power, never the power of the mean fill. An empty histogram falls back to the prior. - `op::Limit` is `Rows(n)` or `Unbounded`, the latter when nothing is known of the stored filters or the confidence is not in (0, 1). `fetch_or(page)` reads either as a fetch size. - The quantile sums the Poisson mass exactly in log space from `λ − 10√λ`, so it costs about `12√λ` terms; past `λ = 1e7` it is the conservative `λ + 8√λ`. Unit tests reproduce the ADR's worked table and its mixed-length column, and pin the quantile to reference values computed with `lgamma`. Closes #1184 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
- The Poisson quantile is a bounded, normalised sum over `[λ − 10√λ, λ + 12√λ + 50]`. Each term is a log relative to the window's first, so a large `λ` no longer loses the starting term to cancellation, and the sums run from the top, where the terms are smallest, so a confidence near 1 is compared against a tail rounding did not swallow. It matches an exact reference to within one up to 1 − 1e-12 (was off by thousands at λ ≈ 4.75e6), and no mutation of it can loop forever. `ln_factorial` is gone. - A `Limit` never exceeds `op::MAX_ROWS` (`i64::MAX`), the largest SQL `LIMIT`; `reltuples` of −1 read as a `u64` no longer overflows a bind. The ADR's statements use `greatest(reltuples, 0)`. - Histogram counts are summed as floats, so huge counts cannot overflow. - The independence assumption is stated in ADR-0009 decision 5 and on `target_limit`: a row sharing tokens with the needle passes more often, so the confidence is nominal on natural text; `verify` still decides. - Docs: the stats SQL filters NULL values; `target_limit` says a fetch that returned the limit is a page only when fewer than `want` verified; `expected_tokens` counts distinct tokens; decision 4's code shows `target_limit` on `Probabilistic`. - Tests: exact references at high confidence and at λ = 1e7 (which kills the threshold mutant), the `MAX_ROWS` cap, float-summed counts, and the measured-column test bounds the fetch against the histogram's own λ. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
CI's mutants gate found one survivor and three timeouts in `poisson_quantile`: - Shifting each log term by the window's peak never mattered: relative to the window's first term no term exceeds about e^100, so `-` → `+` there was equivalent. The shift is gone. - A mutated threshold or window bound sent the `MAX_ROWS` test's λ of about 9e18 into a window of billions of terms. The window is now `22√λ + 50` wide from its start, capped at 2^20 terms whatever the input, so no mutation can hang a test. - The window's end is the answer when the confidence allows no mass (above 1), now pinned by a test. A focused `cargo mutants` run over `op/stats.rs` leaves no survivor and no timeout. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
CI's mutants gate found `start..=start + width` → `start * width` surviving: the loop's range sat outside the cap on the window's width, so the mutation ran for billions of terms instead of failing. The loop now runs over `0..=width`, the capped width itself, and derives each `k` from it. A focused `cargo mutants` run over `op/stats.rs`: 91 mutants, 87 caught, 4 unviable, none missed, no timeouts. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
- A `want` of 0 is `Limit::Rows(0)`. The quantile was added anyway, so a dense column turned a zero-row request into a large fetch. - `Limit::fetch_or(page)` cuts `page` to `MAX_ROWS`, as the type promises, so `Unbounded.fetch_or(u64::MAX)` cannot wrap an `i64` bind. - Past ten million expected false positives the `λ + 8√λ` bound covers a confidence up to `1 − 1e-15` and no more. A higher one is now `Limit::Unbounded`; `poisson_quantile` returns `None` for it. - The row-count recipe no longer clamps `reltuples` to 0. A table never analysed has an unknown count, not an empty one, and 0 gave a confident fetch of exactly `want`. The catalog read now returns no row then, and the docs say to build no `Stats` and page, or run `ANALYZE`. - `rows` counts the non-NULL rows, since the histogram's pass rate is over those: the recipe scales `reltuples` by `pg_stats.null_frac`. Charging NULL rows at the non-NULL pass rate over-fetched in proportion to the column's NULLs. The rustdoc, ADR-0009 decision 5 and its examples, and the CHANGELOG say all of this.
…er, refuse only the query A match value that cuts no token under the index's tokenizer (empty or separator-only text, or text shorter than the n-gram length) used to fail term derivation on both the stored and the query side, so one two-letter name failed a whole row. ADR-0009 splits the two: the stored term is the empty filter, which no query matches, and only a query of such text is refused, because its filter would match every row. `match_term` now derives the empty filter for such text, with no PRF call, so `MatchTerms` as a target, `KeysetCipher::match_terms` and the `dynamic` record path all store it, and the value seals and opens as any other. The refusal moves to the query paths: `op::Supports::check` takes the query's value, the `Match` index refuses token-less text there with `TermError::EmptyTermText` when the query is awaited, before any term is derived or keyset loaded, and `dynamic::term` refuses it for a match kind. The error keeps its code and is reworded for the query. The Go module builds the stack-encrypt guest by path, so its round-trip test of the old refusal is rewritten to the new behaviour: a short `TextMatch` value seals, opens and matches no query, and a short match query is still `ErrTerm`. The package docs and the term-failure message follow. Two tests that used token-less text as a leaf that fails before any I/O now use a match configuration with invalid options, which still fails during the synchronous build. Closes #1185 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
match_terms is no longer a query. In 0.2.0 the same call served write time and query time and refused text that cuts no token; it now returns the empty filter for that text, and EQL treats an empty filter as contained by every stored filter. A caller who still sends its output as a query gets every row and no error, so the CHANGELOG entry moves to Breaking and says so plainly, naming cipher.query(op::Matches, ..) as the query. The query side validated the match options only when it derived the term, after the new no-token check had tokenised the text, so an n-gram length of zero panicked in the tokenizer instead of returning InvalidOptions. Both query prechecks, op::Supports::check and dynamic::term, now validate first, as the stored path always has. Go's term-failure locator asks the query path for each term, and the query path refuses short match text that a record stores as the empty filter. It could stop at an earlier row with a short email and name the wrong row. It now skips that one refusal by its code, and a test pins it with a short email in row 0. ADR-0009 and the Go package doc say that a stored empty filter tells a reader of the column the value is empty, separator-only or shorter than the n-gram length. The tokenizer's doc no longer says match_terms rejects that text.
…tch, stored or queried Rebased onto next, the EQL v4 TextMatch filter met this PR's change to match_terms. eql-bindings builds the stored TextMatch filter and the TextMatchQuery operand from one type, BloomFilter, through the stored-side match_terms. Before this PR that refused text that cuts no token, so both refused it. After it, match_terms returns the empty filter for that text, so a TextMatchQuery of "" or "al" became `bf: []`, which EQL treats as contained by every stored filter: a query matching every row, with no error, through Rust's encrypt_as and through the Go SDK's generated Query. BloomFilter's visitor now refuses an empty filter with TermError::EmptyTermText, for the stored value and the query alike, which is what EQL v4 TextMatch did before and what its docs, the Rust tests and the Go test already say. Storing a short value as the empty filter while refusing it as a query needs the query twin to have a filter type of its own, which is a codegen change left for later. The Go SDK README and the stashgen README still said a plain `match` index refuses short text; they now say it stores the empty filter and refuses only the query, and that an EQL TextMatch field is stricter.
- One rule for a match query, in one place. `QueryTokens::of_query` validates the options, tokenises the text and refuses text that cuts no token. The query chain and `dynamic::term` both call it, so the Rust chain and the Go guest cannot drift on what they refuse; `dynamic::term` no longer repeats the validate, tokenise and refuse steps by hand. - An awaited `op::Matches` query tokenises its text once fewer. `Supports` gains a provided `admit`, which checks and takes the carry together, and a match index overrides it to tokenise once for both. `prepare` calls `admit` instead of a second `check` and then `carry`. The term itself is still derived through the index's own operation, which tokenises again: that is the path a write takes, and sharing it is what makes the query's filter the stored filter's.
… no query_index ADR-0009 decisions 1 and 7. `op::Lt`, `Lte`, `Gt` and `Gte` take one value and `op::Between` a pair of bounds, inclusive at both ends. The ORE and OPE indexes answer them over the integer types (`op::Ordered`), ORE by default and OPE through `.via(Ope)`; an ordering on text does not compile, since text has no order term this crate produces. A comparison's query term is the index's own term for the value: this crate's ORE is CLLW, whose ciphertext has no left half to send alone, so what is sent is what is stored, as for an HMAC, and `Supports` is where the left-only term of block ORE (#1140) goes when it lands. A range's query term is one `sem::RangeQuery` carrying both bounds' terms, derived under the one label, read with `lo()` and `hi()`. A range whose low bound is above its high bound is refused when the query is awaited, before either term is derived and before any keyset is loaded (`TermError::InvertedRange`). An operator now says what a query takes through `op::Arity`: `One` plaintext or a `Pair` of them. `op::Input<A>` relates the value a query is given to the plaintext the plan is checked against, and derives one term per plaintext. `Supports`, `Answers` and `DefaultIndex` are keyed by that input, so `Supports<Eq, S>` reads as before and `Between` is `Supports<Between, (S, S)>`; `Supports::query` takes the input's terms. `StackCipher::query_index` goes, with `QueryIndexBuilder` and `QueryIndex`: every index a plan can declare is reachable through an operator now, and the chain holds its own input, index and label. `Resolves::resolve` takes the index and the call's context and returns the index and the label. ADR-0009's decision 2 shows the shape that ships, and its mentions of `query_index` are past tense. Closes #1186 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
Ordered was public and not sealed, so a downstream crate could implement it for its own type, such as a newtype over folded text that also implements the CLLW traits. Between refuses an inverted range with the type's PartialOrd while the server compares ORE terms, so such a type could refuse a valid range or accept an empty one, and it would open ordering on text, which ADR-0009 leaves to the catalog. Ordered now requires op::sealed::Sealed, which the macro implements for the integer types. A compile_fail doctest pins it with a type that meets every other bound, so the seal is the only reason it fails; without the seal the same snippet compiles. A trybuild test would pin rustc's list of sample implementors, which changes whenever a later layer seals another type.
… and AtPath ADR-0009 decision 1 names three JSON operators. `op::Contains` and `op::ContainedBy` are the containments (`@>` and `<@`), exact, and take one value, the document. `op::AtPath<Op>`, written `op::AtPath(op::Eq)`, applies another operator at a JSON path and inherits that operator's fidelity through `TermFor`: an exact term answers `AtPath<Eq>` and `AtPath<Lt>`, and only a probabilistic one answers `AtPath<Matches>`. No index answers them. This crate has no JSON index, and EQL v4 has no JSON type Stack Encrypt produces, so a containment query or a query at a path does not compile on any plan, which the compile-fail tests pin. `AtPath`'s arity is a new `op::Pathed`, a path and the leaf's input, and no `op::Input` of it exists until the JSON index arrives, since the path becomes a selector only that index can derive. The value-selector and selector-plus-leaf term shapes the issue lists wait for the same index. Closes #1187 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
…scribed once and every leaf records its type Pins the root workspace's vitaminc crates to the branch that implements vitaminc ADR 0001 (cipherstash/vitaminc #439 to #445) with a temporary `[patch.crates-io]`, and moves stack-encrypt and the guest ABI onto it. - A value describes itself with vitaminc's `Plaintext` and reads back through `Decrypt`. `PendingStackCipherText::plan` walks it into vitaminc's pending tree with no I/O; the target layer's `Pending` still fetches every key of an assembly in one batched call, and the leaves are then sealed and opened through the bytes-only `Cipher`: `MintedKeys` seals each leaf under the next generated key, `BoundKeys` opens a leaf under the key bound to it, and vitaminc derives every leaf's AAD and checks the markers and container rules on both sides. - Every leaf is tagged (wire version 2), so a Rust `u32` field and a data plan's `uint32` field seal the same leaf and each opens the other's. The record fixture is regenerated; its terms are unchanged. - `None` is a null leaf, a passthrough is a `PassthroughFrame`, and `StackCipherText` is `Node<SealedValue>`. - `dynamic::FfiValue` and `ValueKind` move into this crate, unchanged in shape, with `Plaintext` and a `Decrypt` sink, and cross a binding's boundary as transport frames. The deeper rework of `dynamic::Value` and `Scalar` is #1140. - stack-guest-abi's outputs and last error are a `HostValue` written as a transport frame. Closes #1118
- The CHANGELOG's `query_index` migration note no longer says `query(op::Matches, text)` is the same call. It returns a `MatchQuery`, refuses text that cuts no token, and compiles only for a configuration with `op::MatchCase`. `query_index(Ore, text)` and `query_index(Ope, text)` have no replacement until the order of text is settled, and the note says so. - `op::MatchCase`'s doc no longer says a configuration without it can be queried by index: that path is gone, and the doc names the one-line impl that makes such a configuration queryable. - `op::Ordered` has a curated diagnostic: an ordering over text, or over any type outside the integers, now says the orderings are integer-only. - `query` carries the `Input<Op::Arity>` bound and `op::Input` has a curated diagnostic, so `query(op::Between, &40u32)` fails at the value and says to pass `&(lo, hi)`, not only with an E0599 on `on`.
- `op::Pathed` carries the leaf operator's arity, `Pathed<A>`, so `AtPath<Eq>` takes `Pathed<One>` (a path and a value) and `AtPath<Between>` takes `Pathed<Pair>` (a path and a pair of bounds). A JSON index's `Input` impls can then derive one term or two as the leaf needs. A unit test pins the three arities. - `AtPath<Op>` takes only a leaf operator, the new `op::Leaf`: `Eq`, `Lt`, `Lte`, `Gt`, `Gte`, `Between` and `Matches`. `AtPath<Contains>`, `AtPath<ContainedBy>` and `AtPath<AtPath<_>>` are no longer operators, since none of them is one selector and value; `op::LeafArity` (sealed: `One` and `Pair`) keeps an accessor from nesting. `tests/ui/op_at_path_not_a_leaf.rs` pins the diagnostic. - The `DefaultIndex` diagnostic says no index answers the JSON operators yet, instead of only pointing at `.via(index)`, which led to a second error. `op::Input` gains a diagnostic naming what each arity takes. - The module intro lists the JSON operators and the `Pathed` arity, and `Operator::Arity`'s doc names all three arities.
…with no absent marker to fuzz The fuzz crate is its own workspace and compiles stack-encrypt by path, so it carries the same temporary vitaminc pin. check_record's model loses the None node, which ADR 0001 retires, and builds its passthroughs as frames.
…guests Both WASI guests build against the same temporary vitaminc pin as the root workspace, and every value crosses as a transport frame. - The encrypt guest decodes inputs with `FfiValue::from_frame`, writes outputs through vitaminc's `FrameWriter` into a buffer still sized before the first byte, and reads and writes ciphertext trees with `vitaminc_aead_frame`'s codec, a passthrough as its frame. - The credential guest's outputs are `HostValue`s. - The Go module moves vcffi and vcvalue to the same vitaminc commit, whose container tags and leaf kinds match the frame. The absent leaf kind is gone from the binding's leaf set. vcffi sends an 8- or 16-bit integer at its own width, which the engine reads as the 32-bit integer of its sign, so a generated field seals as it did.
…and drop the unpinnable capacity hint cargo-mutants found the unused half of MintedKeys and BoundKeys, and the UINT8 arm of FfiValue's sink, untested: no Rust source walks a u8 as an int, so the test reads a host's frame of one. The sink's capacity hint changed nothing a test can see, so it is gone.
… 0001 so the EQL jobs and the EQL guest stay green eql-bindings' `stack-encrypt` feature and tests/encryption compile stack-encrypt by path, so they move with it. - The EQL workspace carries the same temporary vitaminc pin as the root, the fuzz crate and both guests: every workspace that compiles stack-encrypt now resolves one vitaminc. - `encryption::targets` and the generated `v3::targets` take `FfiValue` and `ValueKind` from `stack_encrypt::dynamic` (the feature turns on stack-encrypt's `dynamic`); the generator emits the new import. `vitaminc-aead-value` is gone from both manifests. - The `Ciphertext` target takes `Plaintext` in place of `Encrypt` and `Decrypt + Send + 'static` in place of `Decrypt<'static>`. - The encrypt guest's lockfile drops `vitaminc-aead-value`, which only its `eql` build still pulled in. No other EQL code names a vitaminc API. Rebased onto next, which carries the EQL v4 types: - `encryption::plaintext`'s `Int32`, `Int64`, `Bool`, `Float32` and `Float64` sealed vitaminc 0.5's tagged leaves through `Encrypt` and `DecipherVisitor`, which ADR 0001 removes. They now describe themselves as the primitive they wrap (`Plaintext`), so they seal the leaf a data plan's field of that type seals, and read back through `FromScalar` only a scalar of their own kind and width, as before: vitaminc's primitive targets would convert any integer that fits and widen an f32. `vitaminc-aead-value` and `vitaminc-protected` leave eql-bindings; `vitaminc-aead` stays for the scalar kinds. - `tests/encryption/tests/v4_types.rs` takes `FfiValue` and `ValueKind` from `stack_encrypt::dynamic`. - `v4_rows.json` is regenerated by the Go test over the deterministic EQL guest: ADR 0001 leaves are wire version 2, so the committed ciphertexts no longer opened.
…s Encrypt rename The vitaminc crates came from `feat/aead-remove-legacy-traits` by branch name. That stack has merged into vitaminc main and the branch is gone, so a lockfile refresh could no longer resolve it. Every workspace that compiles stack-encrypt (the root, both Go guests, packages/eql and the fuzz crate) now pins one commit of cipherstash/vitaminc#464, vitaminc main plus the order-term comparison, which is the commit ore.rs's vitaminc implementation (cipherstash/ore.rs#115) builds against. Pinned by rev, not branch, so a merge cannot pull it out from under the build. Between the two commits vitaminc renamed the source trait `Plaintext` to `Encrypt`, to pair with `Decrypt`. stack-encrypt re-exports it under the new name, and every bound, impl, import and doc link in the stack crates, the guests, stack-guest-abi and eql-bindings follows. `TargetError:: Plaintext`, `Supports`'s `In::Plaintext` and eql-bindings' own `targets::Plaintext` are different things and keep their names. The trybuild expectations follow the rename. The other breaking changes since (Timestamp as plaintext, Copy normalizers, vitaminc-ore's column handle and compare) reach no code at this layer. v4_rows.json is unchanged: the leaves are the same format.
…dule to the same vitaminc Review follow-ups for the move onto vitaminc ADR 0001. `HostValue::Text` held a plain `String`. Before this PR the auth guest built its output from `FfiValue::String`, whose bytes live in a `Protected` buffer, so the access token it returns was wiped when dropped; after it, every `token` call left a bearer token in freed guest memory. `Text` now holds `Zeroizing<String>` on every path that builds one (the constructor, JSON, a decoded frame), and a test pins it. The Go module still named vcffi and vcvalue at 2e895cb9caed, the head of the deleted `feat/aead-remove-legacy-traits` branch, which builds only while the Go proxy has it cached. It now names the commit the five Rust workspaces pin, 2413bd0e4bf7 (cipherstash/vitaminc#464); the Go code under bindings/go is the same at both. Docs that still called vitaminc's source trait `Plaintext` (CONTEXT.md, the mixed_user example, two test headers and two manifests) say `Encrypt`. The CHANGELOG no longer says a reader of any other type is refused: an integer reader takes any integer width the value fits, and an `f64` reader an `f32`, which change no value.
This reverts commit c7b1205. Text and TextEq keep their Option<String> plaintext. A null is padded to a value's length only for types of fixed width; for text, None seals as a null of its own. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
The generated docs for Text and TextEq said a None is sealed as long as a value. That holds for types of fixed width only: a text null is not padded, since text's ciphertext is as long as the text, so its length may show that the value is absent. The changeset says the same. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
vitaminc-ore 0.8 encodes the Date, Timestamp and Decimal targets, and
clww-ore 0.6 and ore-rs 0.10 take them, so the engine can now derive their
order terms.
The typed path needs a Rust type for each that the engine can wipe and
clone. chrono's and rust_decimal's types keep private fields, and
`Orderable` is sealed, so eql-bindings' wipeable `v4::plaintext` types
could not join it. They move here as `value::{Date, Timestamp, Decimal}`
(with `RangeError`, and the chrono and rust_decimal conversions behind
features of those names), and become `Orderable` and `op::Ordered`. Their
one `unsafe`, in the test that proves `Drop` wipes in place, is allowed on
that test alone.
The data-plan path dropped a scalar's logical type when it picked an order
target, so a date would have ordered as its bare day count. `order_of` now
reads a date, timestamp or decimal into its `value` type first, and
`IndexSpec::takes` admits one exactly when it reads. With nothing left to
refuse, `Reason::OrderTermPending` is removed; it was never released.
A timestamp orders by the microsecond, floored, and a decimal by value
whatever its scale, as their equality terms already compare. Block ORE
puts a date through the 6-bit fixed-width engine and a timestamp or
decimal through the chained one.
Refs #1236
Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
…umeric types With the engine ordering dates, timestamps and decimals, the nine EQL v4 types it refused become producible: DateOrd, DateOrdOpe, DateOrdOre and their timestamp and numeric twins. eql-codegen lists them in ENCRYPTION_DOMAINS and loses the pending-order reason; the generated bindings, the target table and Go's encrypt/eql table follow. eql-bindings' `v4::plaintext` now re-exports Stack Encrypt's `value` types, and its chrono and rust_decimal features turn on Stack Encrypt's. Its one remaining direct use of vitaminc-aead, the null tag in the by-name dispatch, goes through `stack_encrypt::dynamic::frame`, so the dependency is dropped. Tests: the OPE types join v4_order's ordering checks, the block ORE types join v4_ore_block with a PostgreSQL comparison over dates, timestamps (ties within a microsecond, flooring before the epoch) and decimals (1.5 ties 1.50), and the between smoke test now ranges over every ordered family. Tests that used DateOrdOre as the unproducible example use Json. Go: stashgen's static engine and notices admit order indexes on decimals, dates and timestamps; the ValueKinds model takes the order indexes the value-kinds fixture now declares, and its test checks Lt against the stored terms, folded queries included. Closes #1236 Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
…ncrypt's The changeset for the unordered types still named eql-bindings' v4::plaintext as their owner. This branch moved them to stack_encrypt::value and re-exports them there, and the ordered types' changeset says so, so the two entries would have shipped contradicting each other. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
EQL v4's bindings are to store stack-encrypt's own order terms, typed by scheme and plaintext, rather than v3's untyped hex newtypes. That needs the terms to be serde types. OreTerm, OpeTerm and OreBlockTerm now serialize as lowercase hex in a human-readable format and as raw bytes in a binary one, through serdect, whose hex (base16ct) is constant-time. Deserializing goes through from_bytes, so a term of another scheme or a malformed one is refused at the boundary, with an error that names a length, never a byte. from_bytes now takes the Vec<u8> it decodes and keeps it, so neither deserializing nor a caller that owns the bytes pays a copy; TryFrom<&[u8]> stays and copies once. OpeTerm also checks its leading carry byte is 0 or 1: clww-ore's comparator refuses any other, but its from_bytes checks only the length. Upstream serde for these types is proposed in cipherstash/ore.rs#137, and constant-time FromHex in clww-ore in cipherstash/ore.rs#138. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
… terms, stored as one hex string EQL v4 reused v3's term newtypes for its order terms: `ob` was `OreBlock256(Vec<String>)`, a one-element array of hex, and `op` an `OpeCllw(String)`. Neither said which scheme or plaintext it held, and neither checked anything when read. A v4 `ob` field is now `stack_encrypt::sem::OreBlockTerm<T>` and an `op` field `OpeTerm<T>`, over the type's plaintext (`DateOrdOre::ob` is an `OreBlockTerm<Date>`). Each serializes as one lowercase hex string and is decoded through `from_bytes` when a payload is read, so an EQL v3 array, a term of the other scheme or a malformed term is refused there. `v4::terms` no longer re-exports v3's order terms. `OreBlockHex` and `OpeHex`, generated, describe the JSON to TypeScript and JSON Schema. One catalog entry, `eql_domains::Term::v4_format`, names each term's Stack Encrypt type, its JSON shape and the hex prefixes it starts with (block ORE `0202`/`0203`, OPE carry `00`/`01`). Codegen writes the Rust field, the JSON Schema pattern and the v4 domain CHECKs from it: every v4 domain carrying `ob` or `op`, stored, query or range bound, requires a JSON string with one of those prefixes. The CHECK reads only the prefix, so a write costs the same whatever the term's length; the comparator still checks the full structure. The v4 SQL reads `ob` as one hex string: `has_ore_block_v2` tests for a string, the extractor decodes it straight to the composite, which now holds one term rather than an array, and the one-element array overload of the comparator is gone. The v3 bundle is byte-for-byte unchanged. Exact per-type CHECKs (block count and length) wait on a key-free shape API in ore-rs and clww-ore, cipherstash/ore.rs#139. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
U-003 in the 3.1 upgrade guide says what changed for a hand-written eql_v4 payload: `ob` is one hex string, and every v4 domain carrying `ob` or `op` checks its prefix. A database that installed an earlier v4 bundle, such as the one attached to the Go module's alpha, keeps its stored domains across an install, so it must drop its stored order domains first; the guide gives the query that lists them. A changeset for @cipherstash/eql, CHANGELOG entries for eql-bindings and the Go module, and the bindings README and catalog reference describe the v4 term types. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
… range bounds The changeset linked U-003 relative to the repository root, but Changesets copies it into packages/eql/packages/eql/CHANGELOG.md, where that path names nothing. It now uses ../../docs, as the U-001 and U-002 changesets do. The v4 range reference still said a block ORE range's bounds are ob arrays, which the range domains now refuse with check_violation. It now says both bounds are one hex string, names the prefixes, and links U-003. It also no longer says the envelope waits on the ordered types becoming producible: they are, and a caller builds the envelope from two single-bound queries. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
feat(golang, stack-encrypt): optional sealed fields, stored as a sealed null or SQL NULL
…-decimals feat(stack-encrypt, eql-bindings, golang): order dates, timestamps and decimals
feat(stack-encrypt, eql-bindings, eql): EQL v4 order terms are typed, one hex string, checked when written
…les cargo compiles The protect-ffi binding links eql-bindings by path, so its cache key and the paths: filters of the eight workflows that build it named that crate. They named it as `packages/eql/crates/**`: every EQL crate, with its tests, docs, generated TypeScript bindings and the `v4` / `encryption` modules that only compile with the `stack-encrypt` feature, which protect-ffi never enables. The EQL v4 block ORE pull request (#1280) touched none of what the binding compiles and still started every Drizzle, Prisma Next, Supabase and protect-ffi suite. The key and the filters now list the files cargo reads: the manifest, `src/lib.rs`, `src/sql.rs`, `src/from_v2/**`, `src/v3/**` and the SQL bundle `src/sql.rs` includes. That list is derived, not remembered. A new `scripts/__tests__/lib/rust-compiled-files.mjs` follows a crate's modules from `src/lib.rs` under the features it is built with, plus its include_str! targets; for eql-bindings it agrees file for file with cargo's own dep-info. ffi-binding-action.test.mjs now fails both ways: a compiled file the key misses (a new module, a feature turned on), and a file beside the compiled crates that the key hashes and cargo does not read. The filters are held to the key as before, and wasm-build-inputs-paths-filter.test.mjs now also fails a filter entry reaching into packages/eql/crates beyond it. The reverse gap, found on the way: the Drizzle, Prisma Next, Supabase, Prisma example and bench workflows install EQL from the workspace `@cipherstash/eql` package and did not trigger on it, so an EQL release bump started none of them. They now list `packages/eql/packages/eql/**`.
tests.yml ran its credentialed jobs (Node 22 and 24, Bun, the e2e suite and the WASM / Deno smoke test, each building two native bindings) on every pull request, including ones that changed only Go, the Stack Encrypt crates or EQL's SQL. A new `changes` job now skips them unless the pull request changes a file they can read. The question is asked the safe way round. Instead of an allowlist, which skips a suite on the change that breaks it the day someone forgets a directory, the jobs run unless every changed file sits in an excluded tree. dorny/paths-filter cannot re-include a path inside one list, so the decision is two lists: `rest` (`**` minus the excluded trees) and `direct` (what the jobs do read inside them: the eql-bindings files the binding compiles, the `@cipherstash/eql` package, tests.yml itself). The checks that read the whole tree move to an ungated `repo-checks` job: the lint scripts, the release gate, `test:scripts`, and the suites that read workflow files (protect-ffi's lintWiring and integrationSuiteCi, the e2e supply-chain and skill-map checks). They ran inside `run-tests` once per Node leg; gating that job without moving them would have left a workflow-only change with none of its guards. `tests-required` summarises the run as one check a branch rule can require, since a skipped job required by name never reports. tests-workflow-gate.test.mjs holds the lists to the jobs: every job but the four ungated ones carries the gate; no code under the live roots reads an excluded path (a textual scan that resolves relative paths from the file and its package root, with each exception written down and stale-checked); the crates the auth and profile bindings compile stay selected; and the decision script is run once per event. wasm-build-inputs-paths-filter.test.mjs now reads the gate as tests.yml's effective pull_request filter, so the FFI build inputs are held against the list that actually starts the jobs.
…heck runs repo-checks now runs test:scripts straight after install. Nothing before it builds @cipherstash/eql, so bench-index-expressions' check that dist/sql is a verbatim copy of sql skipped without a word. On next the run-tests turbo builds came first and the check ran. The job builds the package first, and sets EQL_DIST_REQUIRED, which makes the check fail rather than skip when dist/ is missing. Locally it still runs only after a build. tests-required and the workflow's paths filter now say that the check must not be required while plans-only pull requests skip the workflow. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
…tgreSQL Nothing in CI ran the path a Go program takes: rows the Go SDK stores in EQL columns, found by queries the Go SDK derives, through EQL's operators. encrypt/postgres_test.go stores terms in bytea columns with no EQL, and the Rust encryption suite runs EQL in PostgreSQL but derives its own queries, so a change that put Go and the SQL out of step failed nowhere (#1289). TestEQLTypesRoundTripAndQueryInPostgres walks eql.Types, the table generated from the EQL catalog, and for every type the engine produces declares a field of it at run time through gensupport, stores four rows in a column of its domain, and runs each query its indexes answer (=, <, <=, >, >= and @@) through the operand domain, asserting the exact rows. A type with no query for an operator must be refused with EQL's own error. The rows are read back from PostgreSQL, which stores jsonb re-serialised, and opened. A type whose plaintext or index the test has no case for fails rather than skipping, so a type the engine starts producing is covered or loudly not. With DATABASE_URL set and no eql_v4 schema it fails: a skip reads the same as a pass. CI runs it in a new eql-postgres job in tests-golang.yml, after wasi-check and on its guests, against postgres:17 with the bundle `mise run build` writes from the tree, through the new `mise run go:test:postgres` task (also the local entry point, against EQL_TEST_DATABASE_URL). The task fails if either PostgreSQL test skips. The existing bytea test moves there with it, since its database now carries EQL. The workflow's filters gain EQL's SQL sources and the codegen that writes the bundle. `eql_v4.between` is not covered: the Go SDK derives no EQL range term yet (Range on an EQL ordered field is refused by the engine), so there is no Go query to run through it. Closes #1289 Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
next now produces the ten block ORE types (IntegerOrdOre, TextSearchOre and the rest), and the test fails on an index it has no query for, as it is meant to. ore_block answers lt, lte, gt and gte, as ore and ope do. All 49 producible types pass against PostgreSQL 17. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
ci: run JS and FFI suites only on changes that can reach them
…ad back short From review. go test -run with a pattern that matches nothing prints "no tests to run" and PASS, so a renamed test left the job green: the task now requires each test's own --- PASS line. The test also checks rows.Err() after reading the stored rows back, and that it opened as many rows as it stored, so a short read or decrypt cannot check fewer rows and pass. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
Optional fields are what #1298 is for, and nothing ran a stored nil through EQL in PostgreSQL. Each type now gets a fifth row with no value, stored as the engine requires for it. A type of fixed width with no index or equality alone stores a sealed null (17 types): its c must be as long as a value's, EqNull must find it and only it, and IS NULL must find nothing. Text and every ordered or match type store SQL NULL (32 types): IS NULL finds it, and no value query does. The nil row opens as nil. Breaking the length check and the EqNull check on purpose fails all 17 and all 8 types they cover. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
test(golang): run the Go SDK rows and queries through EQL v4 in PostgreSQL
TestEQLTypesRoundTripAndQueryInPostgres runs <, <=, > and >= over four fixed values per type. A comparator that is only partly wrong (one that reads only a block ORE term's first blocks, mishandles a sign bit, or orders a decimal by its scale) still agrees with plaintext order on most pairs, so four values say little. The Go property tests compare terms in Go, and the SQLx property suite covers EQL v3, so nothing ran random Go SDK values through EQL v4's operators. TestEQLOrderMatchesPlaintextInPostgres stores about 360 values in a temporary table for every producible ordered EQL v4 type (29 today): 300 random draws, the edges of the type's order, and thirty duplicates. Each of 20 query values goes through each ordering operator and must find exactly the rows plaintext comparison selects, and ORDER BY the type's order term must return the rows in plaintext order, both ways. Plaintext order is the engine's: text by its fold, -0.0 equal to 0.0 and NaN above +Inf, timestamps to the microsecond, decimals by value. The seed is fixed and logged, EQL_ORDER_SEED replaces it, and each type draws from its own generator so one subtest replays alone. A failure names the type, the operator, the seed and the value pair. The connection and EQL-installed check move into a helper both tests share, and go:test:postgres runs and requires the new test. Closes #1343 Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
The edges hold the ties the order rules make: -0.0 and 0.0, 1.5 and 1.50, two instants in one microsecond. An engine that splits one of those ties orders the tied rows one way every time, and the ORDER BY check accepts tied rows in either order, so only a query on one of the tied values can show it. Four edges were picked at random as query values, and with the default seed four of the six float types passed with a plaintext model that tied two values the engine keeps apart. Every edge is now a query value, and all six fail on that model. Refs #1343. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
eqlOrderIndex kept its own list of order index names, and a type whose index was not on it was skipped, with only a zero-types guard behind it. The round-trip test fails on an index missing from eqlIndexOperators, so a new order index would be added there and this test would then pass without ever running the new types. An order index is now any index whose operators in eqlIndexOperators include lt, so a new one reaches runEQLOrder and fails at the missing extractor until it has one. Refs #1343. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
…nal tokens and the header Two review findings on the record (#1250) read the design as written and found two gaps that the implementation closed in #1329, with nothing in the record saying so. A positional token as first written named the element's children rather than the element, so a scalar element could not be returned or identified; the implementation reaches a position from the parent's token through a `#n:<selector>` gate and returns the reached entry itself. The wire shape called the header opaque to SQL while the walk needs the document's iv and the sealed-root flag from it; the implementation reads both at fixed offsets of the frozen header layout. Each passage now carries a dated note pointing at the implementation and the phase notes, in the form the record already uses for superseded sections. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu
…pt-spike docs(plans): a JSON index for Stack Encrypt as a new EQL v4 domain, with runnable SQL prototypes
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the integration branch for a release candidate of EQL v4 and of Stack Encrypt's Rust and Go support. EQL is the SQL that lets PostgreSQL store and search encrypted values; EQL v4 is a second copy of it for values written by Stack Encrypt, the Rust encryption engine behind the Go SDK. The open pull requests for both are merged here first, so they can be tested together before any of them reaches
main.The release version is not decided yet.
Why a branch, not main
Some of this work depends on vitaminc and ore.rs code that is not yet on crates.io, so it pins those crates to a git commit.
stack-encryptandeql-bindingsare published crates, andcargo publishdrops those git pins, so nothing on this branch can be released until vitaminc 0.6 is published (cipherstash/vitaminc#389, which waits on cipherstash/vitaminc#463). Once it is, the git pins become crates.io versions and this branch merges tomainas a normal release.What lands here, in order
feat/aead-remove-legacy-traitsto one commit on vitamincmain, and takes vitaminc's breaking changes since then.Rules for this branch
rev, neverbranch. A branch is deleted when its stack merges.packages/eqlandpackages/stack-encrypt/fuzz. Two commits mean two copies of vitaminc's traits, and the code stops compiling.Changes
Verification
nextwas created frommainat 2a70a47. Nothing has been built yet, because nothing has changed.Related