Skip to content

Expand Java/C++ cross-language compatibility matrix to full coverage - #1067

Closed
ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:claude/relaxed-hawking-6dc58e
Closed

ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:claude/relaxed-hawking-6dc58e

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

Summary

Closes #1057.

The Java/C++ compatibility matrix now covers every (type, encoding, compression)
combination both implementations can write, at four data scales, in both
directions — and building it out surfaced and fixed two real cross-language
format bugs.

Changes

  • Add a shared spec (compat/compatibility-matrix.csv) consumed by both the
    Java and C++ tests, so the two matrices cannot drift.
  • Replace the table-only Java/C++ compatibility tests with a single
    spec-driven EncodingCompressionCompatibilityTest on each side, covering
    both models (table + tree), all six compressions, and four data scales
    (10 / 1k / 100k / 1M rows) with page sizes that cross page boundaries at
    every scale.
  • Implement the C++ codecs that were missing so every Java-writable
    combination is now C++-readable: FLOAT/DOUBLE RLE, FLOAT/DOUBLE GORILLA_V1,
    DATE/TIMESTAMP ZIGZAG/SPRINTZ, BOOLEAN RLE.
  • Fix two cross-language format bugs found by the expanded matrix:
    • INT64 RLE run values were read little-endian in C++ but written
      big-endian by Java/IoTDB.
    • The SPRINTZ FIR predictor used a 128-bit intermediate in C++, diverging
      from Java's 32/64-bit wrapping multiply on large values.
  • Narrow the compatibility workflow trigger to the binary-format surface
    (main source, the spec, build files) so routine PRs don't pay the ~30 minute
    matrix; the coverage/self-check tests still run in the normal unit suite.
  • Report the per-run coverage/gap matrix in the CI job summaries.

Validation

  • Java generates 3260 fixtures; C++ reads all 3260 (0 skipped).
  • C++ generates 3168 fixtures; Java reads all 3168.
  • C++ full unit suite: 1421 passed, 0 failed.
  • Java encoding/RLE/SPRINTZ tests pass; spotless:check clean.

… scales

Close the Java/C++ compatibility gaps so every Java-writable
(type, encoding, compression) combination is readable by C++, and
expand the shared fixture matrix to four data-scale tiers with
page-size coverage.

- Add C++ FLOAT/DOUBLE RLE and GORILLA_V1 encoders/decoders.
- Wire DATE/TIMESTAMP into ZIGZAG/SPRINTZ and BOOLEAN into RLE.
- Fix INT64 RLE decoder endianness: Java stores run values big-endian.
- Fix SPRINTZ FIR predictor to match Java's 32/64-bit wrapping multiply.
- Replace the table-only Java/C++ tests with a spec-driven
  EncodingCompressionCompatibilityTest consumed by both sides from
  compat/compatibility-matrix.csv; cover tree + table models, four
  scales (10/1k/100k/1M rows), and page-boundary sizes.
- Narrow the compatibility workflow trigger to the binary-format
  surface and report the coverage/gap matrix in CI.
@ColinLeeo ColinLeeo closed this Nov 4, 2026
@ColinLeeo
ColinLeeo deleted the claude/relaxed-hawking-6dc58e branch November 4, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Test] Expand the Java–C++ cross-language compatibility matrix to full coverage

1 participant