Skip to content

docs: explain SQL fixtures that miss Comet expression coverage - #7333

Draft
Abhishek-Mohan wants to merge 1 commit into
apache:mainfrom
Abhishek-Mohan:docs/6619-sql-test-pitfalls
Draft

Abhishek-Mohan wants to merge 1 commit into
apache:mainfrom
Abhishek-Mohan:docs/6619-sql-test-pitfalls

Conversation

@Abhishek-Mohan

Copy link
Copy Markdown

Which issue does this PR close?

Part of #6619. The tolerance-comparison documentation remains separate, so this does not close the issue by itself.

Rationale for this change

A SQL fixture can match Spark's results without exercising the expression its author intended to test. Disabling ConstantFolding does not stop NULL propagation, evaluation over inline VALUES, or Comet's planning-time literal casts. The guide also incorrectly showed binary bit_length using the codegen dispatcher.

What changes are included in this PR?

Update docs/source/contributor-guide/sql-file-tests.md to:

  • Explain the remaining traps: NULL literals, literal casts, inline VALUES, small-insert file layout, regex escaping, and session time zones.
  • Show a Parquet-backed string-to-int cast example with valid, invalid, and NULL inputs, and add the COALESCE(1) insert idiom to the setup steps without promising batch size or row order.
  • Correct the ascii(NULL) example and the all-literal execution claim; replace the binary bit_length dispatch example with explicitly configured lower.
  • Explain that operator coverage does not establish expression execution, including why expect_native can still report a planning-time literal cast as native.

How are these changes tested?

Documentation only. Checked the examples and claims against the SQL-file parser, runner, expression serdes, coverage tagging, existing fixtures, and Spark optimizer/parser sources.

  • git diff --check passes.
  • Structural checks pass for all six local heading links and balanced code fences.
  • SQL runtime tests and the Sphinx build were not run.
  • Prettier was not run because Node, npx, and Prettier are unavailable in the environment.

Council review

Rounds: 1

Delegate Round 1
Claude Opus 5.5 9/10
Gemini 4 Argon 10/10
GPT-6 Astra 10/10

All three reviewers were satisfied with the scoped change and found no required fixes. Reviews followed the tstack council workflow at high reasoning effort.

AI assistance: implemented with Claude Opus 5.5 and coordinated with GPT-6 Astra through OpenCode in T3 Code.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant