Repository navigation
Separate RowFn batch inspection from execution planning - #10899
Conversation
Merging this PR will degrade performance by 9.47%3 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing |
|
im pretty sure the |
aa0745f to
17164f9
Compare
Keep the owned decoder as the required extension point and default borrowed decoding to it. Native primitive inputs share their canonical buffer without executing the array. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Inspect the borrowed inputs once for row counts, batch-constant classification, null literals, and strict validity. Batch execution reads that probe instead of scanning the inputs again, and borrows the columns and dtypes instead of collecting owned copies. Execution still plans with one dispatch and executes with a second one. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Pin the behavior that a later single-dispatch path must keep: valid-only decoding skips invalid payloads, a null literal skips earlier decoding, lazy division, and device transfers, and validity conjunction stays lazy for device-backed validity. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
17164f9 to
0ddfefa
Compare
Tracking Issue: #9128
Summary
Prepares the RowFn batch types for #10675 without changing how a batch executes. Execution still plans with one dispatch and executes with a second one, as on
develop.This is a purely mechanical change that should help with function call and access overhead.
Changes
InputElement::decode_ref, which decodes a borrowed input. The default clones the array fordecode. Native primitives share their canonical buffer without executing the array.BatchProbe: row counts, batch-constant classification, null literals, and the conjoined validity. Execution reads the probe instead of scanning the inputs again. A null literal skips the validity conjunction, because every row is null either way.BatchInputs, which borrows them when the caller holds them as one slice.RowFnExecutionArgsborrows the batch and the dtypes instead of owning copies.execute_row_inputsruns over anyRowInputs, andexecute_rowsadapts a generalExecutionArgsto it.finalize_kernel_outputwhen the output is not nullable, and marks theCachedIdread#[inline].Adds five regression cases covering invalid payloads, null short circuits, lazy division, device-backed validity, and device-backed primitive input. They pass on the existing execution flow, so #10675 shows that its single-dispatch path keeps that behavior.