Repository navigation
perf(csr): aggregate single-term unsorted rows without a scipy row gather - #1127
Merged
Merged
Conversation
…ther When every source row holds exactly one term, the stable cell order of the rows is also the order of their entries, so gather data and indices with plain numpy fancy indexing instead of scipy's CSR row indexing. Multi-term rows keep the existing path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F4nx8KhK8QzSzHhpYKvcSx
Merging this PR will not alter performance
Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
The following content was generated by AI.
CSRStore.aggregate(the kernel behind sparsegroupby().sum(),DenseStore.aggregateand, on #1122,LinearExpression.from_coo) spent most of its unsorted-rows time in scipy's CSR row gathersource[order]. When every source row holds exactly one term — the common case fromfrom_labels/1.0 * var— the stable cell order of the rows is also the order of their entries, sodata/indicesare now gathered with plain numpy fancy indexing and the existingindptris reused. Multi-term rows and sorted rows keep their existing paths. Output is identical (arrays and dtypes) to before.Related: #1124 (unifying the two grouped-sum kernels is out of scope here), #1122.
(1.0*flow).groupby(bus).sum(), 10 × 200k → 20kApproaches measured and rejected
np.repeat(rows, row_nnz)→_grouped/coo_to_csr): 1.36x on single-term rows but 2.1x slower on multi-term rows (grouping over nnz instead of rows).arange+repeatentry gather for multi-term rows: slower than scipy's row gather.np.argsort(kind="stable")instead of_grouped: ~6x slower.Correctness: randomized old-vs-new equivalence check (800 cases; int32/int64 index dtype, empty rows/grid, absent cells, NaN consts, explicit zeros, duplicates) — all identical; new test for stable term order with single- and multi-term unsorted rows.
Generated by Claude Code