Skip to content

Limit sparse constants by nonzeros, not rows times columns - #41

Merged
dance858 merged 2 commits into
mainfrom
sparse-constant-size
Nov 15, 2026
Merged

dance858 merged 2 commits into
mainfrom
sparse-constant-size

Conversation

@dance858

Copy link
Copy Markdown
Collaborator

A scipy.sparse constant is passed to the engine as CSR, so only its dimensions and nonzero count need to fit in a C int. The check from #38 rejected it by rows times columns, which ruled out, for example, the 126,015 x 78,478 branch incidence matrix of the 78,484-bus pglib-opf network (about 9.9e9 entries, 126,015 nonzeros).

  • Constant checks a sparse value with _check_sparse_size (rows, columns and nonzeros at most INT_MAX) instead of _check_shape.
  • Such a constant can be the matrix of a matmul or quad_form. Any other use needs it dense: elementwise operations are still rejected when the node is created, and a constant that reaches the engine as a dense parameter is now rejected in _build_one.
  • Dense constants, variables, parameters and nodes are checked as before.

Tested with a 65536 x 65537 sparse matrix in left and right matmul (test_large_sparse_constant). With this change the OPF benchmark in diffengine-paper-benchmarks builds the 78,484-bus network and matches CasADi to 3e-11.

🤖 Generated with Claude Code

https://claude.ai/code/session_012aJMgvtfPYcdJMWR6ox56t

A scipy.sparse constant is passed to the engine as CSR, so only its
dimensions and nonzero count must fit in a C int. The check from #38
rejected it by rows times columns, which ruled out e.g. the 126015 x
78478 branch incidence matrix of a 78k-bus power network. Such a
constant can be the matrix of a matmul or quad_form; any use that needs
it dense is still rejected, now when the expression is compiled.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aJMgvtfPYcdJMWR6ox56t
Comment thread sparsediffpy/_compile.py Resolved
@dance858
dance858 merged commit 9578d01 into main Nov 15, 2026
17 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant