Repository navigation
Python api - #31
Merged
Merged
Python api#31
Conversation
Reworks the interface from #12 against current main: - Generic Node(make, args, shape) replaces ~30 node classes, the registry and the dispatch module; sp.* atoms are mostly one-liners. - compile(f) is built on the existing problem bindings (f as the only constraint, zero objective), so no new C code; _bindings/ is unchanged. - Parameters sync to the engine automatically when their values change. - Restricted-domain atoms validate that their arguments are plain Variables, matching the engine's requirements. - Tests check every atom's value, Jacobian and Hessian against NumPy and finite differences, and run the README examples. They run in the cibuildwheel step. - scipy becomes a runtime dependency (results are csr_matrix). Co-authored-by: dance858 <danielcederberg1@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The engine's Hessian reuses Jacobian values cached by eval_jacobian, so Problem.hessian() returned wrong values unless gradient() and jacobian() had been called at the same point (7 of 39 atom cases differed). It now refreshes them itself; a regression test covers both Problem and compile. Also drop the unused Scope._parameters list and fold redundant branches in _mul and _index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A NumPy Q now goes to the engine's dense (BLAS) quad_form instead of being converted to CSR; scipy.sparse Q keeps the sparse path, matching matmul. The engine assumes Q is symmetric (gradient 2Qx), so a non-symmetric constant Q gave a correct value but wrong derivatives. Constant Q is now replaced by (Q + Q')/2, which leaves x'Qx unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem._u now validates that an explicit u has exactly n_vars elements (the C bindings read the buffer unchecked, so a short array read past its end) and slices the scope's vector to n_vars so variables declared after compile do not change what the problem sees. Constants in expressions must be scalars or 2-D arrays. A 1-D array used to become a column silently, which turned row + np.ones(n) into an (n, n) outer sum. Every expression is now (n, 1) or (1, n), never (n,). CompiledExpression.forward() returns an array shaped like the expression (column-major reshape of the stacked values). Problem.constraint_forward() still returns the flat stacked vector solvers expect. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Problem remembers the point at which each stage (objective forward, constraint forward, gradient, Jacobian) was last computed and skips the engine call when asked again for the same point. A point is the scope's values version plus the parameter version for u=None, or the last explicit u (one copy kept, compared with array_equal) so scipy's separate fun/jac callbacks at the same x cost one forward. gradient()/jacobian() raise if no forward has run, and hessian() requires both forwards at the same point instead of silently mixing Jacobian values from two points. forward(); jacobian(); hessian(w) now runs one forward and one Jacobian in the engine instead of three and two. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
X[i] on a matrix used to gather by column-major flat position, a silent departure from NumPy's row selection. It now raises; vectors still take a single index and matrices take X[i, j], X[i, :] or X[:, j]. Boolean and multi-dimensional index arrays raise instead of being cast to integers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Transurgeon
commented
Nov 8, 2026
Transurgeon
left a comment
Member
Author
There was a problem hiding this comment.
looks good! let me know if you want me to review also.
Value inputs used to be raveled column-major and only checked for element count, so a (3, 2) or flat (6,) array was silently accepted for a (2, 3) variable. Now: - Variable.value, Parameter.value and CompiledExpression.hessian(weights) take the exact shape of the variable, parameter or f. A 1-D array is also accepted for vector shapes and a scalar for (1, 1). - scope.set_values, the Problem point u and Problem.hessian lagrange must be 1-D with exactly n entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bool subclasses int, so x[True] and x[False] were treated as x[1] and x[0] instead of raising. Boolean index arrays were already rejected; now scalar bools (Python and NumPy) are rejected too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_Builder.__call__ recursed once per tree level, so a sum built term by term in a Python loop raised RecursionError at compile time from about 1,000 terms. Raising the recursion limit does not help: Python 3.14 then reports a stack overflow around 7,000 levels. The builder now walks the tree with an explicit stack, like _find_scope. Nodes are built in the same left-to-right post-order as before, so the capsules and the parameter registration order are unchanged. Tests cover a 5,000-term sum with a parameter (values, Jacobian, Hessian, and a parameter update) and a 3,000-level nonlinear chain through both compile() and Problem, checked against closed forms. Remaining limits are in the engine, not the Python layer: its C recursion overflows the stack somewhere between 1e5 and 1e6 levels, problem_init_derivatives is roughly quadratic in depth for long add chains, memory grows with nodes x n_vars, and a shared subexpression is re-evaluated once per use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Run pytest in the cibuildwheel test step. The [tool.cibuildwheel] test-command in pyproject.toml was overridden by the workflow's CIBW_TEST_COMMAND env var, so the suite never ran on the wheels. - quad_form with a Parameter Q: the engine assumes Q symmetric, so check symmetry when the atom is built and on every later Q.value update. - Expression.__iter__ raises TypeError instead of silently iterating via __getitem__ (a bare expression passed as `constraints` gave 0 constraints). - Reject None in Constant and in value setters instead of storing NaN. - index_flat wraps its argument like the other atoms. - __pow__ accepts any numbers.Real, including NumPy scalars. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Also name the deep-sum test's loop count n_adds rather than n_terms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node shared by the objective and a constraint became one engine node, so a constraint_forward at another point overwrote values that gradient() then read. Build the objective and the constraints with separate builders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each case is evaluated on Variables (the expression under test) and on their values with NumPy stand-ins (the expected result), instead of a registered function plus a separate reference per case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
x / c for a number or array c is x * (1 / c). Dividing by an expression stays unsupported (the error points to x * d ** -1), and dividing by zero raises. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python expression API (supersedes #12)
Reworks the interface from #12 from @dance858 .