Skip to content

perf(highs): build the model in bulk from the row-wise matrix - #1118

Closed
FabianHofmann wants to merge 5 commits into
masterfrom
perf/highs-bulk-passmodel
Closed

FabianHofmann wants to merge 5 commits into
masterfrom
perf/highs-bulk-passmodel

Conversation

@FabianHofmann

@FabianHofmann FabianHofmann commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Closes # (if applicable).

Changes proposed in this Pull Request

Note

The following content was generated by AI.

Highs._build_solver_model (direct API, also used by linopy.io.to_highspy) built the HiGHS model in several calls: addVars, changeColsIntegrality, changeColsCost, then addRows.

This PR passes the model in bulk and hands HiGHS the cached CSR matrix as is:

  • MILP: one h.passModel(...) call (raw-array overload, MatrixFormat.kRowwise) with a full-length int32 integrality array (0 continuous, 1 binary/integer, 2 semi-continuous). HiGHS rejects an empty array.
  • Pure LP: one addCols (cost and bounds, no matrix entries), then one row-wise addRows. passModel is not used here because an all-continuous integrality array makes HiGHS write empty bin/gen/semi sections to LP files, which CBC, GLPK and COPT misread.
  • Row bounds compare M.sense as integer codes (view(np.uint32)) instead of unicode strings.
  • Names, the Hessian and the objective sense work as before.

An earlier version of this PR converted A to CSC first. That held a second copy of the matrix during the build, and CodSpeed flagged ~20% more memory in test_to_solver[highs-*]. Passing the row-wise matrix directly is about as fast and removes that copy.

Benchmark and test details

Synthetic model, 1.26M variables, 4.9M nonzeros. Build stage only, 3 fresh processes per variant, cloud VM. Peak RSS is the increase during the build.

variant LP build LP peak RSS MILP build MILP peak RSS
master 386–450 ms 315 MB 383–458 ms 321 MB
previous PR head (CSC) 291–320 ms 348 MB 216–251 ms 235 MB
this PR 305–316 ms 311 MB 245–314 ms 184 MB

Solve time on the same model is unchanged.

Tests: test_highs_direct_known_optimum covers a semi-continuous model and a model without constraints. test_to_highspy_pure_lp_writes_no_integer_sections checks that LPs get no empty integer sections. pytest test/test_io.py test/test_optimization.py: 655 passed, 16 skipped. ruff check, ruff format and mypy . pass.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included.
  • I consent to the release of this PR's code under the MIT license.

Generated by Claude Code

Highs._build_solver_model built the model with addVars,
changeColsIntegrality, changeColsCost and a row-wise addRows. HiGHS
keeps an addRows matrix row-wise and transposes it at solve time. Pass
the whole LP in one passModel call with a CSC matrix and a full-length
integrality array instead, and compare the constraint sense as integer
codes rather than unicode strings.

On a 1.79M-variable / 4.29M-row / 7.18M-nnz network the HiGHS build
stage drops from ~1.22 s to ~0.60 s with ~110 MB lower peak RSS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4nx8GJ8mSNvkTagja9dMj
@codspeed

codspeed Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 29.8%

⚡ 6 improved benchmarks
✅ 175 untouched benchmarks
⏩ 181 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_to_solver[highs-knapsack-n=100] 100.3 KB 70.3 KB +42.71%
⚡ test_to_solver[highs-milp-n=10] 101.6 KB 75.1 KB +35.24%
⚡ test_to_solver[highs-piecewise-n=10] 101.1 KB 77 KB +31.26%
⚡ test_to_solver[highs-piecewise-n=1000] 2.5 MB 2 MB +26.84%
⚡ test_to_solver[highs-knapsack-n=10000] 1.4 MB 1.1 MB +24.61%
⚡ test_to_solver[highs-milp-n=50] 512.2 KB 428.7 KB +19.47%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing perf/highs-bulk-passmodel (6b8cc1a) with master (22a1882)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@FBumann

FBumann commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Saw the same speedup in specsolve.

Pass enums and the final objective sense to passModel, drop no-op dtype
wrappers and redundant guards, fold the new test into test_to_highspy
plus a two-case known-optimum test, and shorten the release note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4nx8KhK8QzSzHhpYKvcSx
@FabianHofmann
FabianHofmann marked this pull request as ready for review November 3, 2026 14:35
@FabianHofmann

Copy link
Copy Markdown
Collaborator Author

Saw the same speedup in specsolve.

nice!

FabianHofmann and others added 2 commits November 3, 2026 15:36
An all-continuous integrality array makes HiGHS write empty bin/gen/semi
sections to LP files, which CBC, GLPK and COPT misread. Use passModel only
when the model has discrete columns; otherwise add rows, then columns
column-wise via addCols.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4nx8JFY4SxxVLJEByWw8z
@FBumann

FBumann commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Just to mention it to you right now:

Gurobi has more to gain: in specsolve, one gurobipy.loadModel call made loading 3–6× faster than addMVar + addMConstr (fluxopt/specsolve#1942). That's worth a separate PR.

@FabianHofmann

Copy link
Copy Markdown
Collaborator Author

Just to mention it to you right now:

Gurobi has more to gain: in specsolve, one gurobipy.loadModel call made loading 3–6× faster than addMVar + addMConstr (fluxopt/specsolve#1942). That's worth a separate PR.

ah wonderful! let's try that out too!

Converting the cached CSR matrix to CSC held a second copy of A during
the build, which CodSpeed flagged as ~20% more memory in to_solver.
HiGHS accepts the row-wise matrix directly: MILPs now use a row-wise
passModel, LPs addCols without entries followed by one row-wise
addRows. Build time stays at the level of the column-wise version,
LP peak memory is back at master's, and MILP peak memory drops ~40%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4nx8GJ8mSNvkTagja9dMj
@FabianHofmann FabianHofmann changed the title perf(highs): build the model with one column-wise passModel call perf(highs): build the model in bulk from the row-wise matrix Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants