[ocots]

Refs #409 (part 1 of 2). Conventions: ct-conventions v0.10.0, ct-agents v0.4.0.

Plan

Goal. A reference solution says how it was obtained (#409). Part 1 makes the tests record it; part 2 (after the crons have run on main with this change) stores it in ext/MetaData/references/ and exposes it as typed references.

Why two parts. A cron dispatched on a branch publishes to results and updates the cron issues (#348, #350, #351 came from such a run). The references of part 2 must be computed from runs on main that already record the new fields.

Steps (this PR).

1. One function per solve module gives the options of a solve (SolveOptimalControl.solver_options, SolveJuMP.solver_options); the solve uses it, so what is recorded is what ran. settings(config; options) gives modeler, solver, options for result!.
2. outcome returns the constraint violation: OptimalControl.constraints_violation(sol); JuMP: the largest entry of JuMP.primal_feasibility_report.
3. TestStatus.result! records them; <report>-results.tsv gets four columns after the existing ones: modeler, solver, options (inline TOML), violation. The Markdown table is unchanged.
4. Each run with solves writes <report>-environment.toml: Julia, OS, architecture, CPU, threads, commit, run URL, versions of the direct dependencies and of KEY_DEPENDENCIES (CTDirect, CTModels, CTSolvers, ADNLPModels, ExaModels, MadNLP, NLPModelsIpopt, Ipopt_jll, MUMPS_seq_jll).
5. A core test of the recording (test/suite/core/test_status.jl), shown failing on a broken case.
6. test/README.md.

Out of scope (part 2). tools/references/update.jl, the loader, AbstractReference and the API, regenerated reference files.

Done when (part 1 of #409)

• Each row of *-results.tsv has modeler, solver, options, violation; each run writes *-environment.toml; shown on a local run of one problem per modeling (below).
• CI green.
• After merge: the four crons dispatched on main (CronOptimalControl, CronJuMP, CronOptimalControlScalarized, CronRobustness), run ids recorded in [Metadata] Reference solutions with how they were obtained: typed references and API #409.

Evidence

Local runs (macOS, Julia 1.12.7), in the worktree:

julia --project=. -e 'using Pkg; Pkg.test(; test_args=["OptimalControl", "OptimalControlScalarized", "JuMP", "cron", "problem=beam,moonlander"])'
OptimalControlProblems tests           |    3      3  2m04.3s
  environment OptimalControl           |    1      1  1m10.6s
  environment OptimalControlScalarized |    1      1    34.2s
  environment JuMP                     |    1      1    19.0s

julia --project=. -e 'using Pkg; Pkg.test(; test_args=["OptimalControl", "JuMP", "robustness", "problem=beam"])'
OptimalControlProblems tests |    2      2  58.7s

Rows written (problem, configuration, reason, then the new columns):

beam  modeler=exa solver=madnlp barrier=loqo  SOLVE_SUCCEEDED  exa  madnlp  {print_level = "ERROR", tol = 1.0e-8, max_iter = 1000, max_wall_time = 500.0, barrier = "loqo"}  3.0088246754126843e-14
beam  modeler=adnlp solver=ipopt mu_init=0.001  first_order  adnlp  ipopt  {print_level = 0, tol = 1.0e-8, mu_strategy = "adaptive", sb = "yes", max_iter = 1000, max_wall_time = 500.0, mu_init = 0.001}  1.1102230246251565e-16
beam  solver=ipopt  LOCALLY_SOLVED  (no modeler)  ipopt  {tol = 1.0e-8, max_iter = 1000, mu_strategy = "adaptive", linear_solver = "mumps", max_wall_time = 500.0, sb = "yes"}  9.999721425901775e-9

OptimalControl-cron-environment.toml:

arch = "aarch64"
commit = "c3ade6832e8d4b318b88f31ed5760077719e4fbf"
cpu = "Apple M3 Pro"
julia = "1.12.7"
os = "Darwin"
run = "local"
threads = 1

[packages]
ADNLPModels = "0.8.15"
CTDirect = "1.1.8-beta"
CTModels = "0.19.8"
CTSolvers = "0.5.8"
ExaModels = "0.12.2"
Ipopt_jll = "300.1400.1902+0"
MadNLP = "0.10.3"
OptimalControl = "2.2.1-beta"
...

All 16 options cells of the run parse back with TOML.parse. tools/cron/report.py reads the new files unchanged (it reads the TSV by column name): checked on these reports.

The new test fails on a broken case. With value(x) = string(x) in inline_toml (strings not quoted), Pkg.test(test_args=["*status*"]) fails: how a solve was configured is recorded (#409): Error During Test ... test_status.jl:79 (the TOML does not parse). Restored: 29 passed.

Choices

1. Options recorded as given to the solver, before conversion. MadNLP's mu_init and barrier=:loqo are turned into a barrier object by SolveOptimalControl.barrier; the record keeps mu_init / barrier = "loqo", which is what a user writes and what reproduces the solve.
2. Options serialized when the result is recorded, not when the report is written. The report is written from the finally of run_environment.jl, a top-level expression started before the solvers were loaded: printing a MadNLP.LogLevels there fails on the world age (MethodError: no method matching namemap(::Type{MadNLP.LogLevels}), seen on the first run). result! runs inside the tests, in the newest world.
3. Inline TOML in a TSV cell rather than a hand-written k=v format: read back with the standard library, types kept (1000 stays an integer, 1.0e-8 a float).
4. The violation is what each modeling measures. JuMP's report includes the variable bounds, which Ipopt relaxes by about 1e-8 (bound_relax_factor): beam shows ~1e-8 with JuMP and ~1e-16 with OptimalControl. Recorded as is, documented in test/README.md; comparing it across modelings is not meaningful.
5. KEY_DEPENDENCIES, a short list of package names, because the direct dependencies of a test environment miss CTDirect, ExaModels, ADNLPModels, MadNLP's version through OptimalControl… The full manifest of a cron run belongs to [Doc] Results pipeline for the documentation, and a gallery of problems #314 (env/<run id>/ on results).
6. Columns added after the existing ones, Markdown table unchanged: the cron report and the docs (docs: rewrite the documentation, problem pages with one tab per modeling #408) read by column name; nothing they read changes.
7. Formatting. main is not fully formatted ([AUTO] JuliaFormatter.jl run #333 is open): running JuliaFormatter on the touched files rewrites unrelated lines, so only the new code is in the blue style; the formatter run of [AUTO] JuliaFormatter.jl run #333 will cover the rest.

Labels

run ci github-runner, test convergence (runs the cron/ tests changed here). No label runs robustness/ on a pull request: checked locally above.

Main checkout

git -C OptimalControlProblems status --short before: empty; after: empty.

🤖 Generated with Claude Code