Skip to content

refactor: split src into submodules, keeping the public API (#413) - #421

Open
ocots wants to merge 3 commits into
mainfrom
chore/413-src-submodules
Open

ocots wants to merge 3 commits into
mainfrom
chore/413-src-submodules

Conversation

@ocots

@ocots ocots commented Nov 7, 2026

Copy link
Copy Markdown
Member

Closes #413. Conventions: ct-conventions v0.10.0 (MOD1, MOD2, MOD5–MOD8), ct-agents v0.4.0.

Plan

Goal. Split src/OptimalControlProblems.jl (683 lines) into submodules, one per responsibility, the package file only declaring; keep the public API unchanged (decisions recorded in #413).

Layout, in dependency order (MOD7):

Submodule Files Exports
Modelings backends.jl AbstractModelBackend, the three backends
Metadata discovery.jl, solutions.jl, loading.jl, access.jl problem_metadata, grid_size_data, parameters_data, initial_guess_data
References objectives.jl reference_objective, reference_solution_data, solution_kind, SAME_SOLUTION
Catalogue selection.jl, functions.jl problems, status, the problem functions
Problems aliases.jl, problem.jl Problem, model, initial_guess, metadata, solution, features, the accessors

Steps.

  1. Move the code, unchanged except for qualification (Metadata.problem_metadata(…), MOD6); problem.jl and solutions.jl moved with git mv.
  2. Package file: includes, using .<Name>, the same exports as before.
  3. Qualify every internal used outside src/ through its submodule: extensions (Catalogue._warn_if_draft, Metadata._initial_guess, Problems._problem in every problem file), tests, tools/, docs/.
  4. docs/api_reference.jl: the modules and their files are read from the package, not listed.
  5. Test of the layout: test/suite/core/test_modules.jl.
  6. src/README.md; paths in ext/README.md.
  7. Full test suite, documentation build, CI.

Done when (#413)

  • src/OptimalControlProblems.jl only declares.
  • Each submodule follows MOD1/MOD2 and exports its public part.
  • names(OptimalControlProblems) unchanged (evidence below), internals reached through their submodule.
  • Extensions, tools/, test/, docs/ updated; test suite and documentation build green.
  • src/README.md.

Evidence

Public API unchanged. names(OptimalControlProblems) and the qualified functions, recorded on main before the move and after (diff of the two outputs: identical):

JuMPBackend OptimalControlBackend OptimalControlProblems OptimalControlScalarizedBackend Problem balanced_field beam … variable_dimension   (47 names)
---qualified status true features true reference_objective true solution_kind true reference_solution_data true initial_guess_data true SAME_SOLUTION true AbstractModelBackend true

Full local suite (macOS, Julia 1.12.7, Pkg.test()): every environment green.

  environment JuMP                     |    1      1     56.5s
  environment OptimalControl           |    1      1   9m41.1s
  environment OptimalControlScalarized |    1      1   5m36.4s
  environment comparison               |    1      1  36m00.9s
    test_modules.jl                    |  208    208      0.2s

The first run had three failures in core, fixed in the commit: two comments quoted a problem name (no hand-written list), and test_discovery_cache.jl patches the include_dependency lines, now in src/Metadata/discovery.jl and src/Modelings/backends.jl. Rerun: test_consistency.jl, test_discovery_cache.jl, test_modules.jl pass.

The layout test fails on a broken case. With defined_here() = 1; export defined_here added to the package file:

the package defines nothing, it re-exports: Test Failed at test/suite/core/test_modules.jl:49
  Expression: owner(name) in modules
   Evaluated: OptimalControlProblems in Module[OptimalControlProblems.Catalogue, …]

Restored: 208 passed.

Choices

  1. References fills the field :reference_objective of the metadata once Metadata is loaded; before, the metadata loop read the reference files. The field is declared as generated by tools in Metadata.METADATA_SCHEMA; moving the reading to References keeps the dependencies one-way (MOD7: References uses Metadata, not the converse).
  2. The analytic solutions (solutions.jl) are in Metadata: the metadata files call them (:reference_solution), and they are included into Metadata. References reads them through reference_solution_data.
  3. Sibling imports by module name only (using ..Metadata: Metadata) and qualified calls (Metadata.problem_metadata(…)): MOD5, MOD6.
  4. Exports. Each submodule exports its public API; the package re-exports the same names as before (decision 1 of [Dev] Split src into submodules (ct-conventions MOD1, MOD2), keeping the public exports #413). The other public functions (status, features, reference_objective, solution_kind, reference_solution_data, initial_guess_data, SAME_SOLUTION) are now exports of their submodule: reachable as before as OptimalControlProblems.status, and listed in the public part of the API reference.
  5. The problem functions are exported with eval(Expr(:export, Metadata.PROBLEM_NAMES...)), in Catalogue and in the package: the list is the files of ext/MetaData/, never written (MOD2 allows only declarations in a manifest; this is the declaration of a generated list).
  6. No list in docs/api_reference.jl: the submodules are read with names(O; all=true), their files with readdir(src/<Name>). The extensions keep their list (out of scope).
  7. File names never equal the module name in another case (backends.jl, objectives.jl, not modelings.jl, references.jl): on macOS and Windows file names ignore case, and modelings.jl overwrote Modelings.jl during the work. Written in src/README.md.
  8. The test of the API checks a structure, not a frozen list: every export of the package is defined in a submodule (Base.binding_module), every problem function is exported, every export of a submodule is reachable from the package. A frozen list of names would be hand-written; the unchanged API is shown above instead.

Order

Part 2 of #409 (typed references) is stacked on this branch: its types go into src/References/.

Labels

run ci github-runner (scope: everything, src/ changed), run documentation (the API reference is built from the submodules).

Main checkout

git -C OptimalControlProblems status --short: empty before and after.

🤖 Generated with Claude Code

Modelings, Metadata, References, Catalogue and Problems, each in src/<Name>/<Name>.jl
(ct-conventions MOD1, MOD2); the package file only includes them and re-exports the same
names as before. Internals are reached through their submodule: the extensions call
Catalogue._warn_if_draft, Metadata._initial_guess and Problems._problem. The API reference
of the documentation reads the submodules and their files from the package. A core test
checks the layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ocots ocots added run ci github-runner Trigger the CI workflow on GitHub-hosted runners run documentation Trigger the Documentation workflow on this PR labels Nov 7, 2026
Documenter resolves an unqualified @ref in the module of the docstring: Problem and
reference_solution_data are no longer in Metadata.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ocots

ocots commented Nov 7, 2026

Copy link
Copy Markdown
Member Author

Documentation build failed, fixed in 80e191f. VitePress refused two dead links (./@ref in api/api_public.md): Documenter resolves an unqualified @ref in the module of the docstring, and Problem and reference_solution_data are no longer in Metadata after the split. The four cross-references between submodules are now qualified ([Problem](@ref OptimalControlProblems.Problems.Problem)…). The local build had failed at the same step without printing the cause.

Conflict in src/OptimalControlProblems.jl: the fingerprint of #314, added on
main to the old monolithic file, moves to the Catalogue submodule
(Catalogue._fingerprint); its callers are qualified accordingly.
@ocots
ocots marked this pull request as ready for review November 7, 2026 22:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run ci github-runner Trigger the CI workflow on GitHub-hosted runners run documentation Trigger the Documentation workflow on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Dev] Split src into submodules (ct-conventions MOD1, MOD2), keeping the public exports

1 participant