You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Test suite ahead of the ete3 → ete4 migration - #112
Draft: builds a test safety net before migrating from ete3 to ete4. Opened as a draft so that CI runs; this PR only adds tests and CI changes, no changes to FastOMA/.
What's included
pytest infrastructure and a new python-tests workflow; slow tests run nightly.
nf-test snapshots of the inferred orthology instead of line counts, plus tests for the id transformer, MSA/tree output, score flags, archive input and splice files.
Unit and stage tests for rootHOG inference, check-input, batching, subHOG inference (HOG class, level processor, MSA filters, rooting, fragments), collect-subhogs, pairwise relations and the report inputs.
fungi-30 fixture (tests/data/fungi30): 23 rootHOGs selected from a full fungi-30 run.
13 golden snapshots (nesting, TaxRange, scores), identical to the fungi-30 run.
10 rootHOGs checked for "no protein dropped". In the full run, 414 of 22,878 rootHOGs dropped 8,113 proteins.
The snapshots need mafft 7.505 / FastTree 2.1.11, so CI is pinned to ubuntu-24.04 and fails instead of skipping on a version change (FASTOMA_REQUIRE_GOLDEN=1).
Known bugs
Known bugs are pinned as xfail(strict=True) with a file:line reason (pytest -ra lists them). The golden snapshots record current behaviour, including the wrong ImpliedLosses values. They are a regression guard, not a correctness reference.
- tests/conftest.py: conf factories for infer-subhogs/infer-roothogs built from
the real CLI defaults, HOG id counter reset, tmp-dir and hash-seed fixtures,
and a 'tools' marker that skips tests when external binaries are missing
- tests/fastoma_testutils.py: compare outputs by meaning (group partitions,
pair sets, canonical orthoXML) and validate against the OrthoXML 0.5 schema
- vendor orthoxml-0.5.xsd from qfo/orthoxml
- pytest config in pyproject.toml (testpaths, markers, strict xfail)
- new python-tests workflow; slow tests run nightly
- remove the empty test_roothog_example.py stub
The default pipeline test only pinned file names and line counts, so any change
to the HOGs went unnoticed. It now snapshots order- and id-independent views of
the results (rootHOG and OG memberships, ortholog pairs, canonical nesting of
the orthoXML groups) and asserts that no protein is referenced twice. The
helpers live in nf-tests/lib/FastomaOutput.groovy for reuse in further tests.
…chive input and splice files
- UniProt id transformer applied to TSVs and orthoXML, MSAs and gene trees
published, ImpliedLosses/TCS scores omitted when disabled
- a .tar.gz input with precomputed hogmaps is extracted and the hogmaps are
used without re-running omamer
- a splice file makes exactly one isoform of the gene end up in the HOGs and
main_isoform attributes are written
…tching
Covers rootHOG grouping (singleton resolution, merge candidates, HCS clustering,
merge_rhogs2, big-rootHOG filtering), hogmap/proteome/splice parsing and
writing, fastoma-check-input helpers, batch_roothogs and both UnionFind copies.
test_roothog_stage.py runs fastoma-infer-roothogs on testdata and on a synthetic
dataset that reaches the singleton, merge, big-rootHOG, splice and linclust
branches, and checks that every input protein is accounted for.
Known bugs are pinned with strict xfail markers, e.g. merge_rhogs2 losing
proteins when a cluster overflows big_rhog_size and proteins without hogmap
rows vanishing silently.
…mers and report inputs
output_fixtures.py builds collect-subhogs inputs by hand (4 species, nested and
root-level duplications, a fragment group). Tests pin the HOG id scheme,
orthoXML header/genes/scores, RootHOG/OG TSV and FASTA contents (as row sets),
the ortholog/paralog pairs, the UniProt transformer and the orthoXML fields the
report notebook reads.
Strict xfails mark schema violations (single-child root wrapper,
main_isoform attribute), HOG numbering depending on directory listing order,
and the report failing without CompletenessScore.
Unit tests for the HOG class (construction, split_hog, orthoXML shapes,
representatives), LevelHOGProcessor (merging, reconciliation, early return,
root wrapping), the MSA filters, gene tree rooting, species tree preparation
and fragment detection.
test_subhog_stage.py runs fastoma-infer-subhogs on four real rootHOGs
(tests/data/subhog, species trees pruned to the needed species) and checks
that no protein is lost or duplicated, TaxRanges nest along the species tree
and equal the members' MRCA, the output is schema-valid, and sequential and
--parallel runs agree. D0625237 reproduces three known bugs (split_hog losing
members, wrapper TaxRange not the MRCA, single-child wrapper), pinned as strict
xfails like the other bugs found in this stage.
…HOGs
23 rootHOGs (about 1,700 proteins) selected from a full fungi-30 run by the code
paths they reach in fastoma-infer-subhogs (conflict resolution, split_hog, MSA
trimming, fragments, deep nesting, --parallel) and by whether they drop proteins.
- golden (13): the per-rootHOG output (nesting, TaxRange, scores) must equal a
text snapshot in tests/data/fungi30/golden. The snapshots were generated with
the working-tree code in the dessimozlab/fastoma:dev image and are identical
to the fungi-30 run. They pin current behaviour, including the wrong
ImpliedLosses values.
- drops (10): fastoma-infer-subhogs drops proteins of these rootHOGs (in the
fungi-30 run, 414 of 22,878 rootHOGs dropped 8,113 proteins). Checked only
for "no protein dropped" as strict xfail, because the number of dropped
proteins changes with small MSA differences.
The output depends on the mafft/FastTree versions, so the golden tests skip in
other environments. CI is pinned to ubuntu-24.04 (same mafft 7.505 / FastTree
2.1.11 as the image) and sets FASTOMA_REQUIRE_GOLDEN=1 to fail instead of skip.
Regenerate with `python tests/fungi30.py regenerate` (needs docker).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The archive was created with macOS tar, which adds a ._<name> metadata file
for every entry. macOS tar hides them again when extracting, but on Linux they
become real files: check-input then saw ._AQUAE.fa etc. as proteomes that are
missing from the species tree, and the archive-input test failed in every CI
configuration. Same content, with normalised owner and timestamps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Archives made with macOS tar contain a ._<name> AppleDouble file per entry, and
proteome folders can contain .DS_Store. check_proteome_files, check_hogmap_files
and parse_proteomes take them as species, so check-input stops with "._AQUAE is
not in the species tree" (this is what broke the archive-input nf-test on Linux).
Pinned as strict xfails until fixed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_count_untouched_clades compares species tree nodes by identity, but the
pipeline scores against full_species_tree (a copy) and every taxonomic level is
pickled separately, so a HOG and its subHOGs point into different tree copies.
A complete family then gets one extra loss per nested level (22 instead of 0 for
the 1:1 fungi-30 family F0848294). Strict xfail, in-memory and pickled per level.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The archive-success test fails against the unchanged pipeline, while snapshot regeneration can fail on root-owned Docker artifacts or incorrectly associate unchanged snapshots with new tool versions.
…ersions
- run the docker container with the caller's uid/gid (HOME=/tmp, no bytecode),
so a Linux docker daemon doesn't leave root-owned files in the repository
that make the temporary directory cleanup fail
- a partial regeneration (--tier fast|slow) with tool versions different from
ENVIRONMENT.json is rejected, since it would leave the other tier's snapshots
from the old tools while claiming the new ones (with a unit test)
Regenerating the fast tier with these changes leaves all snapshots unchanged.
Addresses review comments on #112.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Unquoted input and species-tree paths passed to sh -c cause snapshot regeneration to fail when the checkout path contains spaces.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Quote paths passed to the Docker shell command
tests/fungi30.py:215
The input and species-tree paths are joined without shell quoting, but _docker_run passes this string to sh -c. A checkout under a path such as /tmp/FastOMA checkout therefore splits both paths into extra arguments, causing snapshot regeneration to fail at argument parsing. Build the command with shlex.join over the complete argument list, passing RUNNER directly rather than repr(RUNNER).
regenerate() joined the infer-subhogs command with plain spaces before passing it
to `sh -c` in the container, so a checkout path containing a space was split
into several arguments. Build it with shlex.join (container_command), with a
test that lets sh parse the command for such a path. Regenerating the fast tier
leaves all snapshots unchanged.
Addresses a review comment on #112.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot's point about unquoted paths passed to sh -c in tests/fungi30.py is fixed in 768c468: the container command is now built with shlex.join (container_command(), passing RUNNER directly), and test_container_command_survives_paths_with_spaces checks how sh parses it for a path containing a space. Regenerating the fast tier with the change leaves all snapshots unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft: builds a test safety net before migrating from ete3 to ete4. Opened as a draft so that CI runs; this PR only adds tests and CI changes, no changes to
FastOMA/.What's included
python-testsworkflow; slow tests run nightly.tests/data/fungi30): 23 rootHOGs selected from a full fungi-30 run.ubuntu-24.04and fails instead of skipping on a version change (FASTOMA_REQUIRE_GOLDEN=1).Known bugs
Known bugs are pinned as
xfail(strict=True)with a file:line reason (pytest -ralists them). The golden snapshots record current behaviour, including the wrong ImpliedLosses values. They are a regression guard, not a correctness reference.