Skip to content

Rogue SPR: score leaf-pair candidates from parent split overlaps - #224

Open
ms609-agent wants to merge 8 commits into
mainfrom
claude/nifty-goldberg-089cff
Open

ms609-agent wants to merge 8 commits into
mainfrom
claude/nifty-goldberg-089cff

Conversation

@ms609-agent

Copy link
Copy Markdown
Collaborator

Follow-up to #223: extends the parent-overlap route from single-leaf to depth-2 pair Rogue candidates.

Change

  • SingleDrop → LeafDrop, with Prepare(i, j). Two removals are replayed on copies of the parent ledgers (leaf i once per outer loop, then j per candidate), recording the split that vanishes from each tree at each step. This covers cherries, sib(i) = par(j), the mirror case and root-child transitions without enumerating them. The root-child test is now structural (parent[parent[tip]] == parent[1]): after removing i, sib(i) can become the root's child while still owning a row.
  • Child overlap = parent overlap − popcount(row_has & col_has) on 2-bit masks. mutual_clustering_bounded_counts() dispatches on SplitCounts::pair to a template, so the depth-1 kernel is unchanged.
  • Shared-split test: parent split pairs whose polarized XOR is exactly two leaves are stored as sorted pair keys. A pair also falls back to ScoreWithout() if either leaf alone makes a shared split, if either leaf is leaf 1, or if the reduction predicate fires on the doubly-reduced ledgers.
  • Prices: when the first vanishing split's merge target vanishes second, the chain resolves to the second's target.

Results (min of 3 interleaved runs vs main, Windows, 1 core)

workload main this PR
rand30, depth 2 3.86 s 1.65 s (−57%)
mid80, depth 2 5.47 s 3.10 s (−43%)
near80, depth 2 0.39 s 0.21 s (−46%)
rand30 / rand80, depth 1 0.22 / 1.03–1.09 s 0.21 / 1.07–1.11 s (no change beyond noise)

SPRDist sums are identical on all workloads. 75% of incremental pair candidates are scored from overlaps. The rest are leaf 1 or disabled trees (14%), single-leaf shared splits (7%), pair-shared splits (1.6%) and reductions (1%).

Verification

  • 295 tree pairs (9–40 leaves, half near pairs), 70,142 pair candidates, built with -DDEBUG (ASSERTs on): argmin sets of single and pair score vectors are identical to incremental = FALSE. The largest difference among near-best scores is 7e-15.
  • test-tree_distance_spr.R passes (its Compare() already checks depth 2 against the rebuild path). All new C++ lines are covered (covr). Spelling is clean.

Note: a -DDEBUG build fails in nni_distance.cpp:212 (n_splits is undeclared in an ASSERT). This predates the PR and is unrelated.

🤖 Generated with Claude Code

ms609-agent and others added 3 commits October 23, 2026 09:50
Extends SingleDrop (now LeafDrop) to depth-2 candidates: removing leaves i
and j replays two removals on the parent ledgers, recording the split that
vanishes from each tree at each step; child overlaps are parent overlaps
less the popcount of shared membership of {i, j} (2-bit masks; the
single-leaf kernel is unchanged via a template).  Pairs fall back to
ScoreWithout() if either leaf is leaf 1, removing either leaf alone makes
two splits identical, the trees have splits differing in exactly {i, j},
or reduce_trees() would reduce the smaller trees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ms609-agent
ms609-agent requested a review from ms609 October 23, 2026 08:59
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.76%. Comparing base (2f6776d) to head (5f765ae).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #224      +/-   ##
==========================================
+ Coverage   97.74%   97.76%   +0.01%     
==========================================
  Files          60       60              
  Lines        6575     6628      +53     
==========================================
+ Hits         6427     6480      +53     
  Misses        148      148              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The 7-leaf NNIDist() bounds test (#222) called NNIDist() on all 945
trees: 446,040 pairs through CompareAll()'s R loop, c. 80 s locally and
over an hour on the R 4.1 leg, which then hit its 60-minute limit.  The
test now calls cpp_nni_distance() on pre-ordered edges (c. 7 s); the
6-leaf case still checks that NNIDist() agrees.

benchmark and rev-dep-check ran on ubuntu-latest, now 26.04, where
setup-r apt-installs ~175 packages and timed out at 15 minutes; their
RSPM serves noble binaries anyway, so pin ubuntu-24.04.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

This comment has been minimized.

ms609-agent and others added 2 commits October 29, 2026 07:16
On R 4.1, ape's [[.multiPhylo (class(x) <- NULL) copies the whole list
on each access.  CompareAll() built two multiPhylo objects with one
element per pair and indexed each once per pair, so its time grew with
the square of the number of pairs: 446,040 pairs of 7-leaf trees took
over 50 minutes on the R 4.1 CI leg, against c. 80 s on R 4.6, where
the access does not copy.  It now indexes a plain list of the trees by
pair indices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Performance benchmark results

Call Status Change Time (ms)
ClusteringInfoDistance(tr280) ⚪ NSD 1.16% 9.32, 9.36 →
9.32, 9.16
ClusteringInfoDistance(tr50) ⚪ NSD 0.83% 6.69, 6.68 →
6.66, 6.6
LAPJV(test2000) ⚪ NSD 2.32% 127, 130 →
125, 127
LAPJV(test40) ⚪ NSD -0.05% 0.0178, 0.0179 →
0.0179, 0.0178
LAPJV(test400) ⚪ NSD -2.73% 3.19, 3.25 →
3.21, 3.41
MutualClusteringInfo(tr280) ⚪ NSD -0.86% 18.4, 18.4 →
18.6, 18.7
MutualClusteringInfo(tr50) ⚪ NSD -0.67% 17.6, 18.1 →
18.2, 17.9
NNIMaxStep(2000, normalize = TRUE) ⚪ NSD 1.02% 0.214, 0.216 →
0.213, 0.213
NNIMaxStep(300, normalize = TRUE) ⚪ NSD -0.26% 0.0315, 0.0314 →
0.0314, 0.0316
NNIMaxStep(300) ⚪ NSD 0.9% 0.00896, 0.00893 →
0.00889, 0.00885
PathDist(postTrees) ⚪ NSD -7.05% 2.34, 2.29 →
2.49, 2.47
PhylogeneticInfoDistance(tr180) ⚪ NSD -0.12% 58.1, 58.6 →
58.4, 58.6
PhylogeneticInfoDistance(tr50) ⚪ NSD -0.09% 44.2, 43.8 →
44.2, 44
RobinsonFoulds(tr280) ⚪ NSD -0.61% 4.79, 4.79 →
4.8, 4.83
RobinsonFoulds(tr280) ⚪ NSD -0.57% 4.61, 4.66 →
4.67, 4.63
RobinsonFoulds(tr50) ⚪ NSD -0.76% 4.57, 4.59 →
4.6, 4.63

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants