Skip to content

Compile Reduce and Fold with a list, tuple or Boolean accumulator - #648

Open
enumeratio wants to merge 2 commits into
cortex-js:mainfrom
enumeratio:compile-fold-accumulators
Open

enumeratio wants to merge 2 commits into
cortex-js:mainfrom
enumeratio:compile-fold-accumulators

Conversation

@enumeratio

Copy link
Copy Markdown
Contributor

Reduce and Fold by a function literal now compile when the accumulator is a Boolean, a flat list of reals, integers or Booleans, or a tuple of scalars. That covers a Fibonacci pair, a constant-length array, an And/Or flag, a zero array of run-time length (Map(z -> 0, Range(1, n))), and a fold seeded with List() that appends to it. Each result equals evaluate(); an empty collection gives the initial value.

reduceBy() was a counted loop whose carried value had to be a real scalar. It is now the same loop with a carried value of the accumulator's kind, as the loop of a Loop carries a list local written whole: a Boolean is a boolean carried value; a list of a stated length, and a tuple always, is one carried value of that length, which every step must keep; a list of a run-time length is carried as its length and then the list, with the type of the list naming the length (mapLengths), as in loop(). An integer accumulator keeps the integer lane by the same trial lowering as for scalars, and widens to reals when a step gives reals. For List() the lane comes from the same trial (integer, then Boolean), and the initial value is an empty build. A fixed-length list whose step changes its length (a Join that grows it) is retried as a run-time one. A fold accumulator that the body reads at a run-time index is typed indexed_collection<number>, which Join refused; its binding is now marked as the carried accumulator and joinLists() accepts such a symbol only when its binding has that mark. An input of that type still declines, as before (tested). The bigint pair is checked against evaluate() past 2^53 (F(100)).

Still declining: a tuple whose step changes its length; a list of points or a matrix as accumulator; a list of a run-time length on GLSL and in the bigint mode (as for a list local written whole); the interval mode, which declines Reduce by a function altogether; and an empty List() whose element type the body does not fix. The carry logic mirrors loop() in lower.ts rather than sharing a helper, since fitWhole and wholeLocal are tied to Bindings; happy to factor it out if you prefer. A new bigint field in the lowering context carries the mode to reduceBy().

Tests are in test/compute-engine/compile-fold-accumulators.test.ts, each comparing the compiled run to evaluate(): the Fibonacci pair, a tuple, a constant array, a Boolean pair, the zero array, List() appends, And/Or, empty collections, and the declines. The existing fold, list-local, loop and IR profile tests still pass.

Fixes #647

@enumeratio

Copy link
Copy Markdown
Contributor Author

Measured on a combinatorics corpus (main 1044ec1 against this branch at 55c526c, every compiled answer re-checked against the interpreter):

main this PR
statistics compiled (of 1431) 657 822
family operations compiled (of 830) 306 700
maps compiled (of 299) 68 79
compile declines (events) 2775 1436

"a list where a scalar is required" (1378) and "Reduce from a value that is not a real number" (101) go to 0. Nothing that compiled on main declines or changes its answer. Of the newly compiled results, all but two agree with the interpreter, and in those two the interpreter is wrong: #655, with the fix in #656.

The next largest decline was a Join of the accumulator, or of a Drop of it, inside a typed let (Apply(Function(…, Typed(z, 'integer')), …)). There the accumulator is typed indexed_collection<integer>, so a Drop of it reads as list | string. 81df99b accepts the carried accumulator and its slices (Drop, Take, Rest, Most, Reverse) as list operands of Join, when the binding holds a flat list. A tuple that grows through Join still declines. Tests are in compile-fold-accumulators.test.ts.

@enumeratio
enumeratio force-pushed the compile-fold-accumulators branch from 81df99b to ce384c7 Compare November 21, 2026 23:48
@enumeratio

enumeratio commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor Author

For a sense of what the interpreted path costs in that corpus (0.194.x, where these decline with "a list where a scalar is required"):

  • The partition-number table p(0..160), built by two nested Folds over a list accumulator, takes 7.3 s interpreted. Every family's count and rank tables have this shape.
  • The count of semistandard tableaux of shape and bound 19 takes 293 s interpreted.
  • 510 of the 830 family operations run interpreted.

The tableaux count compiled on 0.187.1, through the older compiler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Reduce and Fold with a list, tuple or Boolean accumulator do not compile

1 participant