You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Show demo 1's query plans: DataFusion's joins against einfold's fused pipeline #297
🌀 Claude: benchmarks/ml/01_attention.py says einfold, a query optimizer that rewrites SQL plans into array algorithms, runs causal attention as FlashAttention does: tiled scores, a running softmax, and no stored N×N table. Today the demo shows this only as an operator count and an assertion that a fused pipeline ran. A picture of the plan would make the claim visible, and teach how a SQL engine runs the query.
Proposal: for each spelling of attention, and for the Nadaraya–Watson control, show:
DataFusion's plan: the joins and aggregates the SQL describes, where the scores are a table of N² rows;
einfold's plan for the same query: the fused pipeline that replaces them.
DataFusion can draw a plan as Graphviz: LogicalPlan.display_graphviz() in datafusion-python, and EXPLAIN for the physical plan. einfold would need to describe its pipelines in the same form. The output could be:
a --plans DIR option that writes the images;
or a figure in benchmarks/ml/README.md, or on the docs site.
Activity