Repository navigation
feat(engine): Aggregate on the streaming engine (#547 PR 2) - #645
Merged
Merged
Conversation
Aggregate queries and :with run on the engine: one hash-keyed group table (one entry when ungrouped, so constant memory), count and count-distinct on cells, other aggregates on values via aggregate::Agg; groups sorted by value and finished in that order. A group of floats Value's Ord finds equal (-0.0 and 0.0, NaNs) returns the least by value_order, on both engines, so the answer does not depend on the plan. Windows stay on the executor until PR 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…#473) 1200 -> 1230 KiB on v3 until #551 removes the executor's query path; #547 PR 2 brings the release binary to ~1205 KiB. Agreed with the maintainer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
Collaborator
Author
|
Binary size: 1,208 KiB in CI against the 1,200 KiB gate. Shared group finishing with the executor ( |
5 tasks
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 5 of #473 (#547), PR 2 of 3: plan steps 9–12 of
docs/superpowers/plans/2026-10-20-547-aggregate-window.md. Aggregates run on the streaming engine; windows stay on the executor until PR 3.What changes
engine/plan.rs): a:findwith aggregates (and:with) plans like any engine query, with a lastStep::Aggregate { groups, aggs, zero_row }.Plan::findnow names the column of each:findelement in the last step's rows. Window queries returnNone(executor).Aggregate(engine/aggregate.rs): aHashMap<Vec<JoinKey>, Group>keyed on the grouping cells (plain:findvariables, then:with) asValue'sOrdcompares values (JoinKey::group: no keyword→ref normalising,-0.0=0.0, every NaN one key). Ungrouped = one entry, constant memory.countcounts non-nilcells;count-distinctkeeps a set of cell keys; other aggregates convert the cell to aValueand useaggregate::Aggfrom PR 1. At end of input: keys → values, sort byValue'sOrd, finish in that order (first error fails,INT-042drops the group), emit batches of 1,024. Zero row for acount-only query with no plain variable, as the executor.Ord-equal but distinct (0.0/-0.0, NaN payloads) returned the first one the plan met, so engine and executor disagreed (found by the oracle). It now returns the least byvalue_order(-0.0when present). Confirmed with the maintainer. Docs site: v3: aggregate group value for 0.0 / -0.0 (#547) minigraf-docs#46.Deviations from the plan
plan::plandoes not callcheck_find_functions:open_queryalready calls it before planning.Cell::Valuevariant; aggregate results become cells throughCell::of_value(refs and keywords from registered aggregates get ids).countshape ([:find (count ?v) :where [?e ?a ?v]]); the envelope had onlycount_grouped.feat/547-engine-aggregate(the plan'sfeat/547-aggregatewas the merged spec PR's branch).Tests
engine::tests:aggregates_match_executor(21 shapes × 6 temporals + a:db/tx-countshape, in memory/after checkpoint/after reopen, via the oracle and routing counter),ungrouped_aggregate_returns_one_row(counters: rows_in 3000, one batch of one row; zero row; empty answers),grouped_aggregate_orders_groups_and_fails_in_a_later_group;other_queries_stay_on_the_executornow uses windows.tests/aggregation_test.rs::float_group_returns_least_value;tests/plan_independence_test.rs+2 float group shapes.--features test-oracle: 1623 passed, 0 failed. Codex review: no findings.Measurement (#394 envelope, same idle server,
v3234cc3e vs this branch, p50 / peak RSS)countcount_groupedEvery other shape within noise. Recorded in
docs/BENCHMARKS.mdandCHANGELOG.md.Refs #547 (closed by hand after PR 3), #473.
🤖 Generated with Claude Code
https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz