Skip to content

feat(engine): window functions on the streaming engine (#547 PR 3) - #647

Merged
adityamukho merged 6 commits into
v3from
feat/547-window
Oct 20, 2026
Merged

adityamukho merged 6 commits into
v3from
feat/547-window

Conversation

@adityamukho

@adityamukho adityamukho commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Part 5 of #473 (#547), third and last PR: plan steps 13–16 of docs/superpowers/plans/2026-10-20-547-aggregate-window.md. #547 will be closed by hand after merge.

What

  • aggregate::windows — the executor's per-partition window core (partition by Value's Ord, sort by :order-by under value_order, then named variables, then position; SQL peers; rank/row-number) moved into aggregate.rs over &[Vec<Value>]. apply_window_functions now calls it, so both engines run the same code.
  • engine/window.rs — Window holds each input row's passed-on cells and its named variables' values, computes every window at end of input, returns rows in input order.
  • Routing (plan.rs) — Step::Window { find, named, keep } after the :where steps, or after Step::Aggregate for mixed queries (named variables read from the aggregate's group columns, PRS-082 guarantees they are there). With windows, :with does not group, as on the executor.
  • A stream ending in a Window keeps its DICT lookups until it ends (see Measurements).
  • Envelope gains a window shape; CLAUDE.md, CHANGELOG, BENCHMARKS, counts.

No answer changes (test-oracle), so no docs-site PR.

Tests

  • windows_match_executor: every built-in window over each value kind, partitions by keywords/refs/booleans/floats/long strings, :desc, mixed aggregate + window, :with, repeated :find variable, or-bound order key, pseudo-attribute; six temporal variants; across checkpoint and reopen.
  • window_returns_every_row_in_input_order (3,000 rows over several batches, counters, whole-partition sums), window_routing.
  • cargo test: 1,623 (1,613 passing, 10 ignored). cargo test --features test-oracle: all pass.

Measurements (#394 envelope, A/B in parallel on two sockets, nodes swapped between rounds)

window 100K 1M 10M
p50 v3 → PR 41.5 → 32.8 ms 601 → 434 ms 7.89 → 5.65 s
peak RSS 19 → 16 MiB 140 → 109 MiB 1,257 → 1,030 MiB

Every other shape unchanged (within noise; their paths are untouched). The first measurement was 17% slower at 100K (48.2 ms): Window turns each row's ids into values in its first pull, and the projection looked them up again because the DICT cache is dropped after each pull. Fixed in 06af6d2: a stream that ends in a Window keeps its DICT lookups until it ends (peak memory unchanged: the first pull already holds them); window_stream_keeps_dict_cache_until_it_ends fails without it.

Binary size

+6 KiB (local rustc 1.94: v3 1,240 → 1,246 KiB; CI's stable measured ~1,205 KiB for v3, gate 1,230 KiB until #551).

Review

Codex (read-only) over the diff: no findings; its coverage note led to the extra shapes in 1cc8ff8.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz

adityamukho and others added 4 commits October 20, 2026 15:43
Window holds each input row's passed-on cells and its named variables'
values, then computes every window with aggregate::windows, the code the
executor now calls too, and returns the rows in input order. The planner
routes window queries and windows over an aggregate's groups; with
windows, :with does not group, as on the executor. Envelope gains a
`window` shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
…nd order key, a pseudo-attribute (#547)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
)

Window turns every row's ids into values in its first pull; the rows it
returns later read the same ids again. Clearing the cache after each pull
made each lookup twice. Peak memory is unchanged: the first pull already
holds every lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.42857% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/query/engine/window.rs 91.07% 5 Missing ⚠️
src/query/engine/filter.rs 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

adityamukho and others added 2 commits October 20, 2026 17:23
@adityamukho
adityamukho merged commit edebea2 into v3 Oct 20, 2026
25 checks passed
@adityamukho
adityamukho deleted the feat/547-window branch October 20, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant