Skip to content

streaming engine 5/9: Aggregate, Sort and Window (#473) #547

Closed

Description

@adityamukho

Part 5 of 9 of #473. Design: docs/superpowers/specs/2026-10-11-streaming-engine-design.md (branch docs/473-streaming-engine-spec), §6.2, §6.3.

  • Aggregate: constant-memory ungrouped; hash-grouped; sort-grouped on ordered input
  • Sort (skipped when index order suffices; top-k under Limit)
  • Window over partitioned, ordered input
  • Acceptance: count over 10M facts in constant memory

PRs for this part target v3 and must not use a closing keyword for #473.

Activity

  1. added this to the v3.0.0 milestone on Oct 11, 2026
  2. added
    enhancementNew feature or request
    rustPull requests that update rust code
    performanceBenchmark performance regressions
    on Oct 11, 2026
  3. adityamukho commented on Oct 20, 2026

    @adityamukho
    CollaboratorAuthor

    PR 2 (engine Aggregate) merged on v3 as #645 (58c19dd), docs site #46. Aggregate queries and :with run on the engine; a float group (0.0/-0.0, NaNs) returns its least value by value order (-0.0 when present) on both engines. Envelope at 10M: count 29.6 s → 4.46 s, 6.6 GiB → 7 MiB; count_grouped 4.51 s → 614 ms, 901 → 7 MiB. Binary-size gate temporarily 1,230 KiB until #551. Next: PR 3 (engine Window, plan steps 13–16).

  4. adityamukho commented on Oct 20, 2026

    @adityamukho
    CollaboratorAuthor

    Done on v3 in three PRs: #642 (the rules on the executor: value_order, exact sums, SQL peers, PRS-082), #645 (Aggregate on the streaming engine), #647 (Window on the streaming engine; edebea2). Follow-up: #641 (index-order tricks, v3.1.0).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceBenchmark performance regressionsrustPull requests that update rust code

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions