Skip to content

Window partition and sorting state bypass query_memory_limit #765

Closed

Description

@adityamukho

Problem

src/query/datalog/aggregate.rs::windows builds a BTreeMap<&Value, Vec<usize>> of partition members for each window. The map entries and row-index vectors are not charged to OpenOptions::query_memory_limit, and Vec::push grows them infallibly. keyed.sort_by may also allocate a merge buffer without charging it or returning QRY-012 on allocation refusal.

This working state coexists with the Window operator's held input and any earlier result columns. PR #764 fixes the separate result-column gap reported in #761; it does not account for this partition/sort memory. A query with many distinct partition keys can therefore hold materially more memory than its configured limit.

Expected

  • Partition membership and sorting scratch memory count toward the query limit for as long as they are held, including overlap with input values and earlier result columns.
  • Allocation refusal in this path returns QRY-012 instead of relying on infallible growth.
  • Temporary charges are released as each window completes and on errors.
  • A regression uses many distinct partitions and a tight memory limit; another case covers sorting a large partition and verifies unchanged row order.

This concerns query resource accounting, not query answers or stored data.

Activity

  1. added
    bugSomething isn't working
    rustPull requests that update rust code
    on Oct 26, 2026
  2. added a commit that references this issue on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingrustPull requests that update rust code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions