Skip to content

avg: correct v2 aggregate documentation and add ordinary aggregate support in v3 #777

Closed

Description

@adityamukho

Problem

Ordinary aggregate avg is unavailable through the Datalog API on both main and v3, despite examples and built-in aggregate lists in docs/ERROR_REFERENCE.md implying that it works. The fix has two release-line scopes: documentation only for v2; aggregate support plus documentation for v3.

Discovered while reviewing #641. This issue tracks the avg discrepancy separately from that input-order optimization.

Reproduction

(transact [[:a :n 10] [:b :n 20]])
(query [:find (avg ?v) :where [?e :n ?v]])

Actual on both branches:

[PRS-027] 'avg' is a window function and requires an ':over (...)' clause

The window form works:

(query [:find ?v (avg ?v :over (:order-by ?v)) :where [?e :n ?v]])

It returns running averages of 10 and 15 for values 10 and 20. Ordinary count, sum, min and max over the same data return 2, 30, 10 and 20.

Reproduced against exact source snapshots of main (50cff878ce1aa03da9773bfd9db492827af9d291) and v3 (2abf53df05447755490d017bc2daf89f4d59cdf5).

Evidence

  • Both parsers explicitly include avg in WINDOW_ONLY: main parser, v3 parser.
  • main has an average implementation in apply_builtin_aggregate, but the parser prevents ordinary Datalog queries from reaching it: helper.
  • On v3, Agg::new explicitly rejects Builtin::Avg with INT-029, while Running::Avg implements the window form. Removing the parser restriction alone is insufficient: aggregate constructor.
  • The v2 error reference recommends (avg ?score) as an aggregate expression: PRS-024. Its INT-029 description also lists avg as a built-in aggregate. On v3, PRS-022/PRS-024/QRY-010 contain similar claims.
  • The docs-site Datalog reference currently describes the actual behavior correctly: its compatibility table says avg requires :over, and its ordinary aggregate list is count, count-distinct, sum, sum-distinct, min, max. Those statements need version scoping when ordinary avg is implemented.

Requested work

v2 (main): documentation only

  • Correct misleading ordinary-aggregate examples and function lists, including PRS-024 and INT-029; audit the rest of the error reference and related documentation for the same discrepancy.
  • Clearly document that avg requires :over in v2 and that the six ordinary built-ins are count, count-distinct, sum, sum-distinct, min and max.
  • Preserve v2 query behavior. Do not backport the new aggregate feature or remove the parser restriction on this line.

v3: ordinary aggregate avg plus matching documentation

  • Support (avg ?v) for ungrouped and grouped queries; the reproduction above should return a Float average of 15.
  • Implement the shared aggregate/engine path, not just parser acceptance. Reuse v3's exact numeric summation infrastructure and preserve existing window-average behavior.
  • Specify and test result types, nil/nonnumeric handling, empty inputs/groups, mixed integers/floats, large integers, NaNs/infinities and order-independent results before implementation.
  • Cover prepared queries, cursors, rule queries and plan independence; preserve the existing grouping/:with rules and existing window behavior.
  • Update parser/grammar tests, including the current invalid window_only_func_without_over.edn fixture. Preserve rejection of genuinely window-only functions without :over.
  • Synchronize repository documentation and the docs-site compatibility table/aggregation reference, with version annotations that preserve the v2 behavior and identify the v3 release introducing ordinary avg. Do not advertise support before it is implemented.
  • Coordinate shared accumulator changes with perf(query): extend ordered Aggregate beyond one-key count groups #781. perf(query): extend ordered Aggregate beyond one-key count groups #781 is blocked by this issue for its avg optimization decision; work on existing aggregates can proceed independently.

Boundaries

No new dependencies, configuration, storage/WAL format changes or v2 runtime behavior changes. The v3 release milestone should be chosen explicitly during planning. Use the bench machine for heavy verification; obtain Claude spec/plan reviews before implementation and a code review before opening PRs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or requestrustPull requests that update rust code

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions