No working result-limiting/pagination mechanism for plain (non-recursive) :find queries #306

Summary

There is no way to bound the result size of a plain [:find ...] Datalog query. The only documented capping mechanism, :max-results N, is explicitly scoped to "inside the rule evaluator" and the reference states outright that "for non-recursive queries results are not truncated by this value." In practice this means any :find query over a large graph can return an unbounded, arbitrarily huge result set with no way for the caller to prevent it.

This is a real, repeatedly-hit problem on production-scale graphs, not a theoretical one.

Reproduction

Graph under test: ~6,300+ commit entities ingested from the arangodb repo (52,665 commits total, ingestion in progress — this is a small fraction of the eventual graph size).

A simple, reasonable query:

[:find ?e :where [?e :entity-type :type/commit]]

returns a 267,775-character single-line result — already large enough to exceed the calling client's token budget on this partial graph. The graph is only ~12% ingested at time of testing; the fully-ingested graph will be significantly larger.

Attempting to bound it with the documented :max-results 5 clause has no effect (see companion issue #305 — the clause is silently parsed-and-discarded for this query shape rather than applied or rejected):

[:find ?e :where [?e :entity-type :type/commit]] :max-results 5

Result: 267,943 characters — no meaningful reduction.

There is no :limit, :offset, cursor, or streaming mechanism documented or discoverable for plain :find queries.

Impact

• Any caller working with a moderately large graph must either already know the exact narrow shape of every query in advance (to avoid ever selecting a high-cardinality relation), or risk queries that fail entirely due to oversized responses.
• This makes exploratory/ad-hoc querying — one of the core use cases of a queryable graph memory — impractical once a graph reaches realistic production scale (tens of thousands of entities, as seen here).
• Callers have no graceful degradation path: a query either fits, or hard-fails with no partial results.

Suggested fix

Add a genuine result-limiting mechanism for plain :find queries — e.g. a :limit N (and ideally :offset N or a cursor) clause honored directly by the top-level query executor, independent of the rule evaluator's :max-results. This is table-stakes for any query interface expected to work against non-trivial data volumes.

Related

Companion issue #305: the parser silently discards trailing clauses/tokens after a valid query form instead of erroring, which is part of why this gap is easy to hit without realizing it (a caller may reasonably believe :max-results N is working when it silently isn't).