Skip to content

Query parser silently discards trailing input after a complete form (garbage or documented clauses alike) #305

Closed

Description

@adityamukho

Summary

minigraf's Datalog query parser does not require the full input string to be consumed. If the input begins with a syntactically complete, valid form (e.g. a [:find ...] vector), everything after it is silently discarded — whether that trailing content is garbage tokens, unbalanced brackets, or an actual documented clause like :max-results N. No error is raised in either case, and no indication is given that anything was ignored.

This is dangerous because it fails silently in exactly the case a user would expect it to matter most: appending a documented result-limiting clause to a query against a large database, expecting it to take effect.

Reproduction

Environment: minigraf graph with ~6,300+ ingested commit entities (real-world arangodb repo ingestion via temporal_reasoning).

1. Valid query + pure garbage trailing tokens — no error, garbage silently ignored:

[:find ?e :where [?e :entity-type :type/commit]] garbage-trailing-tokens ) ] } 12345

Result: executes successfully as if the trailing garbage weren't there, returning full unfiltered results (267,775 characters).

2. Valid query + documented :max-results 5 clause — silently has no effect:
Per the Datalog Reference wiki, :max-results N "caps the maximum number of result rows returned" (with the caveat that it only applies inside the rule evaluator for recursive queries — see companion issue about the resulting practical gap). Regardless of that scoping caveat, one would expect either (a) the clause to be honored, or (b) a clear rejection if it's not applicable to this query shape. Instead:

[:find ?e :where [?e :entity-type :type/commit]] :max-results 5

Result: 267,943 characters — statistically identical in size to the ungated query, confirming the clause was parsed-and-discarded rather than applied or rejected.

Control — an input that does NOT start with a valid form correctly errors:

totally not even valid edn at all ] } )

Result: {"ok": false, "error": "msg='Unexpected token: RightBracket'"} — correct behavior, included here to show the parser can detect malformed input, it just doesn't apply that same rigor once a valid leading form has already been parsed.

Impact

  • Users cannot trust that any query modifier/clause after the main [:find ...] form actually took effect — it may be silently swallowed.
  • Typos or malformed trailing clauses in production queries fail silently instead of surfacing as errors, making bugs in calling code much harder to catch.
  • Directly caused two separate multi-hundred-KB response blowups against a real production-scale graph in this session, because the natural mitigation (append :max-results N) had no effect and gave no indication it hadn't worked.

Suggested fix

The parser should either:

  • Require full consumption of the input (raise a syntax error on any unconsumed trailing tokens), or
  • Explicitly define and document the full grammar for trailing clauses, and reject anything that doesn't match it.

Either would have surfaced this immediately instead of silently returning misleading "success."

Related

Filed alongside a companion issue: no working pagination/limit mechanism exists for plain (non-recursive) :find queries, which is what makes this parsing gap especially costly in practice.

Activity

  1. modified the milestones: , v1.3.0 on Aug 24, 2026
  2. added a commit that references this issue on Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions