test(grammar): bounded-exhaustive parser vs grammar.pest verifier (#643 PR 3) - #652

PR 3 of #643 (plan docs/superpowers/plans/2026-10-20-643-grammar-verifier.md, steps 6–13): the verifier. Part of #643; does not close it.

What

src/query/datalog/parser/verifier.rs, #[cfg(all(test, not(target_arch = "wasm32")))], pest-derived from tests/grammar/grammar.pest. For every generated input: pest rejects ⇒ parser rejects; pest accepts ⇒ parser accepts or rejects with a SEMANTIC code.

• Lexical: every 1–3 character string over the spec's 102 characters, edn_value_only vs parse_edn (1,071,714 inputs).
• Structural: every balanced filler of up to MINIGRAF_GRAMMAR_BOUND (default 3) of the 32 tokens in 15 templates (plan step 9), command vs parse_datalog_command, plus a no-separator variant for fillers of ≤ 3 tokens. Filler counts are tested against the spec (18,516 at k=3, 487,678 at k=4).
• SEMANTIC (built from an empty-list run at k=3 and k=4, every site reviewed): PRS-021, PRS-022, PRS-080 (2 sites), PRS-084, PRS-107, each with a witness tested to be pest-accepted and raise exactly that code.
• KNOWN_DRIFT, issues filed: Grammar: document or drop line comments and non-ASCII identifiers #650 (comments, non-ASCII letters: documentation decision), Parser accepts inputs the grammar rejects (empty sections, unknown keywords, rule/call heads, lexical edge cases) #651 (parser tightenings, v3.0.0: unterminated string, bare :, lone -, empty query/:find/:where/:with, ignored unknown keyword, section resumed after :as-of etc., call/aggregate/rule heads), Fact with more than 4 elements: elements after the valid-time map are silently dropped #649 (fact of 5+ elements loses data: bug/data-integrity/known-issue, listed in Known issues in the current release (v2.x) #421; v3 fix then main backport).

Deviations from the plan (need approval)

1. The grammar was too loose in three more places, fixed in grammar.pest instead of listed as drift (like plan deviation 1). Without these, 155k of 187k structural failures were pest's fault:
   • rule_invocation matched not, not-join, or, or-join as rule names, so (not 1) was a "rule call" to pest. Now !reserved_head; inside an or, (and …) is never a rule call.
   • Literal keywords (":find", ":order-by", …) had no end-of-word check, so (:order-by?x) was :order-by ?x to pest and the unknown option :order-by?x to the parser (PRS-039). Each is now an atomic kw_* rule with !word_cont. The plan put this with the empty-:find fix; it showed up now, so it is here.
2. Causes, not overlapping predicates. Each failure is put down to its first cause in text order (lexical_cause, structural_cause), so an input matches at most one entry and the "exactly one" rule holds by construction; unmatched failures and stale entries still fail. The structural causes read the parser's own EDN tree of the input, not the filler tokens: in the glued variant, tokens merge (:find?x is one keyword to the parser).
3. Entries differ from the plan's expected list. New: UnterminatedString, EmptyQuery, EmptyWhere, ResumedSection, AggregateHead (k=4). Merged: rule-head-args is RuleHead. Dropped: udf-var-head (needs 5 tokens, beyond the scheduled bound; listed in Parser accepts inputs the grammar rejects (empty sections, unknown keywords, rule/call heads, lexical edge cases) #651).
4. SEMANTIC is smaller than the plan expected. With the grammar fixes, PRS-051, PRS-093 and INT-016 are now syntax (pest rejects too). The others the plan expected (PRS-023, 027, 031/032/034, 057, 064, 065, 082, INT-018) are raised by no generated input at k ≤ 4, so they have no generated witness and are left out (the rule in step 10).
5. Reports: the failure message groups unknown failures by reason (clause and code), with the first of each, plus the first 20.

Timing (step 12)

Debug cargo test, 16-thread machine, load ~7 from another session (not idle, so approximate). Levels run on scoped threads (lexical by length × first-character range, structural one thread per template), merged in input order.

Default stays k=3 (≈35 s on a 4-core runner); the scheduled job (PR 5) runs k=4. k=5 does not fit 30 minutes.

Review

Codex: one finding (unknown valid-time keys counted as FactArity), declined — the parser rejects an unknown key with PRS-088 both with and without trailing elements, so pest and parser agree; only a valid map followed by extra elements is FactArity.

Tests

1650 tests (1640 passing, 10 ignored): 4 new. grammar_conformance passes with the two arity files moved to invalid/syntax/. MINIGRAF_GRAMMAR_BOUND=0..4 all pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_019kY61Y11gcRD91Y6b7jbGT