Skip to content

feat(engine): the engine runs every query (#551 PR 1) - #715

Merged
adityamukho merged 13 commits into
v3from
feat/551-engine-everywhere
Oct 23, 2026
Merged

adityamukho merged 13 commits into
v3from
feat/551-engine-everywhere

Conversation

@adityamukho

Copy link
Copy Markdown
Collaborator

#551 PR 1 of 3 (plan: docs/superpowers/plans/2026-10-23-551-executor-removal.md, spec #714): the streaming engine runs every query. There is no fallback in any build. The executor's query path compiles only under cfg(any(test, feature = "test-oracle")), where it is the engine's oracle. PR 2 deletes it, together with the oracle and the test-oracle → test-hooks rename, and turns the 1,200 KiB size gate back on.

What changed, by the survey's causes (spec §2)

Cause Now
A :as-of timestamp TxBound in storage. The plan keeps :as-of unresolved, and engine::open resolves it once per query (Snapshot::tx_bound). When tx_id rises with tx_count over the committed, pending and staged transactions, it becomes a counter bound (Upto). Otherwise it is Timestamp: committed keys are kept by tx_count ranges read from the DICT (one tx_id per tx_count, STG-037), and pending facts by their own tx_id. The ranges are charged to the query's meter.
B or branches A branch is planned the way a :where is (clauses). Nested or/or-joins are allowed. Imports count what the nested ors read.
C, J no pattern first Start from Source::Seed
D head constant HeadArg::Const, read the way the executor reads a head
E entity constant Term::Never: the scan reads nothing
F, K, L, N, H, G Coded errors, each the code the executor reports: INT-028, INT-022 (extra head arguments are ignored), INT-054, INT-025 (:valid-at slot) and INT-032 (:as-of slot), INT-031, INT-032. The parser rejects G and H first.
I not bodies Planned as clause lists. The parser rejects every nested shape first (PRS-051, PRS-093, INT-016).
M reader without keys INT-049 at open

Error order: pseudo-attributes, then planner/rule errors, then unknown predicates (QRY-007). One test covers all three.

Answer changes

Each one has a test that checks both answers, with the oracle exempted (oracle::exempt). They go into CHANGELOG.md and onto the docs site in PR 3.

  1. A not or nested or inside an or branch sees a variable bound by one of the branch's expressions (spec §4.1).
  2. An unknown predicate in an or branch is QRY-007 even when no row reaches it (spec §4.2).
  3. Not in the spec (plan deviation 4): an unknown predicate in a not/not-join body is also QRY-007 at open. The executor evaluated it per body row, where it failed, so the not kept every outer row.

disjunction_test 20b asserted the executor's branch short-circuit, where a predicate in a covered branch is never called. The engine runs every branch, so the test now checks only the answer.

Deviations from the spec

  1. For committed keys, §3.1's per-version tx_id filter becomes a tx_count range filter. It keeps the same versions.
  2. §3.5's required key_source becomes INT-049 at open.
  3. §3.6's test-time panic is structural: no fallback path exists.
  4. The third answer change above.
  5. PendingStore::first_irregular also marks a falling tx_count, so a monotonic pending prefix rises in both tx_count and tx_id. TxOrder carries the newest (tx_count, tx_id), so last_tx_at does not depend on the meta's counter.

Verification

  • Full suite, --features test-oracle: 1,782 passed, 16 ignored. Without it: 1,767 passed, 12 ignored. The executor ran no query outside the oracle.
  • clippy -D warnings: --all-features, no features, --features test-oracle, wasm32-unknown-unknown --features browser, wasm32-wasip1
  • cargo doc (-D warnings), cargo fmt --check
  • Release binary: 1,140 KiB (1,168,000 bytes), down from 1,287 KiB (1,317,592) on 1096e6a2, so it already fits the 1,200 KiB gate PR 2 turns back on.
  • Codex review: no correctness findings. One coverage gap (timestamps after a checkpoint) is fixed in the last commit.

None of these commits closes #551 or #473.

🤖 Generated with Claude Code

https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW

adityamukho and others added 12 commits October 23, 2026 13:52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
plan, plan_rules, engine::open and open_rules return Result<_, Refusal>:
Unsupported(Cause) when the executor still runs the query, by the letter
of the spec's survey, or Fail when it fails. Lookups that cannot fail are
INT-032. open_query falls back on Unsupported only; no answer changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
TxBound replaces as_of in the key walk and the pending reads: every
version, tx_count <= n, or for :as-of timestamps where tx_id does not
rise with tx_count, committed keys whose tx_count is in the ranges of
transactions with tx_id <= t and pending facts with tx_id <= t.
KeySource gains tx_order (cached per reader), last_tx_at and tx_runs
over the DICT; PendingStore tracks first_irregular; Snapshot::tx_bound
picks a counter bound when the snapshot is monotonic. Wired into
queries in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
The plan keeps :as-of unresolved (AsOfSpec); engine::open and open_rules
resolve it once against the snapshot (Snapshot::tx_bound), charge a
timestamp's committed ranges to the query's meter, and every scan of
the query, its bodies and its rules reads the bound from Ctx. Cause A
is gone. Tested on LogWriter files whose tx_ids rise and whose tx_ids
fall and rise, with committed, pending and staged facts, under the
oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
A branch or a not body is planned as a :where is (clauses): its
patterns and expressions in plan_seeded order, its ors in text order,
then its deferred clauses. What a branch or body imports counts the
variables nested ors read (reads), and a branch whose nested or reads
an import its patterns do not bind is not hashed (closed). runs_branch
keeps only the two executor behaviours step 6 changes: unknown
predicates and expression-bound variables read by a not or nested or.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
… at open (#551)

runs_branch goes: a not or nested or in a branch sees what the branch's
expressions bind, and a branch's predicates run on the engine. Every
predicate a query names, at any depth, and those its rules' bodies name,
is resolved when it opens (QRY-007), as :find functions are. These are
the spec's two answer changes, plus an unknown predicate in a not body
(plan deviation 4); their tests check both answers with the oracle
exempted (oracle::exempt).

disjunction_test's 20b asserted the executor's branch short-circuit
(predicate never called); it now checks the answer only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
… row (#551)

An expression before any pattern, or not clauses with no pattern after
them, run on Source::Seed, as an or with nothing before it already did.
Causes C and J are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
Builder::anti no longer filters a body's clause kinds; a body is planned
by clauses as a branch is, and closed counts the variables a nested not
reads at any depth. The parser rejects nested not, not-join and or in a
not body first (PRS-051, PRS-093, INT-016), so cause I goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
HeadArg::Const holds a magic-set head constant (#608), read as the
executor reads a head (a keyword's or UUID's entity, else QRY-013; the
value through edn_to_value), once per rule run. Term::Never is a
constant of a kind its position does not take; a scan with one reads
nothing, as the executor returns no row. Causes D and E are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
An invocation of three or more arguments is INT-028 (from
rule_invocation_to_pattern), a head without arguments INT-022 and extra
head arguments are ignored, a stratification error passes through
(INT-054), a :valid-at slot is INT-025 and an :as-of slot INT-032; or
branches whose variables differ (INT-032) and an unbound or-join
variable (INT-031) are rejected by the parser first. A planner error
comes after the pseudo-attribute check and before unknown predicates.
Causes F, G, H, K, L and N are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
)

Refusal and Cause go: plan, plan_rules, engine::open and open_rules
return anyhow::Result, a committed reader without keys is INT-049 at
open, and open_query/open_rule_query have no executor branch. The
executor's query path (execute_query_on_executor,
execute_query_with_rules, PatternMatcher, RecursiveEvaluator,
StratifiedEvaluator, evaluate_not_join and the storage reads only they
use) compiles under cfg(any(test, feature = "test-oracle")) only, as
the oracle. Module notes and test counts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
Codex review: the pending facts of each file, checkpointed into a new
committed generation, answer the same timestamps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

The coverage gate counts test code: the empty case always has a
generation, and the two && asserts never short-circuited, which took
persistent_facts.rs branches below its floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
@adityamukho
adityamukho merged commit b682876 into v3 Oct 23, 2026
20 checks passed
@adityamukho
adityamukho deleted the feat/551-engine-everywhere branch October 23, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant