Repository navigation
feat(query): joins on the streaming engine (#545 PR 2) - #593
Merged
Merged
Conversation
#545) Under the wasm feature the planner keeps source order, so [?b :db/valid-at ?t] runs first and binds ?b as a ref; the test's premise (the planner runs it last) holds only natively. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR
Planner v2 runs queries of several patterns with stored attributes and expression clauses: the first pattern is a Scan, each later one an IndexNestedLoopJoin when its entity is a bound variable (one EAVT range per outer row, committed and pending) or a HashJoin otherwise, built on the side with the smaller estimate. Cells join by JoinKey: floats by IEEE ==, a keyword with a ref to the entity it names (IdSpace::keyword_eid), long strings as one form (pending strings the value pages hold become Long). A join keeps the outer cell unless the executor's Binders say the pattern gives the value (first binding in :where text order). Pseudo-attribute patterns stay with single-pattern queries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #545 (#473 part 3), PR 2 of 3 in
docs/superpowers/plans/2026-10-14-545-joins.md(steps 4–9). #545 stays open until PR 3.What
Queries of several patterns with stored attributes, with or without expression clauses, now run on the streaming engine.
plan.rs): clauses inoptimizer::planorder. The first pattern is aScan. Each later pattern is anIndexNestedLoopJoinwhen its entity is a variable already bound, otherwise aHashJoin(value and attribute positions, cross products), built on the side with the smallerbranch_cost(ties build on the inner side). Pseudo-attribute patterns are still accepted only in single-pattern queries; across joins they wait for PR 3.join.rs): the inner pattern is aScanover its own slots, so a variable repeated inside it compares exactly, as it does in the executor's hash path. INLJ probes one entity range per outer row, committed and pending (Scan::probe; the attribute and value constants are resolved once). Both joins resume across output batches.cell.rs):JoinKey. Floats compare by IEEE==(-0.0=0.0, NaN equals nothing). A keyword equals a ref to the entity it names (IdSpace::keyword_eid), except between two attribute positions. Long strings have one form:Cell::of_valueturns a string the value pages hold intoLong, both for pending facts and for expression results.Binders(nowpub(crate)), so a variable keeps the value from its first pattern in the:wheretext.Tests
query::engine::tests(every query also runs on the executor) gains 33 join shapes at all 6 temporals, in memory, across checkpoints and after reopen. The shapes cover INLJ via keyword, ref and UUID-named keyword, value joins of keyword against ref in both text orders, floats0.0/-0.0, long strings, attribute joins from attributes and from entities, cross products, and expressions between joins. New test data: attributes that name entities.engine_joins_read_many_batchescovers one probe with more than a batch of matches, cross products, and tables built on either side, over committed and pending facts.planner_chooses_the_joinandkeyword_eid_is_the_named_entitys_idare new unit tests.cargo testandcargo test --features test-oraclepass: 1493 passed, 10 ignored. Most multi-pattern queries in the integration suites now run through the oracle. Clippy is clean for native,--all-features, wasm32/browser and wasip1. The engine tests also pass with--features wasm.join_rules_test::pseudo_attribute_pattern_never_replaces_a_valuefails under--features wasmonv3already. With source order, the pseudo-attribute pattern runs first and binds?b. The test is now native-only. CI does not run--features wasmtests.Measurements
#394 envelope on the idle bench server, A/B against
eab15bb, two rounds each, p50 and peak RSS at 1M facts:v3join_2join_3_selectivejoin_3_scanPoint,
:as-of,:valid-at, hot-entity and attribute-scan queries are unchanged. 100K rows are indocs/BENCHMARKS.md. Release binary: 1,165,920 → 1,182,944 bytes (+16.6 KiB, limit 1,200 KiB).Known deviation
Two different keywords that name the same UUID (for example
:<uuid>in upper and lower case), joined between two attribute positions, compare by ident here. The executor's hash path normalises them in that case, and its nested-loop path does not.No API change and no new error code, so no binding cascade. The docs site and the
CLAUDE.mdmodule list beyond this PR follow with PR 3.🤖 Generated with Claude Code
https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR