perf(engine): recursive rounds spend their time on duplicate candidate derivations #672

Found while measuring #548 PR 3 (#671).

Shape. tests/recursive_acceptance_test.rs: 63 layers of 50 nodes, complete bipartite :edges between neighbouring layers (155,000 facts), reach = transitive closure ((reach ?x ?z) (reach ?x ?y) [?y :edge ?z]). It derives 4,882,500 facts in 62 rounds in 226 s (laptop, --profile bench), at 169 bytes of RSS per derived fact, so memory is fine; time is not. The envelope recursive shape shows the same rate: 10.6M derived facts in 53.8 s at 10M base facts (about 3–5 µs per derived fact).

Where the time goes. Each new reach(x, z) is derived through 50 middle nodes y, so the rounds produce about 250M candidate rows for 4.9M distinct facts. A flat profile (perf, no single hot spot):

• the delta join probes [?y :edge ?z] once per delta row (IndexNestedLoopJoin), decoding the committed leaf each time (keys::Reader, node::read_leaf_entry, decode_leaf, live_keys): ~20%;
• the relation's identity set hashes every candidate with SipHash (DefaultHasher): ~8%;
• per-batch allocation and copying (RawVec, Vec::append_elements, malloc/free): ~15%.

Possible directions (to be measured, none decided):

• probe each distinct bound entity once per batch (or cache a probe's result within a round) instead of once per row: the delta of a round has few distinct ?y values here;
• a cheaper identity check on id cells before building the encode_value key (keeping a keyed hasher: maps over fact data keep RandomState, not FxHash, for HashDoS, perf: point query on a bound entity scans the entity's whole version history — O(retract/reassert count), not O(live facts) #323);
• deduplicate a round's candidates before the identity set.

The answer must not change (oracle). Acceptance: reach_5m and the envelope recursive shape faster with peak RSS no worse; #379 point queries unchanged.

Not a regression: the executor took > 60 s and > 8.8 GiB at 10M on the same shape (docs/BENCHMARKS.md, #548 section).

🤖 Generated with Claude Code