Repository navigation
feat: fact-log iterator — Minigraf::fact_log (#430) - #463
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn
Streams every fact version (assertions and retractions, with tx_count, tx_id and valid-time bounds) as FactRecords, without Datalog. FactFilter selects exact attributes, attribute prefixes, entities and a tx range, checked on index keys before decoding. FactOrder::Tx (default) returns ascending tx_count. v8 has no tx-ordered index, so each pass keeps the next `window` smallest (tx, key) entries in a bounded heap: memory O(window), ceil(N/window) key scans. FactOrder::Storage reads in one pass. An open log pins the committed generation it opened on (pages freed by a checkpoint are reused by the next one): auto-checkpoints and the checkpoint on drop are deferred, and checkpoint()/rebuild_indexes() with work to do fail with the new API-013. Opening takes the write lock briefly, so the snapshot never holds half a transaction. Design: docs/superpowers/specs/2026-10-07-fact-log-iterator-design.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #430 (Rust API; the binding objects follow with #462, read-only open is #429, the raw writer is #431).
What
Minigraf::fact_log(&FactFilter) -> FactLogstreams every fact version (assertions and retractions) asFactRecord { entity, attribute, value, tx_count, tx_id, valid_from, valid_to, asserted }, without Datalog. LikeCursor,FactLogis owned andSendand providesnext_batch(max),close()and an iterator.Filters (ANDed): exact attributes, attribute prefixes (the two are ORed), entities, and a
tx_countrange. They are checked on index key bytes, so excluded records are never decoded. An entity set walks EAVT entity prefixes, an exact-attribute set walks AEVT attribute prefixes, and anything else walks all of EAVT with a memoised attribute check.Order, chosen by the caller:
FactOrder::Tx(default) returns records in ascendingtx_count. v8 has no tx-ordered index and the format is frozen, so each pass keeps thewindowsmallest(tx, key)entries after the last one emitted, in a bounded heap. Memory is O(window) (default 262,144) and the cost isceil(N/window)key scans. Order within one transaction is unspecified but deterministic.FactOrder::Storagereads in one pass.Snapshot and pin. A reader must not span two checkpoints, because freed pages are reused by the next one. An open log therefore pins the generation it opened on:
checkpoint()andrebuild_indexes()fail with the new API-013 when they have work to do;Opening takes the write lock briefly, so a log never sees half a transaction. Closing, dropping or finishing the log releases the pin.
The issue also asked for two filters this PR does not provide:
entities.The issue's "untouched page ranges" doesn't apply either: v8's copy-on-write trees share a dictionary, so a range of pages can't be copied on its own. All three are written up in the spec.
Design:
docs/superpowers/specs/2026-10-07-fact-log-iterator-design.mdTests
tests/fact_log_test.rshas 22 tests and there are 2 new doctests, for 1309 total (1300 passing, 9 ignored). The tests cover:executeandWriteTransaction::commitwaits for the log; close, drop or finish releases it.INT-001on a write-transaction thread; a log outlives its handle and the deferred writes replay on reopen.A local release run on 200k facts reads about 200k records/s, with decoding dominating. One Tx-order key pass over 200k takes about 0.15 s.
Docs updated: CHANGELOG, ROADMAP, README, CLAUDE.md, TEST_COVERAGE, ERROR_REFERENCE (API-013). The wiki is left for the v3.0.0 doc sync, as with the earlier v3 PRs.
🤖 Generated with Claude Code
https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn