All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
No changes to the core crate. Released so the language bindings have a version to republish on, carrying a fix that lives in a binding rather than here.
v1.2.2 made a second handle on an already-open file an error (#304). The
UniFFI bindings and the C API already had a way to release a handle on demand
(destroy() and minigraf_close respectively), but the Node binding did not,
and JavaScript has no deterministic destructor — so reopening a .graph file
in one process became impossible there. minigraf-node gained a close()
method (project-minigraf/minigraf-node#1); this release is what lets it ship.
Binding versions track the core version, so all seven bindings republish at 1.2.3.
Drop-in replacement for v1.2.1. No file-format changes, no public API changes.
-
Two handles on one file in the same process (#304):
FileLock::acquiretreated a lock file whose holder PID equalled our own as stale, deleted it, and opened anyway — on the theory that it could only be a leaked handle. The far more likely cause is a handle that is open and in use right now elsewhere in the process, so the "self-heal" silently produced two liveFileBackends on one file. Each caches its ownheader.page_count, allocates new pages from that count, and bounds-checksread_pageagainst it, so the two page tables diverge. That — not an allocation off-by-one — is the source of the intermittentPage N out of bounds (total pages: M). It also corrupted the lock itself: whichever handle dropped first ranFileLock::dropand removed the lock file that by then belonged to the survivor, leaving a live handle unlocked and admitting other processes too. A lock held by our own PID is now refused with an error naming the same-process case; a lock held by a different, dead process is still reclaimed as before. Behaviour change: callers that relied on reopening a path they already hold open now get an error instead of silent corruption, and should reuse the existing handle (Minigrafis cheap to clone and all clones share one database). -
String comparison in predicates:
[(< ?a ?b)],>,<=and>=now order two strings lexicographically instead of failing. Previously every ordering predicate routed throughto_float_pair, which errors on non-numeric operands, so a string comparison silently matched no rows — and because both directions returned nothing, a range filter dropped the entire relation rather than partitioning it. This most often bit ISO-8601 timestamps stored as strings. The predicate path now agrees withvalue_lt/value_cmp, whichmin,maxand:order-byalready use. Mixed-type comparison (string vs number) remains an error, and NaN comparisons remain false.
Drop-in replacement for v1.2.0. No file-format changes, no public API changes.
- Magic sets
fbadornment: queries with a literal keyword in value position (e.g.(reports-to ?emp :alice)) now return correct results instead of empty results (#298). Root cause: seed fact used an ephemeral random UUID as entity; the 1-arg guard bound the variable to that UUID rather than the keyword value. Fixed with a deterministic sentinel entity (Uuid::new_v5) and a 2-arg guard that binds from value position. - Magic sets recursive rules:
(reach :a ?y)with intermediate variable?midand keyword entity binding now evaluates correctly (#297).
Drop-in replacement for v1.1.x. No file-format changes, no public API breaking changes. Upgrading requires no code changes.
- Magic Sets rewriting — recursive Datalog queries with bound arguments are now automatically rewritten top-down via magic sets, propagating bound values into recursive rules to avoid full-relation scans (#289)
- Adornment classification, seed fact generation, magic guard injection, SCC-aware propagation rule generation, full
rewrite()wired intoexecute_query_with_rules - Limitation: mutual recursion through negation is not rewritten (documented in ROADMAP §9.6)
- Adornment classification, seed fact generation, magic guard injection, SCC-aware propagation rule generation, full
- Per-query complexity limits —
:max-derived-factsand:max-resultsclauses added toquery; global default raised to 1,000,000 derived facts (#288, #290)
selective_fact_fetch: includeassertedflag in dedup key — previously retracted facts could shadow live facts under certain access patterns (#285, #286)selective_fact_fetch: restore per-pattern entity priority lost in a prior refactor (#283)- v5→v6 migration: fix hang caused by using
header.page_countas the B-tree start page instead of the correct offset (#272)
- Eliminate backend mutex hold on cache hits in
CommittedFactLoaderImpl::resolve— read path no longer acquires the write lock when the page is cached (#279) - Pre-build
MutexStorageBackendinCommittedFactLoaderImplto eliminate oneArc::cloneperresolve()call (#280, #281)
- Split Java, Android, Swift, and C bindings into independent repos under
project-minigraforg — completes #231 repo split (#231 phase 2)minigraf-java: https://github.com/project-minigraf/minigraf-javaminigraf-android: https://github.com/project-minigraf/minigraf-androidminigraf-swift: https://github.com/project-minigraf/minigraf-swiftminigraf-c: https://github.com/project-minigraf/minigraf-c
minigraf-ffiyanked from crates.io — UniFFI layer inlined into each binding repo's private shim; use the per-language packages insteadcascade.ymlupdated to dispatchcore-releaseto all 7 binding repos on every version tag- All binding repos migrated to OIDC trusted publishing —
NPM_TOKENandCARGO_REGISTRY_TOKENsecrets removed; crates.io usesrust-lang/crates-io-auth-action@v1, npm usessetup-node@v6OIDC - Python, Node, and WASM binding repos now include a
preparejob that pins theminigrafdependency version inCargo.tomland commits before building — consistent with Java, Android, Swift, and C
- Add
docs/ERROR_REFERENCE.md: full inventory of user-facing errors (PRS/QRY/STG/WAL/API categories, 113 entries) with cause, resolution steps, and bad-input examples; docs-only reference codes PRS-001…API-009 (#192)
Patch release. Fixes cargo-dist Windows build failure that prevented REPL binaries and crates.io publish from completing for v1.1.0. No code changes to the library itself.
- Exclude
fuzzcrate from workspace so cargo-dist can build on Windows (MSVC linker incompatible with#![no_main]libFuzzer targets) — fixes #263
Drop-in replacement for v1.0.0. No file-format changes, no public API changes, no query surface changes. Upgrading requires no code changes.
- Hash-join replaces nested-loop join for multi-clause queries — O(N) instead of O(N²) for large fact sets (#202, #203, #204)
- Selective B+Tree fact fetch: queries with bound entity/attribute skip full-scan and read only relevant index pages (#208)
- Predicate push-down into join ordering (#207)
- Cost-based clause ordering for
not/orrules (#206, #205) - SIMD crossover analysis and benchmarking infrastructure (#229)
- Fixed critical correctness and durability bugs found during deep audit (#225): fact visibility edge cases, WAL entry ordering, checkpoint atomicity
- Fixed read-only handle
Droptriggering unnecessary checkpoint, modifying the file on close (#226) QueryResult::TransactedandRetractedreverted to tuple variants — struct-variant form introduced post-1.0 was a breaking change (#261)- Added
Minigraf::current_tx_count() -> u64as additive API to expose the:as-ofmonotonic counter without breaking existing pattern matches
- WAL fault injection harness (
FaultInjectingBackend) with 9 crash-recovery tests (#209, #210, #214) - Storage/migration resilience: migration matrix, index corruption recovery, concurrency stress tests (#215, #216, #217)
- Property-based query correctness tests against a reference evaluator (proptest, #212)
- Datalog parser/evaluator fuzz targets with seed corpus (#213)
- Per-module branch coverage gates in CI (#219)
- Long-haul smoke suite: 500 entities × 10 write/read/checkpoint cycles (#220)
- XTDB and Datomic semantic compatibility tests (#221)
- Workspace-wide clippy lint enforcement — 336 violations fixed (#232)
- Grammar conformance test harness: pest shadow grammar + EDN corpus (#233)
- CI: codecov-action v3→v5, actions-rs→dtolnay migration
Six PRs hardening the v1.0.0 codebase: WAL fault injection, storage/migration resilience, query correctness (property-based + coverage gates), long-haul smoke testing, and XTDB/Datomic semantic compatibility. No API changes, no file-format changes, no new runtime dependencies.
PR #254 — #209 + #210 + #214: WAL Fault Injection
FaultInjectingBackend(#[cfg(test)]-only wrapper aroundStorageBackend) — configurable write-fail, flush-fail, read-fault injection- 9 new WAL fault tests: write-fail propagation, flush-fail without data corruption, read-fault on WAL replay, CRC corruption discard, checkpoint atomicity under backend failure, partial checkpoint recovery via WAL replay, multi-writer serialisation, concurrent write+checkpoint, backend error propagation as
Errnot panic
PR #257 — #215 + #216 + #217: Storage & Migration Resilience
tests/migration_matrix_test.rs— 5 migration tests: v7 round-trip, v3 empty migrate, corrupt magic returnsErr, unsupported version returnsErr, WAL replay idempotenttests/index_corruption_test.rs— 5 corruption-resilience tests: checksum mismatch triggers index rebuild, btree leaf/internal corrupt pages returnErrwithout panic, root pointer mismatch handled, non-critical corruption still serves queries on good data- 5 new concurrency stress tests in
tests/concurrency_test.rs: stress readers during writer, failed write then success, rollback after partial work, open/write/checkpoint/query loop per thread, nightly stress loop (#[ignore])
PR #256 — #212 + #213 + #219: Property-Based Testing & Coverage Gates
tests/property_test.rs(proptest,cfg(not(wasm32))): 3 property tests — EAV model correctness vs naive reference evaluator, bi-temporal monotonicity, retract visibility invariant.github/workflows/coverage-gates.yml— per-module branch coverage thresholds; CI fails if coverage drops below gate
PR #258 — #220: Long-Haul Smoke Suite
tests/smoke_test.rs(#[ignore]nightly):smoke_large_graph_10_cycles— 500 entities × 10 attributes × 10 update cycles; 7 invariants: active count (333), retracted count, fact count bounds, temporal snapshot integrity, prepared query consistency, recursive rule transitive closure, WAL checkpoint round-trip.github/workflows/smoke.yml— nightly 5am UTC, 15-min timeout, runs--include-ignored
PR #259 — #221: XTDB & Datomic Compatibility Corpus
tests/xtdb_compat_test.rs— 10 semantic ports (Apache 2.0): EAV, tx-time:as-of, valid-time:valid-at, retraction (current + historical), Datalog join, negation, recursive rules, parameterised queries, combined bi-temporaltests/datomic_compat_test.rs— 9 independently written semantic ports: datom model, multi-entity attribute, tx-time:as-of, retract-entity, multi-variable:find, ground-value binding, parameterised query (prepared), named reusable rules, predicate expression filter
935 tests passing (943 total, 8 ignored: 6 or+neg-cycle stratification doc tests, 1 nightly concurrency stress, 1 nightly smoke).
Three optimisation PRs extending the v1.0.0 performance work. No API changes, no file-format changes, no new runtime dependencies. Test count unchanged at 850.
PR #249 — #207 + #206: Predicate Push-Down & Mixed Rule Optimization
optimizer::plan()extended to acceptExpr(predicate/arithmetic) clauses; they are interleaved at the earliest position where all their variables are bound, minimising intermediate binding setsStratifiedEvaluatorgains mixed-rule path: when a rule stratum contains both positive-only rules and rules withnot/not-join, the evaluator now evaluates all positive rules first and applies negation filters in a second pass within the same stratum — eliminates a class of ordering-dependent bugs in cross-stratum negation
PR #251 — #205: Cost-Based not/or Ordering
optimizer::plan()assigns selectivity estimates tonot/not-joinandor/or-joinclauses; they are sorted after their ground-variable producers but before unconstrained pattern scansnot/not-joinblocks placed after the patterns that produce their join variables (was: end of plan regardless);or/or-joinblocks placed by estimated output cardinality
PR #253 — #229: SIMD Benchmarking & Crossover Analysis
benches/simd_helpers.rs:valid_time_filter_simd,as_of_filter_simd,sum_simd_i64— portable SIMD kernels usingwide::i64x4/u64x4- Criterion benchmark groups:
simd_temporal,simd_as_of,simd_aggregate— scalar vs. SIMD crossover analysis at 1K–1M facts - Analysis result: SIMD crossover occurs at ~8K–16K facts per query; scalar path preferred below that threshold; SIMD integration deferred to post-1.0 backlog pending real-workload profiling
850 tests passing (844 passing + 6 ignored: confirmed or+neg-cycle stratification bug, deferred to post-1.0 backlog). All optimizer PRs are optimisations; no new integration tests added.
Four O(N²) query-engine bottlenecks eliminated. No API changes, no file-format changes, no new dependencies.
PR #246 — #208: B+Tree Selective Lookup
get_facts_by_entity,get_facts_by_attribute,get_facts_by_entity_attributepromoted from#[cfg(test)]to production insrc/graph/storage.rs- New
selective_fact_fetchhelper inexecutor.rs: inspects query patterns for bound entity literals and bound attribute strings; calls index-driven fetches instead ofget_all_facts()when ≤4 distinct lookups detected as_ofqueries continue to use full scan (required for correctness)- New benchmark groups:
btree_lookup/entity_point,btree_lookup/attribute_scan
PR #247 — #202 + #203 + #204: Hash-Join Cluster
- #202 (
executor.rs):not/not-joinbodies pre-computed once intoHashSet<Vec<(String, Value)>>keyed on join variables; O(1) probe per outer binding replaces O(N) re-scan.normalize_valuehandleskeyword→Refrepresentation asymmetry in value position. - #203 (
executor.rs):or/or-joinbranches now evaluated from a single empty seed; branch results hash-joined back onto incoming bindings on shared user-visible variables (__-prefixed metadata keys excluded).or-joinprojects tojoin_varsbefore joining. - #204 (
matcher.rs):join_with_patterndetects join variable (entity position first, value position second), buildsHashMap<Value, Vec<Bindings>>once, probes per existing binding O(1).normalize_join_valuehandleskeyword→Ref. Falls back to nested-loop when no join variable found.
850 tests passing (844 passing + 6 ignored: confirmed or+neg-cycle stratification bug, deferred to post-1.0 backlog).
This is the v1.0.0 release. The public Rust API and the .graph file format are now stable
and committed to semantic versioning. File format stability is guaranteed from this release.
All Phase 8 cross-platform targets have shipped:
- 8.1a — Browser WASM (
BrowserDb,IndexedDbBackend,@minigraf/browseron npm) — v0.20.0 - 8.1b — Server-side WASM (
wasm32-wasip1/ WASI, Wasmtime/Wasmer CI) — v0.20.0 - 8.2 — Mobile bindings (Android
.aaron GitHub Packages, iOS.xcframeworkvia SPM, UniFFI) — v0.21.0 - 8.3a — Python (
minigrafon PyPI, pre-built wheels) — v0.22.0 - 8.3b — Java/JVM (
io.github.adityamukho:minigraf-jvmon Maven Central, fat JAR) — v0.23.0 - 8.3c — C FFI (
minigraf.h+ platform tarballs on GitHub Releases) — v0.24.0 - 8.3d — Node.js (
minigrafon npm, pre-built.nodebinaries) — v0.25.0
pkg/renamed tominigraf-wasm/,swift/renamed tominigraf-swift/— consistent top-level naming across all workspace packages (issue #179)@minigraf/browsernow published to npm on every tagged release (issue #179)@minigraf/wasipublished to npm on every tagged release (issue #178) — WASI binary packaged for Node.js WASI consumers- Per-platform READMEs added:
minigraf-wasm/,minigraf-node/,minigraf-ffi/python/,minigraf-c/,minigraf-ffi/java/
795 tests passing (788 passing + 7 ignored: confirmed or+neg-cycle stratification bug,
deferred to post-1.0 backlog).
- Phase 8.3d: Node.js bindings published to npm as
minigraf. Install withnpm install minigraf. No build step required — prebuilt.nodebinaries for Linux x86_64/aarch64, macOS universal2, Windows x86_64. API:new MiniGrafDb(path),MiniGrafDb.inMemory(),.execute(datalog),.checkpoint(). Full TypeScript definitions included.
- Phase 8.3c: C bindings distributed as GitHub Releases tarballs.
Download
minigraf-c-v0.24.0-<platform>.tar.gz(Linux/macOS) or.zip(Windows) from the release page. Each archive contains the prebuilt shared library plusminigraf.h. API:minigraf_open,minigraf_open_in_memory,minigraf_execute,minigraf_string_free,minigraf_checkpoint,minigraf_close,minigraf_last_error. Memory contract mirrors SQLite:minigraf_executereturns a heap-allocated JSON string; callminigraf_string_freeto release it. minigraf-c/: new workspace crate (cdylib+staticlib) —Cargo.toml,src/lib.rsminigraf-c/cbindgen.toml: cbindgen 0.29.2 configurationminigraf-c/include/minigraf.h: committed stable header (cbindgen-generated).github/workflows/c-ci.yml: PR test matrix on 4 platforms + header drift check.github/workflows/c-release.yml: release workflow — builds + packages platform tarballs, uploads to GitHub Releases
795 tests.
- Phase 8.3b: Java desktop JVM bindings published to Maven Central as
io.github.adityamukho:minigraf-jvm:0.23.0. Add to Gradle:implementation("io.github.adityamukho:minigraf-jvm:0.23.0"). Fat JAR with embedded natives for Linux x86_64/aarch64, macOS universal2, Windows x86_64. API:MiniGrafDb.open(path),MiniGrafDb.openInMemory(),.execute(datalog),.checkpoint(). minigraf-ffi/java/: Gradle 8.11 project —build.gradle.kts,settings.gradle.kts,NativeLoader.kt(runtime native extraction from JAR resources), and Gradle wrapperminigraf-ffi/java/src/test/kotlin/.../BasicTest.kt: JUnit 5 suite (in-memory, transact/query, error handling, file-backed persistence).github/workflows/java-ci.yml: PR test matrix on 4 platforms (Linux x86_64, Linux aarch64, macOS universal2, Windows x86_64).github/workflows/java-release.yml: release workflow — cross-compiles natives on 4 platforms, assembles fat JAR, publishes to Maven Central via Sonatype OSSRH
795 tests.
- Phase 8.3a: Python bindings published to PyPI as
minigraf. Install withpip install minigraf. API:MiniGrafDb.open(path),MiniGrafDb.open_in_memory(),.execute(datalog),.checkpoint(). Pre-built wheels for Linux x86_64/aarch64, macOS universal2, Windows x86_64.
src/lib.rs: added Feature Flags section and WebAssembly targets subsection to crate-level docs — browser feature,wasm32-unknown-unknowntarget switcher note, and WASI build commandREADME.md: updated "For Mobile Apps" section — replaced Phase 8 placeholder with current state, added Kotlin/Swift quick-start snippets and link to wiki integration guide- Wiki
Use-Cases.md: replaced Integration placeholder with full Android (Gradle setup, Kotlin API, error handling, threading) and iOS (SPM setup, Swift API, error handling, async) integration guides
795 tests.
minigraf-fficrate: UniFFI 0.31 bindings exposingMiniGrafDb(open, openInMemory, execute, checkpoint) andMiniGrafError(Parse, Query, Storage, Other) to Kotlin and Swift- Android
.aarrelease artifact, published to GitHub Packages (io.github.adityamukho:minigraf-android) - iOS
.xcframeworkrelease artifact, distributed via Swift Package Manager (Package.swiftat repo root) mobile.ymlCI workflow: cross-compiles Android targets withcargo-ndk, generates Kotlin/Swift UniFFI bindings, assembles AAR with Gradle, assembles xcframework withxcodebuild, and publishes both on every tagdocs-checkCI job inrust.ymlandrelease.yml— gates releases oncargo doc --all-featurespassing cleanly
release.yml: addeddocs-checktohostjob'sneedsandifconditionwasm-release.yml/mobile.yml: retry loops extended from 20 to 40 attempts;inputs.tag || github.ref_nameordering correctedminigraf-ffi/android/gradlew: removed inner double-quotes fromDEFAULT_JVM_OPTSand replaced xargs/sed eval block with directexec— fixes "Could not find main class" and garbled usage outputminigraf-ffi/android/build.gradle.kts: addedandroid { publishing { singleVariant("release") } }— fixes AGP 8.x "SoftwareComponent 'release' not found"mobile.ymlPackage.swift commit: pushes to unprotectedswift-releasesbranch and moves tag viagh api -F force=true— avoids branch-protection blocks and string/boolean type mismatch
795 tests.
browsermodule now appears on docs.rs: addeddocsrsto thecfggate anddoc(cfg(...))badge annotation (src/lib.rs)
- Phase 8.1a — Browser WASM (
wasm32-unknown-unknown+wasm-bindgen):BrowserDbpublic API:open_in_memory(),execute(),checkpoint(),export_graph(),import_graph()BrowserBufferBackend— in-memoryStorageBackendover a flat page buffer, identical byte layout to the native.graphformatIndexedDbBackend— page-granular IndexedDB storage (one 4 KB entry per page); only dirty pages written on checkpointwasm-packbuild workflow (wasm32-unknown-unknown --features browser) generatingminigraf-wasm/with JS glue and TypeScript definitionswasm-bindgen-testbrowser integration tests (Chrome + Firefox viawasm-pack test)
- Phase 8.1b — Server-side WASM (
wasm32-wasip1/ WASI):FileBackendverified under WASI's capability-based filesystem (no backend changes needed)- CI workflow (
wasm-wasi.yml) builds, unit-tests, and smoke-tests under Wasmtime and Wasmer on every push/PR - Thread-dependent tests gated with
#[cfg(not(target_os = "wasi"))]
- Cross-platform compatibility tests (issue #150):
tests/cross_platform_compat_test.rs— native round-trip (raw page byte copy) and fixture-readability teststests/fixtures/compat.graph— committed v7 binary fixture containing:alice :name "Alice"and:alice :age 30examples/generate_compat_fixture.rs— reproducible fixture generator (native only; no-op on wasm32)native_fixture_readable_by_browser_dbwasm-bindgen-test — loads native fixture viaBrowserDb::import_graph, verifies both facts
- Release workflow: WASM artifacts (WASI binary + browser tarball) built and attached on every tag;
cargo publishto crates.io on release
795 tests.
Minigraf::repl()factory method replaces directRepl::new(FactStorage)constructor — users calldb.repl().run()instead- All internal types narrowed to
pub(crate):FactStorage,PersistentFactStorage,FileHeader,StorageBackend,DatalogExecutor,PatternMatcher,Fact,TxId,VALID_TIME_FOREVER,Wal, and all related internals Minigraf::inner_fact_storage()removed (was unused)
Minigraf::repl(&self) -> Repl<'_>— constructs an interactive REPL session;Replnow borrows&Minigraffor lifetime safety- Full rustdoc on all public API items with
# Examplesdoctests [package.metadata.docs.rs]inCargo.toml— docs.rs builds withall-features = true#![warn(missing_docs)]— enforces documentation coverage going forward- crates.io and docs.rs badges in
README.md - Installation section in
README.md(cargo add minigraf/[dependencies]block) - macOS and Windows added to CI test matrix (
rust.yml) - Strict
cargo clippy -- -D warningsstep inrust-clippy.yml
- Bare
.unwrap()in library code replaced with.expect("lock poisoned")(RwLock operations incache.rs,evaluator.rs) and.expect("WAL not initialized")(db.rs) FileHeader::to_bytesnow takesselfby value (clippywrong_self_convention)- Broken intra-doc link
[Repl::run]indb.rsfixed to[crate::repl::Repl::run]
788 tests.
Minigraf::prepare(query_str) -> Result<PreparedQuery>— parse and plan a query once, returning aPreparedQuerythat can be executed many times with different bind valuesPreparedQuery::execute(bindings: &[(&str, BindValue)]) -> Result<QueryResult>— substitute named$slottokens and run against the current fact store state; plan is reused on each callBindValueenum —Entity(Uuid),Val(Value),TxCount(u64),Timestamp(i64),AnyValidTime; each variant is permitted only in the appropriate bind-slot position$identifierbind slot tokens in parser — accepted in entity position, value position,:as-of, and:valid-at; attribute position is intentionally rejected at prepare timeEdnValue::BindSlot(String),AsOf::Slot(String),ValidAt::Slot(String),Expr::Slot(String)AST variants (parse-only; panic at runtime if unsubstituted)BindValueandPreparedQueryre-exported fromlib.rs(public API surface)tests/prepared_statements_test.rs— 17 integration tests covering all slot positions, combined temporal + entity parameterisation, plan reuse, and all error paths
src/query/datalog/prepared.rs— new module:prepare_query(), substitution logic, 19 unit tests; manualDebugimpl forPreparedQuery(avoidsFactStorage: Debugbound)- Panic guards (no slot-name interpolation) in
executor.rs(4 sites) andstorage.rs(1 site) for unsubstituted slot variants; CodeQL-safe (no user-controlled string in panic message)
db.execute(str)string API — no breaking change- Executor, optimizer, matcher — no changes required
Minigraf::register_aggregate(name, init, step, finalise)— register a custom aggregate function usable in both:findgrouping and:over(window) clausesMinigraf::register_predicate(name, f)— register a single-argument filter predicate usable in[(name? ?var)]:whereclausesFunctionRegistry::register_aggregate_desc/register_predicate_desc(internal API)WindowFunc::Udf(String)andUnaryOp::Udf(String)AST variants for runtime-resolved functionsUdfOps,AggImpl,PredicateDesctypes infunctions.rs
AggregateDescnow usesAggImpldiscriminator instead ofwindow_compatible+window_opsapply_expr_clausesnow returnsResult<Vec<Binding>>and accepts&FunctionRegistryeval_expracceptsOption<&FunctionRegistry>for UDF predicate resolutionWindowSpec::func_name()now returnsStringinstead of&'static str- Parser emits
Udfvariants for unknown names instead of erroring (runtime validation)
- Window functions in Datalog
:findclause:(sum ?v :over (...)),(count ?v :over (...)),(min ?v :over (...)),(max ?v :over (...)),(avg ?v :over (...)),(rank :over (...)),(row-number :over (...))with unbounded-preceding (cumulative from partition start to current row) frame :partition-by ?varoptional clause: absent means whole result set is one partition:order-by ?varrequired in every:overclause;:descoptional (default ascending)FunctionRegistry(src/query/datalog/functions.rs): string-keyed registry of aggregate descriptors; all built-in aggregates migrated into it;window_ops(init/step/finalise) on window-compatible entries;is_builtinflag separates built-ins from future UDFs- Mixed queries: regular aggregates and window functions may coexist in the same
:findclause; aggregates collapse rows first, windows annotate over collapsed rows AggregateDesc,AggState,WindowOpstypes infunctions.rsWindowFunc,Order,WindowSpec,FindSpec::Windowtypes intypes.rstests/window_functions_test.rs: 12 integration tests (cumulative sum, running count/min/avg, rank with ties, row-number, partition-by, desc ordering, mixed aggregate+window, single-row and empty-result edge cases, lag/lead parse rejection)
FindSpec::Aggregate { func }: type offuncchanged fromAggFuncenum toString; dispatch goes throughFunctionRegistry— internal change, no public API impactAggFuncenum removed fromtypes.rs; all aggregate dispatch centralised infunctions.rsapply_aggregationandapply_agg_funcremoved fromexecutor.rs; replaced byapply_post_processing+ helpers
707 tests (unit + integration + doc)
- Temporal pseudo-attributes:
:db/valid-from,:db/valid-to,:db/tx-count,:db/tx-id, and:db/valid-atare now first-class bindable values in Datalog:wherepatterns PseudoAttrenum andAttributeSpecwrapper type intypes.rs— clean type-safe representation for real vs. pseudo attributes inPatternparse_query_patterninparser.rs— detects:db/*keywords in the attribute position; rejects them in entity/value positions (parse error)PatternMatcher::from_slice_with_valid_atconstructor — passes query-levelvalid_atinto the matcher- Hard-error guard in executor: per-fact pseudo-attrs (
:db/valid-from,:db/valid-to,:db/tx-count,:db/tx-id) require:any-valid-time; error message tells user exactly what to add :db/valid-atbinds the effective query timestamp: explicit:valid-at <ts>→Value::Integer(ts), no:valid-at→Value::Integer(now),:any-valid-time→Value::Null:any-valid-timenow accepted as a standalone top-level query keyword (previously required:valid-at :any-valid-timeform)tests/temporal_metadata_test.rs: 16 new integration tests covering time-interval range queries, time-point lookups, tx-time correlation,:db/valid-atsemantics, and all parse/runtime error guards
647 tests (438 unit + 209 integration)
tests/production_patterns_test.rs: 8 cross-feature integration tests combining not+as-of, not-join+count, count+not, count+valid-at, recursion+not, or+count, or+sum, count+as-of-sequencetests/error_handling_test.rs: 8 integration-level error-path tests covering runtime type errors (sum/string, sum/mixed, max/boolean), stratification errors (negative cycles), and parse safety errors (not-join unbound join var, or mismatched vars, aggregate unbound var)- Stream 3: ~109 unit tests for parser-unreachable branches and aggregation/arithmetic edge cases in
executor.rsandevaluator.rs cargo-llvm-covbranch coverage command documented inCONTRIBUTING.md- CI coverage enforcement:
cargo-tarpaulin --fail-under 75gates every PR; Codecov 75% threshold with 2% drop tolerance;fail_ci_if_error: true - Nightly
cargo-llvm-cov --branchworkflow: uploads LCOV to Codecov (branch-coverageflag) and attaches HTML artifact (30-day retention); also triggerable viaworkflow_dispatch - Codecov badge added to
README.md
- Branch coverage:
executor.rs~85.71% (from ~75%),evaluator.rs~89.29% (from ~73%) - Remaining uncovered branches: NaN-check defensive code not reachable via public API
- Total: 617 tests (424 unit + 187 integration + 6 doc)
or-with-negative-cycle: stratification does not currently detect negative cycles insideorbranches. Tracked via#[ignore]intests/error_handling_test.rs::or_negative_cycle_rejected.
filter_facts_for_querysnapshot fix — function now returnsArc<[Fact]>instead of a throwawayFactStorage, eliminating the O(N) four-BTreeMap index rebuild that occurred on every non-rules query call.execute_querypath constructs zeroFactStorageobjects.execute_query_with_rulesstill convertsArc<[Fact]>back toFactStorageforStratifiedEvaluator(deferred).- ~62–65% speedup on non-rules queries at 10K facts:
query/point_entity/10k22 ms → 8.6 ms;aggregation/count_scale/10k28 ms → 9.7 ms. - Evaluator loop:
accumulated_factscomputed once per iteration (was 4 separateget_asserted_facts()calls).
PatternMatcher::from_slice(Arc<[Fact]>)constructor — creates a matcher from an immutable fact snapshot without index reconstruction.
apply_or_clausesandevaluate_not_joinsignatures updated to acceptArc<[Fact]>instead of&FactStorage.- 6 new tests: 4 in
matcher.rs(unit), 2 inexecutor.rs(unit).
- Total: 568 tests passing (390 unit + 172 integration + 6 doc)
- Disjunction (
or/or-join): queries and rule bodies can now use(or branch1 branch2 ...)and(or-join [?v...] branch1 branch2 ...)where-clauses. Branches support all other clause types includingnot,not-join,Expr, and nestedor/or-join.(and ...)groups multiple clauses into a single branch. match_patterns_seededonPatternMatcherfor seeded branch evaluation.evaluate_branchandapply_or_clausesaspub(crate)helpers inexecutor.rs.
WhereClauseenum gainsOr(Vec<Vec<WhereClause>>)andOrJoin { join_vars, branches }variants.DependencyGraph::from_rulesrefactored with recursivecollect_clause_depshelper;Or/OrJoinbranches contribute positive dependency edges.- Rules with
or/or-joinin their bodies route to themixed_rulespath inStratifiedEvaluator.
BinOpenum (14 variants:Lt,Gt,Lte,Gte,Eq,Neq,Add,Sub,Mul,Div,StartsWith,EndsWith,Contains,Matches) intypes.rsUnaryOpenum (5 variants:StringQ,IntegerQ,FloatQ,BooleanQ,NilQ) intypes.rsExprenum (Var,Lit,BinOp,UnaryOp) — composable expression AST intypes.rsWhereClause::Expr { expr: Expr, binding: Option<String> }variant —None= filter,Some(var)= arithmetic bindingparse_expr_arg/parse_expr/parse_expr_clauseinparser.rs; dispatch at all 4 clause sites (query:where, rule body,notbody,not-joinbody)- Parse-time regex validation for
matches?patterns viaregex-lite; invalid patterns are rejected with a clear error check_expr_safety+check_expr_safety_with_boundinparser.rs— forward-pass safety check; recurses intonot/not-joinbodies; unboundExpr::Varreferences are rejected at parse timeouter_vars_from_clauseupdated forWhereClause::Expr— binding variable contributes to scope for subsequent clauseseval_expr,eval_binop,is_truthy,apply_expr_clausesinexecutor.rs— evaluate expression trees against a binding; type mismatches and div/0 silently drop the rowapply_expr_clauses_in_evaluatorinevaluator.rs— sibling helper for rule body andnot-joinevaluation pathsnot_body_matchesinexecutor.rsupdated to seed with outer binding for expr-onlynotbodiestests/predicate_expr_test.rs— 28 integration tests covering all operators, silent-drop semantics, integer division, NaN, int/float promotion, string predicates, regex, expr innotbody, expr in rule body, bi-temporal + expr, arithmetic into aggregate
- Comparison operators (
<,>,<=,>=) require both operands to be numeric (IntegerorFloat); type mismatch → row dropped =/!=use structural equality onValue— type mismatch returnsfalse/true, not an error- Integer
+Floatpromotes toFloat; integer division truncates; division by zero → row dropped; NaN result → row dropped is_truthy:Boolean(true)→ true; non-zeroIntegerorFloat→ true; everything else (includingKeyword,Ref,Null, zero, empty string,Boolean(false),-0.0) → falsematches?pattern compiled at eval time viaregex-lite; pattern must be a string literal validated at parse time
- Added
tests/predicate_expr_test.rs(28 integration tests) - Total: 527 tests passing (365 unit + 156 integration + 6 doc)
- Aggregation in
:findclause:count,count-distinct,sum,sum-distinct,min,max :withgrouping clause — variables that participate in grouping but are excluded from output rowsAggFuncenum andFindSpecenum insrc/query/datalog/types.rs;DatalogQuery.findmigrated fromVec<String>toVec<FindSpec>;DatalogQuery.with_vars: Vec<String>field addedapply_aggregationpost-processing step inexecutor.rs— runs after binding collection when any aggregate is presentextract_variableshelper inexecutor.rs— non-aggregate extraction path (replaces inline loops)apply_agg_funcandvalue_type_namehelpers inexecutor.rsparse_aggregatehelper inparser.rs;:findarm extended to acceptEdnValue::List(aggregate expressions);:withkeyword arm added- Parse-time validation: aggregate variables must be bound in
:where;:withwithout any aggregate is rejected tests/aggregation_test.rs— 24 integration tests covering all aggregates,:with, rules, negation, temporal filters
count/count-distinctwith no grouping vars on zero bindings →[[0]](SQL behavior)- All other aggregates on zero bindings → empty result set
- All aggregates skip
Value::Nullsilently (SQL behavior) - Type mismatches (e.g.
sumonString) fail fast with a runtime error min/maxon mixedInteger/Floatis a runtime error:with ?vadds?vto the grouping key without adding it to output columns
- Added
tests/aggregation_test.rs(24 integration tests) - Total: 461 tests passing (327 unit + 128 integration + 6 doc)
src/query/datalog/stratification.rs—DependencyGraphandstratify(): analyse rule dependency graphs at registration time; programs with negative cycles are rejected with a clear errorWhereClause::Not(Vec<WhereClause>)andWhereClause::NotJoin { join_vars, clauses }variants intypes.rs; all exhaustive matches updated(not clause…)in:whereand rule bodies — stratified negation where all body variables must be pre-bound by outer clauses(not-join [?v…] clause…)— existentially-quantified negation with explicit join-variable declaration; body variables not injoin_varsare fresh/unbound- Safety check at parse time: every
notbody variable must be bound by an outer clause; everyjoin_varsvariable innot-joinmust be bound by an outer clause - Nesting constraint:
not-joincannot appear insidenotor anothernot-join— rejected at parse time StratifiedEvaluatorinevaluator.rs: stratifies rules, runs positive rules first, then appliesnot/not-joinfilters per binding for mixed rulesevaluate_not_joinfree function inevaluator.rs: builds partial binding fromjoin_vars, convertsPatternandRuleInvocationbody clauses to patterns, runsPatternMatcher; returnstrueif body is satisfiable (reject outer binding)rule_invocation_to_patternextracted aspub(super)free function fromRecursiveEvaluator- Two not-post-filter sites in
executor.rsnow handle bothNotandNotJoinviaevaluate_not_join tests/negation_test.rs— 10 integration tests fornot(Phase 7.1a): basic absence, multi-clause, rule body, time-travel, negative cycle rejectiontests/not_join_test.rs— 14 integration tests fornot-join(Phase 7.1b): basic exclusion, multiple join vars, multi-clause body, rule body,:as-of,:valid-at, negative cycle at registration,not+not-joincoexistence,RuleInvocationin body end-to-end
Rule.bodychanged fromVec<EdnValue>toVec<WhereClause>to support negation clauses alongside patternsexecutor.rsexecute_query_with_rulesnow delegates toStratifiedEvaluatorinstead ofRecursiveEvaluatordirectlyrules.rsregister_rulerunsstratify()after each registration; returnsErron negative cycle (rules are not registered on error)
src/storage/btree_v6.rs— proper on-disk B+tree for all four covering indexes (EAVT, AEVT, AVET, VAET); each B+tree node is one 4KB page (internal + leaf), withbuild_btreefor bulk-load andrange_scanfor leaf-chain traversalOnDiskIndexReaderstruct +CommittedIndexReadertrait — page-cache-backed index lookup replacing the full in-memory BTreeMap; index memory usage is now O(cache_pages), not O(facts)MutexStorageBackend<B>adapter — holds backend mutex only for the duration of a singleread_pagecall on a cache miss; cache-warm pages require no lock, enabling concurrent range scans to proceed in paralleltests/btree_v6_test.rs— 8 integration tests covering B+tree insert/range-scan, multi-page leaf chains, concurrent scan correctness with Barrier-synchronised threads, and v5→v6 migration roundtriptest_concurrent_range_scans_correctnessunit test inbtree_v6.rs— verifies all 8 concurrent threads return identical non-empty scan resultsbench_concurrent_btree_scanCriterion benchmark — measures wall-clock latency at 2/4/8 concurrent EAVT range scans; results updated inBENCHMARKS.mdFileHeaderv6 (80 bytes): addsfact_page_count u64field at bytes 72–80; automatic v5→v6 migration on first checkpoint
FORMAT_VERSIONbumped 5→6; v5 databases auto-migrated on first saveBENCHMARKS.mdupdated with v6 open/memory improvements, concurrent B+tree scan results, heaptrack v6 numbers, and a "How to read these numbers" methodology sectionREADME.mdandBENCHMARKS.md: performance table updated to reflect v6 open-time reduction (~2.4×) and peak-heap reduction (~21%)
- Concurrent B+tree range scans no longer serialise on cache-warm pages —
4→8 threadscaling ratio improved from ~2.2× to ~1.9×
BENCHMARKS.md— full Criterion benchmark results at 1K/10K/100K/1M facts with machine spec, HTML report references, and heaptrack memory profilesexamples/memory_profile.rs— heaptrack profiling binary; accepts fact count as positional argCargo.tomlmetadata:repository,keywords,categories,readme,documentationfields- Memory profile table in
README.md"Performance" section
README.mdPerformance section now links toBENCHMARKS.mdfor full benchmark details- Phase badge and status text updated to reflect Phase 6.4b completion
- crates.io publish deferred to Phase 7.8 (API cleanup + publish prep; file format v6 now complete)
- Dead
clapdependency from[dependencies]—clapwas listed but never imported in library or binary code
- Retraction semantics in Datalog queries:
filter_facts_for_queryStep 2 now computes the net view per(entity, attribute, value)triple vianet_asserted_facts(). Previously, retracted facts continued to appear in query results because the original assertion record remained in the append-only log. Now, for each EAV triple in the tx window, only the record with the highesttx_countis considered — if it is a retraction, the triple is excluded from results. - Oversized facts are now rejected early in
db.rs(check_fact_sizes) before any WAL write, using theMAX_FACT_BYTESconstant (4 080 bytes) exported frompacked_pages.rs. Previously, oversized facts could cause a panic deep in the page-packing path.
net_asserted_facts(facts: Vec<Fact>) -> Vec<Fact>helper insrc/graph/storage.rs: groups facts by EAV triple, keeps the record with the highesttx_count, and discards the triple if that record is a retraction. Used by bothexecutor.rsandstorage.rs.check_fact_sizes(facts: &[Fact])insrc/db.rs: validates all facts againstMAX_FACT_BYTESand returns a descriptiveErrbefore writing to the WAL.MAX_FACT_BYTES: usizeconstant insrc/storage/packed_pages.rs:PAGE_SIZE - PACKED_HEADER_SIZE - 4= 4 080 bytes.tests/retraction_test.rs— 7 integration tests covering: assert/retract with no:as-of, as-of snapshot before/after retraction boundary, re-assert after retract,:any-valid-timewith retraction, recursive rule retraction visibility at and before the retraction boundary.tests/edge_cases_test.rs— 4 integration tests covering: oversized-fact file-backed error path,MAX_FACT_BYTESexact boundary (accepted),MAX_FACT_BYTES + 1(rejected), in-memory database has no size limit.
- Packed fact pages (
page_type = 0x02): ~25 facts per 4KB page, ~25× disk space reduction vs v4 - LRU page cache (
src/storage/cache.rs): configurable capacity (default 256 pages = 1MB) OpenOptions::page_cache_size(usize)— tune page cache capacityCommittedFactReadertrait: index-driven fact resolution via page cache (no startup load-all)- File format v5:
fact_page_formatheader field; auto-migration from v4 on first open - Page-based CRC32 checksum (v5): streams raw committed pages instead of all facts
PersistentFactStorage::new()takespage_cache_capacity: usizeas second argument- Committed facts no longer loaded into
Vec<Fact>at startup; only pending facts held in memory FactStorage::get_facts_by_entity,get_facts_by_attributeuse EAVT/AEVT index range scans
- v4 databases auto-migrated to v5 packed format on first open (no data loss)
- Four Datomic-style covering indexes (EAVT, AEVT, AVET, VAET) with bi-temporal keys (
valid_from,valid_toin all key tuples) FactRef { page_id: u64, slot_index: u16 }— forward-compatible disk location pointer (slot_index=0 in 6.1)- Canonical value encoding (
encode_value) with sort-order-preserving byte representation - B+tree page serialization for index persistence (
src/storage/btree.rs) FileHeaderv4 (72 bytes): addseavt_root_page,aevt_root_page,avet_root_page,vaet_root_page(4×8=32 bytes),index_checksum(u32), replacing thereservedfield- CRC32 sync check on open: index mismatch triggers automatic rebuild
FactStorage::replace_indexes()andindex_counts()for index lifecycle management- Query optimizer (
src/query/datalog/optimizer.rs):IndexHintenum,select_index(),plan()with selectivity-based join reordering - Join reordering skipped under
wasmfeature flag Cargo.toml[features]section withdefault = []andwasm = []- 6 integration tests in
tests/index_test.rsfor save/reload, bi-temporal, recursive rules regression
FactStorageinternal structure:FactData { facts, indexes }under singleArc<RwLock<FactData>>for consistent snapshotsPersistentFactStorage::save()writes index B+tree pages and updates header checksumPersistentFactStorage::load()performs sync check and fast-path index loadexecutor::execute_query()now callsoptimizer::plan()before pattern matching- File format version bumped 3→4; automatic v1/v2/v3→v4 migration on first save
FORMAT_VERSIONconstant updated to 4
- NaN values in
Value::Floatnow canonicalize to a single bit pattern in index encoding (deterministic sort order)
- Write-ahead log (WAL): fact-level sidecar
<db>.walwith CRC32-protected binary entries WriteTransactionAPI:begin_write()/commit()/rollback()for explicit ACID transactions- Crash recovery: WAL entries replayed on open; corrupt/partial entries discarded at first bad CRC32
- Checkpoint:
checkpoint()flushes WAL facts to.graphand deletes the WAL; auto-checkpoint on configurable threshold FileHeaderv3:last_checkpointed_tx_countfield (repurposes unusededge_countslot)FactStoragehelpers:get_all_facts(),restore_tx_counter(),allocate_tx_count()OpenOptionsbuilder:OpenOptions::new().path("db.graph").open()orMinigraf::in_memory()--file <path>CLI flag for the REPL binary- 41 new tests covering WAL, crash recovery, transactions, and checkpoint
src/minigraf.rsreplaced bysrc/db.rs—Minigraf,OpenOptions,WriteTransactionpublic API- File format version bumped 2→3; automatic v1/v2→v3 migration on first checkpoint
- REPL version string now tracks
CARGO_PKG_VERSIONautomatically
- WAL-before-apply ordering: facts are now applied to in-memory state only after the WAL entry is fsynced, ensuring crash safety for both implicit (
execute()) and explicit (WriteTransaction) write paths
- Bi-temporal support: every fact now carries transaction time (
tx_id,tx_count) and valid time (valid_from,valid_to) :as-of Nquery modifier for transaction time travel (counter or ISO 8601 timestamp):valid-at "date"query modifier for valid time point-in-time queries:valid-at :any-valid-timeto disable valid time filtering(transact {:valid-from ... :valid-to ...} [...])syntax for specifying valid time- Per-fact valid time override in transact (4-element fact vectors with metadata map)
- File format version 2 with automatic migration from version 1
- Breaking behaviour: queries without
:valid-atnow return only currently valid facts (valid_from <= now < valid_to). Existing Phase 3 databases are unaffected because all migrated facts havevalid_to = MAX. FactStorage::transact()now accepts an optionalTransactOptionsparameter
PersistentFactStorage::load()previously discarded originaltx_idwhen loading facts from disk, making time-travel queries on persisted databases incorrect
- Datalog core implementation with recursive rules
- Entity-Attribute-Value (EAV) data model
- Pattern matching with variable unification
- Semi-naive evaluation for recursive rules
- Transitive closure support with cycle handling
- Rule registry for rule management
- Persistent storage with postcard serialization
- REPL with multi-line command support and comments
- 123 comprehensive tests (94 unit + 26 integration + 3 doc)
- Replaced GQL-inspired syntax with Datalog EDN syntax
- Data model changed from property graph to EAV triples
- Query executor rewritten for Datalog pattern matching
- Persistent storage backend with
.graphfile format - StorageBackend trait for platform abstraction
- FileBackend implementation (4KB pages, cross-platform)
- MemoryBackend for testing
- PersistentGraphStorage layer for serialization
- Embedded API (
Minigraf::open(),Minigraf::execute()) - Auto-save on drop
- Graph storage now supports persistence
- Initial release
- In-memory property graph implementation
- Basic graph operations (nodes, edges, properties)
- Interactive REPL
- Thread-safe storage with
Arc<RwLock<>>