From 2763d1fdf132968ea7c7c58bb6b9bf050c6f4ab8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:49:52 +0000 Subject: [PATCH] docs: fix unsafe WAL recovery advice and align docs with milestones - ERROR_REFERENCE: WAL-001/WAL-002/STG-011 said deleting the .wal loses no data, or that a database can be rebuilt from the WAL alone. Both are wrong. Replace with safe steps. Fix ~50 wiki links that pointed at a local .wiki path. - ROADMAP/CLAUDE.md: match the v2.0.2, v3.0.0 and v3.1.0 milestones. Document error.rs, magic_sets.rs, fault_inject.rs and src/browser/. - README: state MSRV 1.89, show the #371 known issue, fix Maven coordinates (io.github.project-minigraf), Android on Maven Central, and point binding downloads to the binding repos. - SECURITY.md: 2.x is the supported line. Refs #383, #397, #399 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01BBxgR4mnF8VMVGa42idcxx --- .github/SECURITY.md | 4 +- CHANGELOG.md | 7 +++ CLAUDE.md | 14 +++-- README.md | 25 +++++---- ROADMAP.md | 58 ++++++++++++++++----- docs/ERROR_REFERENCE.md | 111 +++++++++++++++++++++------------------- 6 files changed, 137 insertions(+), 82 deletions(-) diff --git a/.github/SECURITY.md b/.github/SECURITY.md index 9509777b8..41c5befe8 100644 --- a/.github/SECURITY.md +++ b/.github/SECURITY.md @@ -9,8 +9,8 @@ patched unless a separate security advisory explicitly says otherwise. | Version | Supported | | ------- | --------- | -| latest 1.x minor line | ✅ | -| older 1.x minor lines | ❌ | +| 2.x (latest release) | ✅ | +| 1.x | ❌ | | 0.x | ❌ | ## Reporting a Vulnerability diff --git a/CHANGELOG.md b/CHANGELOG.md index 4cb3be676..ef8d9b1b6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Bound-entity queries could drop rows when one `WriteTransaction` wrote the same attribute in several valid-time windows (#323).** Both facts share a `tx_count`, and the selective lookup path de-duplicated on `(entity, attribute, tx_count, asserted)`, so one window's value was silently lost, while the same query via a full scan returned both. That de-duplication has been removed, and bound-entity queries now always match a full scan. +### Documentation + +- **Unsafe WAL recovery advice removed from `docs/ERROR_REFERENCE.md`.** The WAL-001, WAL-002 and STG-011 resolutions said a `.wal` file could be deleted with no data loss, or that a database could be rebuilt from the WAL alone. Both are wrong: transactions committed since the last checkpoint exist only in the WAL, and the WAL holds nothing older than that checkpoint. The resolutions now say to copy both files first, to checkpoint with the version that wrote the WAL, and what is lost if the WAL is deleted. +- ROADMAP and CLAUDE.md now match the milestones: v2.0.2 is the final planned v2.x release, v3.0.0 is the format v8 and data-integrity release (#383), and features move to v3.1.0. +- README: MSRV (1.89) stated, v2.x known issue (#371) shown near the top, Maven coordinates corrected to `io.github.project-minigraf`, Android listed on Maven Central, and binding download locations point to the binding repos. +- `.github/SECURITY.md` lists 2.x as the supported line. About 50 broken wiki links in `docs/ERROR_REFERENCE.md` now use full wiki URLs. + ### Notes - On v2.x, reading a heavily rewritten attribute still costs time proportional to its history, because v7 index keys carry neither the value nor the assert/retract flag; the structural fix needs the v8 keys and is tracked for v3.0.0 in #379. diff --git a/CLAUDE.md b/CLAUDE.md index 78c9a8578..0d671c8d2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -107,6 +107,7 @@ cargo run < demos/demo_negation.txt - `mod.rs`: `StorageBackend` trait, `FileHeader` v7 (84 bytes), `CommittedFactReader` / `CommittedIndexReader` traits - `backend/file.rs`: Single `.graph` file backend (4KB pages, cross-platform) - `backend/memory.rs`: In-memory backend for testing + - `backend/fault_inject.rs`: `FaultInjectingBackend` — injects I/O errors for durability tests (test builds only) - `index.rs`: EAVT / AEVT / AVET / VAET key types, `FactRef`, `encode_value` - `btree_v6.rs`: On-disk B+tree (`build_btree`, `OnDiskIndexReader`, `MutexStorageBackend`) - `btree.rs`: Legacy v5 B+tree (migration only) @@ -124,6 +125,7 @@ cargo run < demos/demo_negation.txt - `types.rs`: `EdnValue`, `Pattern`, `DatalogQuery`, `AsOf`, `ValidAt`, `WhereClause` (incl. `Not`, `NotJoin`); `PseudoAttr` enum, `AttributeSpec` wrapper - `optimizer.rs`: Selectivity-based join reordering; disabled under `wasm` feature - `prepared.rs`: `BindValue`, `PreparedQuery` — parse-once/execute-many with named `$slot` bind slots; `prepare_query`, `substitute` + - `magic_sets.rs`: Magic-sets rewriting for demand-driven recursive rule evaluation (not applied to rules with `not`/`not-join`) 4. **`src/temporal.rs`** — UTC-only timestamp parsing (avoids chrono CVE GHSA-wcg3-cvx6-7396) @@ -133,6 +135,10 @@ cargo run < demos/demo_negation.txt 7. **`src/wal.rs`** — Fact-level sidecar WAL, CRC32-protected entries, crash recovery +8. **`src/error.rs`** — Structured error codes: `MinigrafError`, `ErrorCategory`, `ErrorCode` registry (PRS/QRY/STG/WAL/API/INT codes); must match `docs/ERROR_REFERENCE.md` + +9. **`src/browser/`** — Browser WASM backend (`browser` feature): `buffer.rs` (`BrowserBufferBackend`, in-memory pages with dirty tracking), `indexeddb.rs` (IndexedDB persistence) + ### Data Model ```rust @@ -176,9 +182,11 @@ See `docs/TEST_COVERAGE.md` for the full per-file breakdown. **v2.0.1 released** (2026-09-25) — patch fixing index rebuild on open (#370). **v2.0.0** introduced kernel file locking (#317, #304), structured runtime error codes (#277), `OpenOptions` `#[non_exhaustive]`, and an MSRV of Rust 1.89. See `CHANGELOG.md` for the full rationale and release history. -Relevant areas for the planned query-profiler work (see `ROADMAP.md`): -- `src/query/datalog/` — query executor where profiling hooks will go (#185) -- `docs/BENCHMARKS.md` — post-1.0 performance baseline updates +Current plan (see `ROADMAP.md` and the milestones): +- **v2.0.2** (final planned v2.x release, format v7, no API change) ships from `main`. +- **v3.0.0** (format v8 and data integrity, tracker #383) is developed on the long-lived `v3` branch. Merge `main` into `v3` regularly. +- **v3.1.0** holds new features (query profiler #185, `:limit` #306/#310, lag/lead #182, UniFFI additions). +- Known v2.x data issue: same-transaction multi-valued facts (#371). Fix ships in v3.0.0. ## Testing Conventions diff --git a/README.md b/README.md index 874758f15..ff6a23c5b 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,8 @@ [![Clippy Status](https://github.com/project-minigraf/minigraf/actions/workflows/rust-clippy.yml/badge.svg)](https://github.com/project-minigraf/minigraf/actions/workflows/rust-clippy.yml) [![Coverage](https://codecov.io/gh/project-minigraf/minigraf/branch/main/graph/badge.svg)](https://codecov.io/gh/project-minigraf/minigraf) [![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue.svg)](https://github.com/project-minigraf/minigraf#license) -[![Rust Edition](https://img.shields.io/badge/rust-2024-orange.svg)](https://blog.rust-lang.org/2024/10/17/Rust-1.82.0.html) +[![Rust Edition](https://img.shields.io/badge/rust-2024-orange.svg)](https://doc.rust-lang.org/edition-guide/rust-2024/index.html) +[![MSRV](https://img.shields.io/badge/MSRV-1.89-orange.svg)](Cargo.toml) > **Embedded graph memory for AI agents, mobile apps, and the browser** — the SQLite of bi-temporal graph databases @@ -50,6 +51,10 @@ Or via cargo: cargo add minigraf ``` +Requires Rust 1.89 or newer (the minimum supported Rust version, set in `Cargo.toml`). + +**Known issue on v2.x:** two values of the same attribute for one entity written in a single `transact` (or retracted in a single `retract`) can read back as one value (#371). Write or retract each value of a multi-valued attribute in its own call. The fix changes the file format and ships in v3.0.0. See the [CHANGELOG](CHANGELOG.md). + ## Quick Start ```rust @@ -114,7 +119,7 @@ No other database offers this combination: | **Embedded** | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | ✅ Yes | | **Graph Native** | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | | **Rust** | ✅ Yes | ❌ Clojure | ✅ Yes | ❌ Java | ❌ C | -| **WASM Ready** | ✅ Yes (browser + WASI + 6 targets) | ❌ No | ⚠️ Limited | ❌ No | ✅ Yes | +| **WASM Ready** | ✅ Yes (browser + WASI) | ❌ No | ⚠️ Limited | ❌ No | ✅ Yes | ## Platform support @@ -122,13 +127,13 @@ No other database offers this combination: |---|---|---| | Rust (native) | `minigraf` on crates.io | `cargo add minigraf` | | Browser WASM | `@minigraf/browser` on npm | `npm install @minigraf/browser` | -| WASI | `@minigraf/wasi` on npm, `.wasm` on GitHub Releases | `npm install @minigraf/wasi` | +| WASI | `@minigraf/wasi` on npm | `npm install @minigraf/wasi` | | Node.js | `minigraf` on npm | `npm install minigraf` | | Python | `minigraf` on PyPI | `pip install minigraf` | -| Java/JVM | `io.github.adityamukho:minigraf-jvm` on Maven Central | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | -| Android | `.aar` on GitHub Packages | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | -| iOS / macOS | `.xcframework` via Swift Package Manager | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | -| C / FFI | header + tarball on GitHub Releases | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | +| Java/JVM | `io.github.project-minigraf:minigraf-jvm` on Maven Central | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | +| Android | `io.github.project-minigraf:minigraf-android` (`.aar`) on Maven Central | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | +| iOS / macOS | `.xcframework` via Swift Package Manager ([minigraf-swift](https://github.com/project-minigraf/minigraf-swift)) | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | +| C / FFI | header + tarball on [minigraf-c releases](https://github.com/project-minigraf/minigraf-c/releases) | see [wiki](https://github.com/project-minigraf/minigraf/wiki/Use-Cases) | **Embedded graph memory for agents, mobile, and the browser — SQLite's simplicity + Datomic's temporal model.** @@ -140,8 +145,8 @@ No other database offers this combination: | Node.js | [`minigraf` on npm](https://www.npmjs.com/package/minigraf) | [minigraf-node](https://github.com/project-minigraf/minigraf-node) | | Browser WASM | [`@minigraf/browser` on npm](https://www.npmjs.com/package/@minigraf/browser) | [minigraf-wasm](https://github.com/project-minigraf/minigraf-wasm) | | WASI | [`@minigraf/wasi` on npm](https://www.npmjs.com/package/@minigraf/wasi) | [minigraf-wasm](https://github.com/project-minigraf/minigraf-wasm) | -| Java | `minigraf-jvm` on Maven Central | [minigraf-java](https://github.com/project-minigraf/minigraf-java) | -| Android | Android bindings | [minigraf-android](https://github.com/project-minigraf/minigraf-android) | +| Java | `io.github.project-minigraf:minigraf-jvm` on Maven Central | [minigraf-java](https://github.com/project-minigraf/minigraf-java) | +| Android | `io.github.project-minigraf:minigraf-android` on Maven Central | [minigraf-android](https://github.com/project-minigraf/minigraf-android) | | iOS/macOS | Swift bindings | [minigraf-swift](https://github.com/project-minigraf/minigraf-swift) | | C | C bindings | [minigraf-c](https://github.com/project-minigraf/minigraf-c) | @@ -155,7 +160,7 @@ Pairs well with vector stores (GraphRAG pattern): the vector store answers "what ### For Mobile Apps -Offline-first storage with retroactive corrections — the bi-temporal model lets you correct a mis-entered value while preserving the original record. Native Kotlin and Swift bindings ship as an Android `.aar` (GitHub Packages) and an iOS `.xcframework` (Swift Package Manager) via [UniFFI](https://github.com/mozilla/uniffi-rs). No Rust required. +Offline-first storage with retroactive corrections — the bi-temporal model lets you correct a mis-entered value while preserving the original record. Native Kotlin and Swift bindings ship as an Android `.aar` (Maven Central) and an iOS `.xcframework` (Swift Package Manager) via [UniFFI](https://github.com/mozilla/uniffi-rs). No Rust required. ```kotlin // Android (Kotlin) diff --git a/ROADMAP.md b/ROADMAP.md index 45b27a6b2..0babc1f57 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,6 @@ # Minigraf Roadmap -> The path from a production-ready bi-temporal Datalog database to a stronger ecosystem. +> The path to a production-ready bi-temporal Datalog database, and then to a stronger ecosystem. Production readiness is tracked in #383. **Philosophy**: Embedded graph memory for agents, mobile, and the browser — built on the SQLite approach: be boring, be reliable, be embeddable. @@ -8,20 +8,51 @@ Completed releases and their implementation details live in the [CHANGELOG](CHAN --- -## v3.0.0 — File Format Policy +## v2.0.2 — Final Planned v2.x Release -**v3.0.0 supports file format v8 only.** Fixing #287's `EavtKey`/`AevtKey` missing-value-bytes data-loss bug bumps the format from v7 to v8. Support for v1–v6 is dropped at the same time. The v1–v6 → v7 auto-migration code in `persistent_facts.rs` can be removed when cutting this release; a v7→v8 migration replaces it. Any database opened at least once under a v1.x release will already be on v7; there are no known users on older formats. +v2.0.2 is a patch release on file format v7 with no API change. It is the last planned v2.x release. New feature work targets v3.x. + +Scope (milestone [v2.0.2](https://github.com/project-minigraf/minigraf/milestone/8)): + +- Merged performance mitigations (#323, #381, #315) +- Parent-directory fsync after creating or deleting files (#389) +- `or-join` returning an error when the clauses before it match no rows (#405) +- A clear error for `$slot` queries run through `execute()` (#407) +- CI hardening: benchmark alerting (#393), MSRV, semver and dependency checks (#395) +- Stability, support and known-issues docs and process (#397, #399, #400). These define how v2.x is supported after v3.0.0. + +Known issue on v2.x: same-transaction multi-valued facts can read back as one value (#371). The fix needs file format v8 and ships in v3.0.0. Workaround: write or retract each value of a multi-valued attribute in its own call. + +--- + +## v3.0.0 — File Format v8 and Data Integrity + +**v3.0.0 supports file format v8 only.** Fixing the #287/#371 data-loss bug adds the value to the `EavtKey`/`AevtKey` index keys, which bumps the format from v7 to v8. Support for v1–v6 is dropped at the same time. The v1–v6 → v7 auto-migration code in `persistent_facts.rs` can be removed when cutting this release; a v7→v8 migration replaces it. Any database opened at least once under a v1.x or v2.x release is already on v7. This was the GitHub milestone named “2.0” before v2.0.0 used that version number for the kernel-locking and structured-error-code breaking changes; it was consequently renumbered to v3.0.0. -Scope: +Scope (milestone [v3.0.0](https://github.com/project-minigraf/minigraf/milestone/1)): + +- v8 index keys and v7→v8 migration (#287, #371; done on the `v3` branch) +- Per-page checksums for fact and B+tree pages (#388) +- Crash-atomic `save()` (#374) +- Index verify and rebuild (#373) +- O(live) point queries on v8 keys (#379) +- Release-gate test suite: golden files (#391), crash data checks (#384), model-based tests (#385), wider property tests (#386), fuzz corpus and operation-sequence target (#387, #375), fault injection (#390), soak test (#392), coverage gates (#396) +- Performance envelope (#394) and durability guide (#398) + +--- + +## v3.1.0 — Additive Features + +Scope (milestone [v3.1.0](https://github.com/project-minigraf/minigraf/milestone/9)): -- `EavtKey`/`AevtKey` missing-value-bytes fix and file-format v7→v8 migration (#287) -- `lag`/`lead` window functions (#182) -- Sliding row frames — `:rows N preceding` (#183; builds on #182) -- `PreparedQuery` over UniFFI (#181) -- UDF registration over UniFFI (#180) -- `OpenOptions` exposed over UniFFI, so embedders can suppress the close-time checkpoint (#322) +- Rule persistence (#241) +- Query profiler (#185) +- `:limit` / `:offset` (#306, #310) +- Set membership predicates (#316) +- `lag`/`lead` window functions (#182) and sliding row frames, `:rows N preceding` (#183) +- UniFFI: `PreparedQuery` (#181), UDF registration (#180), `OpenOptions` (#322) - Temporal graph-traversal research (#273; may result in documentation or a blog post rather than code) --- @@ -105,8 +136,9 @@ When evaluating features, ask: ## Current Focus -- v2.x minor releases (v2.1.0–v2.3.0) on `main` -- The v3.0.0 file-format policy and its scoped work above, on the `v3` branch +- v2.0.2, the final planned v2.x release, on `main` +- v3.0.0 (format v8 and data integrity) on the `v3` branch, tracked in #383 +- v3.1.0 features after v3.0.0 ships - Ecosystem work tracked in [`minigraf-examples`](https://github.com/project-minigraf/minigraf-examples) - Developer tools tracked in `minigraf-inspector` and `minigraf-visualizer` @@ -114,4 +146,4 @@ See [GitHub Issues](https://github.com/project-minigraf/minigraf/issues) for spe --- -**Last updated**: September 2026 — v2.0.1 is the current release. v2.x minor releases (v2.1.0–v2.3.0) ship from `main` first; v3.0.0 work, including the file-format v8 fix for #287/#371, is developed on the long-lived `v3` branch and merges into `main` at the v3.0.0 cut. +**Last updated**: September 2026. v2.0.1 is the current release. v2.0.2 is the final planned v2.x release and ships from `main`. v3.0.0 work, including the file-format v8 fix for #287/#371, is developed on the long-lived `v3` branch and merges into `main` at the v3.0.0 cut. New features follow in v3.1.0. diff --git a/docs/ERROR_REFERENCE.md b/docs/ERROR_REFERENCE.md index d4b1fe9df..84538bd10 100644 --- a/docs/ERROR_REFERENCE.md +++ b/docs/ERROR_REFERENCE.md @@ -215,7 +215,7 @@ with no `CodedError` anywhere in its chain. Parser errors occur when Minigraf cannot parse the Datalog/EDN input string. They are returned immediately from `db.execute()` before any fact is read or written. -See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidance. +See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference) for syntax guidance. ### PRS-001 Unexpected end of input @@ -259,7 +259,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Check surrounding delimiters for balance. -- Consult the [Datalog Reference](../../.wiki/Datalog-Reference.md) for expected syntax at that position. +- Consult the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference) for expected syntax at that position. **Example**: ```datalog @@ -431,7 +431,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add a value: `:as-of 5` (transaction count) or `:as-of "2024-01-01T00:00:00Z"` (ISO 8601 wall-clock time). -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -448,7 +448,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use a non-negative integer. - `:as-of 0` returns the database before any transaction; `:as-of 1` returns the state after the first transaction. -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -464,7 +464,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `42` (tx-count) or `"2024-01-01T00:00:00Z"` (wall-clock). -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -480,7 +480,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add an ISO 8601 timestamp: `:valid-at "2024-06-01T00:00:00Z"`. -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -496,7 +496,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `"2024-06-01T00:00:00Z"` or `:any-valid-time`. -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -512,7 +512,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `"2024-01-01T00:00:00Z"` format. -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -529,7 +529,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `"2024-12-31T23:59:59Z"` format. - To express "forever", omit `:valid-to` (the default is `VALID_TIME_FOREVER`). -- See the [Datalog Reference — Time Travel](../../.wiki/Datalog-Reference.md#time-travel). +- See the [Datalog Reference — Time Travel](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#bi-temporal-queries). **Example**: ```datalog @@ -545,7 +545,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add an aggregate to `:find`, or remove `:with` if no aggregation is needed. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -563,7 +563,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add a `:where` clause that binds the variable, or remove it from `:with`. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -581,7 +581,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add a `:where` clause that binds the variable. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -598,7 +598,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `(count ?e)`, `(sum ?amount)`, `(avg ?score)` — one function name, one variable. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -615,7 +615,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `sum`, not `:sum`. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -632,7 +632,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use a `?variable`, not a literal value. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -649,7 +649,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add an `:over` clause: `(rank :over (:order-by ?score))`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -666,7 +666,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Provide at least `:order-by` or `:partition-by` inside the `:over` list. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -683,7 +683,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `rank`, not `:rank`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -717,7 +717,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Drop the `:over` clause and use the function as a plain aggregate. -- See the [Datalog Reference — Aggregates](../../.wiki/Datalog-Reference.md#aggregates). +- See the [Datalog Reference — Aggregates](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#aggregation). **Example**: ```datalog @@ -734,7 +734,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add the variable argument before `:over`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -751,7 +751,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add `:over` after the variable. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -768,7 +768,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Remove the variable: `(rank :over (:order-by ?score))`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -785,7 +785,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Always follow `:over` with a list: `(rank :over (:order-by ?score))`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -802,7 +802,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - The window expression must end after the `:over` clause. Remove trailing tokens. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -819,7 +819,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use a `?variable`: `(rank :over (:partition-by ?dept :order-by ?score))`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -836,7 +836,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use a `?variable`: `(rank :over (:order-by ?score))`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -853,7 +853,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use only `:order-by` and `:partition-by`. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -870,7 +870,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - The `:over` list must contain only `:order-by` / `:partition-by` followed by variables. -- See the [Datalog Reference — Window Functions](../../.wiki/Datalog-Reference.md#window-functions). +- See the [Datalog Reference — Window Functions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#window-functions). **Example**: ```datalog @@ -1024,7 +1024,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Remove the empty list or replace it with a valid clause. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference). **Example**: ```datalog @@ -1042,7 +1042,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Double negation is logically equivalent to the positive pattern; use the pattern directly. - For complex negation use `not-join` with explicit join variables. -- See the [Datalog Reference — Negation](../../.wiki/Datalog-Reference.md#negation). +- See the [Datalog Reference — Negation](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#negation). **Example**: ```datalog @@ -1059,7 +1059,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add at least one pattern inside `(not ...)`. -- See the [Datalog Reference — Negation](../../.wiki/Datalog-Reference.md#negation). +- See the [Datalog Reference — Negation](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#negation). **Example**: ```datalog @@ -1076,7 +1076,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add at least one branch. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md#disjunction). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#disjunction). **Example**: ```datalog @@ -1093,7 +1093,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use the form `(or-join [?e] [[?e :a 1]] [[?e :b 2]])`. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md#disjunction). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#disjunction). **Example**: ```datalog @@ -1110,7 +1110,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - The form is `(or-join [?var1 ?var2] ...)`. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md#disjunction). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#disjunction). **Example**: ```datalog @@ -1127,7 +1127,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `?variable` not `:keyword` or a string. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md#disjunction). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#disjunction). **Example**: ```datalog @@ -1145,7 +1145,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Restructure branches so they all bind the same new variables. - Or use `or-join` to specify exactly which variables to share. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md#disjunction). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#disjunction). **Example**: ```datalog @@ -1163,7 +1163,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Add at least one clause inside `(and ...)`. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md#disjunction). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#disjunction). **Example**: ```datalog @@ -1180,7 +1180,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `(not-join [?e] [?e :deleted true])`. -- See the [Datalog Reference — Negation](../../.wiki/Datalog-Reference.md#negation). +- See the [Datalog Reference — Negation](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#negation). **Example**: ```datalog @@ -1197,7 +1197,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Each `:where` element must be a vector `[e a v]`, a `(not ...)`, `(not-join ...)`, `(or ...)`, `(or-join ...)`, or expression `[(expr) ?out]`. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference). **Example**: ```datalog @@ -1215,7 +1215,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Valid query map keys are `:find`, `:where`, `:with`, `:as-of`, `:valid-at`. - Check spelling. -- See the [Datalog Reference](../../.wiki/Datalog-Reference.md). +- See the [Datalog Reference](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference). **Example**: ```datalog @@ -1231,7 +1231,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Provide a function: `[(+ ?a ?b) ?sum]`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1248,7 +1248,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use `+` not `:+`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1266,7 +1266,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Check the argument count. - Single-argument operators: `abs`, `not`, `str` (for coercion). -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1284,7 +1284,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Check the argument count. - Two-argument operators: `+`, `-`, `*`, `/`, `mod`, `quot`, `=`, `!=`, `<`, `<=`, `>`, `>=`, `starts-with?`, `ends-with?`, `contains?`, `matches?`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1301,7 +1301,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Pass the pattern as a string literal: `[(matches? ?name "alice.*")]`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1319,7 +1319,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Check spelling against the supported operator list above. - For missing operators, register a custom predicate via `db.register_predicate()`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1336,7 +1336,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Ensure the expression is wrapped in exactly one outer `[...]`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1353,7 +1353,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Use a `?variable`: `[(+ ?a ?b) ?sum]`. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1371,7 +1371,7 @@ See the [Datalog Reference](../../.wiki/Datalog-Reference.md) for syntax guidanc **Resolution**: - Expressions accept variables (`?x`), integers, floats, strings, booleans, and keywords. - Restructure to avoid passing complex types. -- See the [Datalog Reference — Expressions](../../.wiki/Datalog-Reference.md#expressions). +- See the [Datalog Reference — Expressions](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#arithmetic--predicate-expressions). **Example**: ```datalog @@ -1791,7 +1791,9 @@ See the [file format section in README](../README.md#file-format) for version hi **Cause**: An internal B+tree page was found with an empty children list, which violates B+tree invariants. This indicates index corruption. **Resolution**: -- Restore from backup. If the corruption occurred after a recent write, the WAL file may contain a recoverable state — delete the `.graph` file and try opening from the WAL only (by restoring the last good checkpoint and replaying the WAL). See the [file format section in README](../README.md#file-format). +- Stop writing to the database and copy both the `.graph` file and its `.wal` sidecar before trying anything else. +- Restore from backup. Do not delete the `.graph` file: the WAL holds only the transactions written since the last checkpoint, so it cannot rebuild the database on its own. If you restore an older `.graph` file, you can keep the matching `.wal` only if it was written after that file's last checkpoint. +- Index integrity checking and rebuilding from fact pages are tracked in #373. **Scenario**: A crash mid-checkpoint left an internal B+tree page in an inconsistent state. @@ -1998,7 +2000,8 @@ The WAL is replayed on open and deleted on checkpoint. **Cause**: The sidecar `.wal` file does not start with the expected WAL magic bytes. The file may have been replaced, corrupted, or created by an incompatible tool. **Resolution**: -- If the WAL file is stale or corrupt, delete `.wal` and reopen the database — Minigraf will replay only from the committed state in the `.graph` file. +- Copy both the `.graph` file and the `.wal` file before changing anything. +- Deleting `.wal` makes the database open, but **every transaction committed since the last checkpoint is lost**, because those transactions exist only in the WAL. Do this only if you have a backup or accept that loss. - Do not manually create or edit `.wal` files. **Scenario**: `my-db.wal` was accidentally replaced with an empty file before `Minigraf::open("my-db.graph")` was called. @@ -2010,8 +2013,8 @@ The WAL is replayed on open and deleted on checkpoint. **Cause**: The `.wal` file was written by a version of Minigraf with a different WAL format. This can occur when downgrading the library after a WAL was written by a newer version. **Resolution**: -- Delete the `.wal` file if it is from an incomplete or stale session (no data is lost — committed facts are in the `.graph` file). -- If the WAL contains uncommitted in-flight data you need to recover, upgrade the library to the version that wrote the WAL before reopening. +- Open the database with the Minigraf version that wrote the WAL and call `checkpoint()`. This moves the WAL's transactions into the `.graph` file and removes the WAL. Then open it with the new version. +- Deleting the `.wal` file instead loses **every transaction committed since the last checkpoint**, because those transactions exist only in the WAL. Do this only if you have a backup or accept that loss. **Scenario**: A `.wal` file written by a pre-release version of Minigraf is opened with the stable release, which uses a different WAL version number — e.g. `Unsupported WAL version: 3 (expected 2)`. @@ -2025,7 +2028,7 @@ The WAL is replayed on open and deleted on checkpoint. - Store large payloads in an external file or object store. - Store the file path or URL as a `Value::String` attribute on the entity. - Or create a dedicated entity for the content and reference it with `Value::Ref`. -- See [BENCHMARKS.md](../BENCHMARKS.md) for size guidance. +- See the [Datalog Reference — Constraints and Limits](https://github.com/project-minigraf/minigraf/wiki/Datalog-Reference#constraints-and-limits) for the size limit. **Example**: ```datalog