Skip to content
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- **`:query-memory-limit`, limit bind slots and `PreparedQuery` limit setters** (#870, part of #862). `:query-memory-limit N` replaces `OpenOptions::query_memory_limit` for one query, up or down; `0` is allowed and lets only a query that holds no state run (QRY-012 otherwise). It is gated by the new `Overridable::QueryMemoryLimit`, which is **not** in `Overridable::DEFAULTS`, so a database refuses it with API-025 unless the embedder lists it; `BrowserDb` always refuses it. `:max-derived-facts`, `:max-results` and `:query-memory-limit` take a `$slot` filled with `BindValue::Val(Value::Integer(n))` (INT-014/INT-015 out of range as for a literal, API-024 for another type, API-022 when missing). `PreparedQuery::with_max_derived_facts` and `with_query_memory_limit` replace the database values for one statement; they are not gated, and an allowed keyword still overrides them. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md). The v3 binary-size gate rises from 1,234 to 1,240 KiB (+672 B measured here; the rest is room for #861).

- **Per-transaction WAL sync mode and chunk size: `TxOptions` and `Minigraf::begin_write_with`** (#871, part of #862). `TxOptions` (`#[non_exhaustive]`; fields `synchronous: Option<SyncMode>` and `chunk_facts: Option<usize>`, builders `new`, `synchronous`, `chunk_facts`) sets either option for one `WriteTransaction`; `None` means the database's value, and `begin_write()` is `begin_write_with(TxOptions::default())`. `synchronous` decides whether that transaction's WAL entry is synced, whatever `OpenOptions::synchronous` is, so a bulk load can commit with `Normal` on a `Full` database; a `Full` commit syncs the whole WAL, so earlier `Normal` entries become durable with it. It has no effect on a transaction that becomes large (a checkpoint, durable in both modes; its WAL guard header is always synced, a deviation from the #862 design recorded there) or on an in-memory database. `chunk_facts` replaces `OpenOptions::transaction_chunk_facts` for that transaction, with the same clamp (1..=1,000,000). The options are not stored in the file, not gated by `allow_overrides`, and apply to that transaction only. `begin_write_with` fails as `begin_write` does (`INT-001`, `API-014`). The WAL writer no longer keeps a sync mode: each append is given its transaction's. See [docs/DURABILITY.md](docs/DURABILITY.md) for what survives a crash when modes are mixed.

- **Configurable recursion round limit** (#861, part of #862). `OpenOptions::max_iterations` sets the most rounds one stratum of a rule query may run (default 1,000,000, see Breaking changes); `:max-iterations N` or `$slot` replaces it for one query, up or down (INT-012/013/014/015 as for `:max-derived-facts`), gated by the new `Overridable::MaxIterations`, which **is** in `Overridable::DEFAULTS`; `PreparedQuery::with_max_iterations` replaces it for one statement, ungated, and an allowed keyword still overrides it. Rounds are still counted per stratum. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md).

### Changed
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,13 +187,13 @@ cargo run < demos/demo_negation.txt

5. **`src/repl.rs`** — Interactive REPL; TTY-aware (suppresses prompts/banner for piped input)

6. **`src/db.rs`** — Public API: `Minigraf::open/execute/query/fact_log/prepare/begin_write/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `IntegrityReport`, `OpenOptions::page_cache_size`, `OpenOptions::read_only` (#429: shared lock, nothing written, writes are `API-014`), `OpenOptions::query_memory_limit` (#549: `QRY-012`, off by default), `OpenOptions::transaction_chunk_facts` (#550), `OpenOptions::max_iterations` (#861: rounds per stratum, default 1,000,000), `OpenOptions::allow_overrides` (#869: which query keywords may override settings, `API-025`; see `docs/CONFIGURATION.md`)
6. **`src/db.rs`** — Public API: `Minigraf::open/execute/query/fact_log/prepare/begin_write/begin_write_with/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `TxOptions` (#871: one transaction's `synchronous` and `chunk_facts`; `None` = the database value; the WAL writer keeps no mode, each append gets its transaction's; no effect on a large transaction, whose WAL guard is always synced), `IntegrityReport`, `OpenOptions::page_cache_size`, `OpenOptions::read_only` (#429: shared lock, nothing written, writes are `API-014`), `OpenOptions::query_memory_limit` (#549: `QRY-012`, off by default), `OpenOptions::transaction_chunk_facts` (#550), `OpenOptions::max_iterations` (#861: rounds per stratum, default 1,000,000), `OpenOptions::allow_overrides` (#869: which query keywords may override settings, `API-025`; see `docs/CONFIGURATION.md`)
- `src/overrides.rs`: `Overridable` (`DEFAULTS`, `keyword`; `QueryMemoryLimit` not in `DEFAULTS`, #870; `MaxIterations` in it, #861) and `check_overrides`, called after parsing at every query entry point (`BrowserDb` uses `DEFAULTS`)
- `src/cursor.rs`: `Cursor` / `Batch` — owned, `Send` cursor returned by `Minigraf::query` and `PreparedQuery::query` (#432); pulls engine queries one internal batch at a time (`close`/drop stops the work; a read error after some rows comes on the next call), other queries computed at open; holds a `Snapshot` (one state of the store, its committed generation pinned, #543) until dropped
- `src/fact_log.rs`: `FactLog` / `FactFilter` / `FactRecord` / `FactOrder` — every fact version streamed by `Minigraf::fact_log` (#430): tx order (multi-pass, bounded window) or storage order, key-level filters; an open log pins the committed generation (`storage::pins`), so checkpoints run but do not reuse its pages (`API-013` deprecated, #543)
- `src/log_writer.rs`: `LogWriter` — builds a new file from `FactRecord`s with their tx and valid-time kept (#431): tx-ordered appends with holes (`API-015`/`API-016`), batched copy-on-write checkpoints at tx boundaries, no WAL, built at `<path>.partial` and renamed by `finish` (`STG-043` if the target exists)

7. **`src/wal.rs`** — Fact-level sidecar WAL, CRC32-protected entries, crash recovery. Like `storage::verify` and `storage::dir_sync`, not compiled for wasm32 (no file backend there)
7. **`src/wal.rs`** — Fact-level sidecar WAL, CRC32-protected entries, crash recovery; `append_entry` takes the `SyncMode` of each append (#871), and in tests the writer tracks its synced length (lost-sync fault, `Minigraf::power_loss`). Like `storage::verify` and `storage::dir_sync`, not compiled for wasm32 (no file backend there)

8. **`src/error.rs`** — Structured error codes: `MinigrafError`, `ErrorCategory`, `ErrorCode` registry (PRS/QRY/STG/WAL/API/INT codes); must match `docs/ERROR_REFERENCE.md`. `MinigrafError::invalid_argument` (API-017) and `MinigrafError::closed` (API-018) exist for language bindings

Expand Down Expand Up @@ -250,7 +250,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**2108 tests** (2087 active, 21 ignored; unit + integration + doc). Engine answers once checked by the executor are frozen in `tests/answers/` (#551).
**2132 tests** (2111 active, 21 ignored; unit + integration + doc). Engine answers once checked by the executor are frozen in `tests/answers/` (#551).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

The #760 deterministic storage suite adds three native unit tests in a separate `--cfg minigraf_shuttle` build, outside the default count. Run only `graph::storage::deterministic::` with that cfg; ordinary unit tests do not enter its scheduler. `deterministic-schedules.yml` runs relevant v3 PRs on Linux/macOS/Windows plus Linux MSRV, and larger nightly/manual searches with replay artifacts. The identical main workflow activates the v3 schedule after both PRs merge. See `docs/testing/deterministic-schedules.md` for scope, controls and findings.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ Point lookups, time travel on one entity, and joins and rule queries that a cons

File-backed databases limit string values to 4,068 bytes (`WAL-003`). In-memory databases have no limit.

**Durability tuning:** every write is `fsync`'d immediately by default (`SyncMode::Full`). Bulk loaders/migrations that can safely re-run from a checkpoint watermark on failure can trade that for throughput with `OpenOptions::new().synchronous(SyncMode::Normal)` — `checkpoint()` still fsyncs unconditionally in both modes. See the [Performance Tuning](https://project-minigraf.github.io/minigraf-docs/latest/performance-tuning/#configuration-knobs) for the full tradeoff and the write-batching pattern that pairs with it.
**Durability tuning:** every write is `fsync`'d immediately by default (`SyncMode::Full`). Bulk loaders/migrations that can safely re-run from a checkpoint watermark on failure can trade that for throughput with `OpenOptions::new().synchronous(SyncMode::Normal)`, or for one transaction with `db.begin_write_with(TxOptions::new().synchronous(SyncMode::Normal))` on an otherwise `Full` database — `checkpoint()` still fsyncs unconditionally in both modes, and a later `Full` commit also makes earlier `Normal` ones durable. See the [Performance Tuning](https://project-minigraf.github.io/minigraf-docs/latest/performance-tuning/#configuration-knobs) for the full tradeoff and the write-batching pattern that pairs with it.

**Configuration:** [docs/CONFIGURATION.md](docs/CONFIGURATION.md) lists every `OpenOptions` setting, which ones query text can override, and how `OpenOptions::allow_overrides` lets the embedder refuse those keywords (`API-025`, #869).

Expand Down
4 changes: 2 additions & 2 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ code are not gated.
| `max_results` (deprecated) | no limit of its own | query text: `:max-results N` or `$slot` | Alone, replaces the database value; with `:max-derived-facts`, the lower applies |
| `query_memory_limit` | no limit | query text: `:query-memory-limit N` or `$slot` (not in `DEFAULTS`); code: `PreparedQuery::with_query_memory_limit` | Replaces the database value for that query, up or down; a keyword overrides the setter |
| `max_iterations` | 1,000,000 rounds per stratum | query text: `:max-iterations N` or `$slot`; code: `PreparedQuery::with_max_iterations` | Replaces the database value for that query, up or down; a keyword overrides the setter |
| `transaction_chunk_facts` | 65,536 | — (per-transaction `TxOptions` planned, #871) | — |
| `synchronous` | `SyncMode::Full` | — (per-transaction `TxOptions` planned, #871) | — |
| `transaction_chunk_facts` | 65,536 | code: `TxOptions::chunk_facts` with `Minigraf::begin_write_with` | Replaces the database value for that `WriteTransaction` only, up or down, with the same clamp (1..=1,000,000); later transactions and `execute` use the database value |
| `synchronous` | `SyncMode::Full` | code: `TxOptions::synchronous` with `Minigraf::begin_write_with` | Decides whether that `WriteTransaction`'s WAL entry is synced, either way; later transactions and `execute` use the database value. A `Full` commit also makes earlier `Normal` ones durable. No effect on a large transaction (always durable) or an in-memory database. See [DURABILITY.md](DURABILITY.md) |
| `wal_checkpoint_threshold` | 1000 | open only | Database-wide checkpoint policy |
| `page_cache_size` | 256 pages | open only | Shared cache |
| `allow_unlocked` | `false` | open only | Decided when the file is opened |
Expand Down
11 changes: 11 additions & 0 deletions docs/DURABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,17 @@ A crash during a checkpoint is no different: the file keeps the previous checkpo

Batching is the other lever: `begin_write()`, several `execute()` calls and one `commit()` write one WAL entry with one sync, in either mode.

### Per-transaction mode

`Minigraf::begin_write_with(TxOptions::new().synchronous(mode))` chooses the mode of one `WriteTransaction`'s commit, whatever the database's mode (#871); `begin_write()` and `execute()` use the database's. The mode is not kept: the next transaction uses the database's again. So one WAL can hold entries committed both ways:

- A **process crash** keeps every committed transaction, in either mode.
- A **power loss** keeps a prefix of the commit order, never a transaction in part and never a later one without an earlier one. The prefix includes every transaction up to the last `Full` commit, the last checkpoint, or whichever is later: a `Full` commit syncs the whole WAL file, so the `Normal` entries before it become durable with it. Entries committed with `Normal` after the last `Full` commit may be lost.
- A `Normal` commit never weakens a transaction committed before it.
- A **large transaction** (more facts than `transaction_chunk_facts`) commits as a checkpoint, with no WAL entry, so it is durable in both modes; the header of the WAL it creates before its first page write is always synced. The per-transaction mode has no effect on it.

For example, a database opened `Full` can run a bulk load as `Normal` transactions and end it with one `Full` transaction (or a `checkpoint()`), which makes the whole load durable at once.

**What durability assumes.** A sync is only as good as the storage under it. Minigraf assumes that when `fdatasync`/`fsync` (`FlushFileBuffers` on Windows) returns, the data is on stable storage. Disks or controllers with a volatile write cache that ignore flushes, virtual machines with an "unsafe" disk cache mode, and some network or FUSE filesystems break that assumption, and a power loss there can lose committed transactions or tear a page. A torn or rotted page is detected by its checksum on the next read (`STG-029`); it is not silently read.

**Browser (`BrowserDb`).** There is no WAL. Each `transact` and `retract` is a checkpoint whose pages, meta page and freed page keys are written in one IndexedDB `readwrite` transaction, and the returned promise resolves after that transaction completes. Closing the tab before then loses that write, whole. How durable a completed IndexedDB transaction is on a power loss is up to the browser.
Expand Down
15 changes: 12 additions & 3 deletions docs/TEST_COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

| | Default features | With `--features test-hooks` |
|---|---|---|
| **Active** | 2,087 | 2,125 |
| **Active** | 2,111 | 2,149 |
| **Ignored** | 21 | 30 |
| **Total** | 2,108 | 2,155 |
| **Total** | 2,132 | 2,179 |

Counts above are the current Cargo test inventory (active versus ignored),
including #777, #789, #781, #794, #795, #837, #865, #869, #870, #861 and #863 (Linux). Verification results below identify which suites ran.
including #777, #789, #781, #794, #795, #837, #865, #869, #870, #861, #863 and #871 (Linux). Verification results below identify which suites ran.

**#863 nightly candidates**: four engine unit tests check the `test-hooks`
path events and memory peaks (both builds). With `test-hooks` only,
Expand Down Expand Up @@ -219,6 +219,7 @@ These tests compare the database against an independent answer rather than a sin
- Kills a child process running a seeded workload (multi-valued batches, batched retracts, five checkpoint modes) at a random point.
- The reopened file must hold exactly the transactions the child logged as committed, or those plus the one in flight.
- Checked through full-scan, AEVT, AVET, EAVT and `:as-of` queries and `verify`; again after a second reopen; and after a checkpoint plus reopen.
- A `PerTxSync` variant (#871, every build) commits every transaction through `begin_write_with` with a sync mode chosen per transaction from the seed (the first `Normal`, the second `Full`) on a database opened `Full`.
- With `test-hooks`, a `Large` variant makes most transactions large, runs every third one as a `WriteTransaction` of three statements, and rolls back a large statement after writing pages (#550).
- **Golden-file corpus** (`tests/golden_corpus_test.rs`, #391 *(v2.x)*)
- `tests/golden/` holds v7 files written by minigraf 2.0.3: one checkpoint, several checkpoints, indexes rebuilt on open, a crash-left WAL, an already-checkpointed WAL, same-transaction multi-values.
Expand Down Expand Up @@ -396,6 +397,14 @@ These are not `#[test]`s; run them with `cargo kani` or `.github/workflows/kani.
- An append that fails part-way makes every later append `WAL-007`. A rejected oversize fact (`WAL-003`, nothing written) does not.
- Through `Minigraf`: the failed transaction is not applied, later `transact` and `commit` are `WAL-007`, and a `checkpoint()` makes writes work again.
- In `tests/wal_test.rs`: a commit made after reopening a crash-left WAL with each kind of torn tail survives a second crash.
- **Per-transaction options** (full default and `test-hooks` suites passed on the alternate bench machine with Rust 1.89, 2026-11-10; `src/tx_options_tests.rs`, 17 tests; `src/wal.rs`, 2 tests; `src/fault_matrix.rs`, 3 tests; 2 doctests; `src/db.rs`, 1 test rewritten; #871)
- `TxOptions` builders and fields; `begin_write_with` fails with `INT-001` and `API-014` as `begin_write` does and leaves the handle usable.
- A small commit syncs the WAL exactly when its effective mode is `Full`, for the database modes × `None`/`Full`/`Normal`, whether the commit creates the WAL or appends to it. The mode never stays with the WAL writer: implicit transactions and `begin_write()` follow the database mode after a commit, a rollback or a drop, including after a `Normal` transaction that became large (creating the WAL guard) and was rolled back. The stored-mode test of #302 now checks that a `Normal` database's commits leave the WAL unsynced at both writer call sites.
- Test models: a lost sync loses every WAL byte since the last good sync, including whole `Normal` entries; `Minigraf::power_loss` ends a handle with the WAL cut back to its last sync.
- On `Full` and `Normal` databases: `Normal` then kill keeps A; `Normal` then power loss keeps a prefix (A lost), with a later commit surviving a kill; `Normal`, `Full`, power loss keeps both; `Full`, `Normal`, power loss keeps A. Every reopen is compared record by record through `fact_log`.
- A large `Normal` transaction survives a power loss whole with `verify` clean; a `Normal` commit at the checkpoint threshold still checkpoints; options on an in-memory database change nothing.
- `chunk_facts`: `Some(2)` makes three facts large (one statement or staged ones), `Some(1000)` keeps them small on a database with chunk 2, `0` acts as 1 and `usize::MAX` as 1,000,000; it applies to that transaction only; the thread-local test hook still wins. The facts committed by the same statements are the same with any options.
- Fault matrix: every fault at every write and sync of a `Full` commit after a `Normal` one, a `Normal` commit on a `Full` database (it makes no sync, so the sync faults never fire there), and a `Full` commit on a `Normal` database. The reopened file must be a prefix of the commit order from the last sync or checkpoint, never part of a transaction; only a lost sync may lose a `Normal` entry; `verify` is clean and a commit after recovery survives a kill.
- **WAL entries of any size** (`src/wal.rs`, 3 tests, one replacing the 1,000,000-fact cap test; #686)
- An entry of 1,000,001 facts and the entries after it replay, and reopening the writer cuts none of them off.
- A fact count the bytes after it cannot hold (one more than a quarter of them, or `u64::MAX`) ends replay at that entry, and reopening the writer cuts the WAL off there.
Expand Down
Loading
Loading