Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Performance

- **Resolve expression keyword literals once per operator** (#774). `=` and `!=` reuse a literal keyword's entity UUID across input rows, including nested expressions, rule bodies and prepared slots. The cache contains only literals from that expression and is discarded with the operator; dynamic operands retain the existing comparison semantics. No API, dependency or file-format change. See the [measurements](docs/benchmarks/774-expression-keywords.md).

- **Avoid whole-attribute reads for selective in-memory ancestors** (#723). Eligible three-pattern recursive delta runs can start from a small derived relation and probe stored facts by entity, preserving the same semi-naive ranges and answers. The choice is reconsidered each round using bounded pending-index counts. Populated committed generations retain AVET value probes; general value joins retain their existing behavior. No persistent index, dependency, API or format change. The approved v3 binary-size budget increases from 1,230 to 1,234 KiB for this change.

- **Skip sorting proven ordered input** (#641, first slice). Plain queries can omit `Sort` when a committed integer, float, or Boolean range already provides the complete ascending result order. Attribute scans use AVET; entity-and-attribute scans retain their selective EAVT range. `:offset` and `:limit` then stop reading early, so memory-limit (`QRY-012`) or I/O errors in unread later input may be avoided. The proof uses the query's pinned reader, including a large write transaction's working trees, and rejects pending or staged records in the source range, mixed kinds, unsupported comparators, tie suffixes, order-changing operators, user callbacks and `collect-set`. Unproven queries retain their original plan.
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**2005 tests** (1988 passing, 17 ignored; unit + integration + doc). Engine answers once checked by the executor are frozen in `tests/answers/` (#551).
**2006 tests** (1989 passing, 17 ignored; unit + integration + doc). Engine answers once checked by the executor are frozen in `tests/answers/` (#551).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

The #760 deterministic storage suite adds three native unit tests in a separate `--cfg minigraf_shuttle` build, outside the default count. Run only `graph::storage::deterministic::` with that cfg; ordinary unit tests do not enter its scheduler. `deterministic-schedules.yml` runs relevant v3 PRs on Linux/macOS/Windows plus Linux MSRV, and larger nightly/manual searches with replay artifacts. The identical main workflow activates the v3 schedule after both PRs merge. See `docs/testing/deterministic-schedules.md` for scope, controls and findings.
Expand All @@ -227,6 +227,8 @@ The #760 deterministic storage suite adds three native unit tests in a separate

## Current Maintainer Context

On `v3`, expression `=` / `!=` resolves literal keywords once per operator (#774), including nested expressions and substituted prepared slots. Dynamic values do not populate the cache. This is a performance-only change, with no v2.x backport; see `docs/benchmarks/774-expression-keywords.md`.

**v2.0.5 released** (2026-10-11) — short-file open refused with STG-045 (#506), commit after a torn WAL entry no longer lost, WAL-007 (#513); backported rule-query `max_results` (#528), all-checkpointed WAL removal (#457) and `usize::MAX` close-time save (#476); weekly soak test (#392), real coverage gates (#396). **v2.0.4** (2026-10-08) — `BrowserDb` IndexedDB flush returns an error for an unreadable dirty page (#470); golden-file v7 corpus (#391) and SIGKILL committed-data crash test (#384). **v2.0.3** (2026-10-06) — data-integrity patch under the support policy: indexed queries with >65,535 uncheckpointed facts (#445), tx counter rewound on reopen next to an already-checkpointed WAL (#447). **v2.0.2** (2026-09-27) — last planned v2.x feature/fix release: query fixes (#297, #405, #407, #410), directory fsync (#389), point-query/attribute-scan/checkpoint perf mitigations (#323, #381, #315), CI policy checks (#395), support policy and tiers (#397, #399, #400). **v2.0.1** (2026-09-25) fixed index rebuild on open (#370). **v2.0.0** introduced kernel file locking (#317, #304), structured runtime error codes (#277), `OpenOptions` `#[non_exhaustive]`, and an MSRV of Rust 1.89. See `CHANGELOG.md` for the full rationale and release history.

Current plan (see `ROADMAP.md` and the milestones):
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ On the unreleased v3 branch, selective in-memory ancestor queries can use entity

**Status**: See [ROADMAP.md](ROADMAP.md) for planned work and current direction.

The unreleased v3 branch also avoids resolving the same keyword literal on every row of an expression comparison such as `[(= ?entity :alice)]` (#774); see the [benchmark report](docs/benchmarks/774-expression-keywords.md).

## Why Datalog?

**Datalog is fundamentally better for graphs than SQL-like languages:**
Expand Down
1 change: 1 addition & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ Scope — release-gate testing and documentation:
- Fault injection across save/WAL/recovery (#390)
- Long-running soak test at scale (#392; done on `v3`: `tests/soak_test.rs`, 10M facts and a 5-hour churn weekly, alternating `main` and `v3`)
- Benchmarks at 100K–10M facts and a published performance envelope (#394; on `v3`: `examples/perf_envelope.rs`, results in `docs/BENCHMARKS.md`; follow-ups #528–#532, with #532 superseded by #551's removal of repeated committed-fact net-assert; [audit and measurements](docs/benchmarks/532-net-assert-audit.md))
- Expression keyword/ref equality (#774; on `v3`): resolve literal keywords once per operator, with memory bounded by the expression. [Measurements and regression coverage](docs/benchmarks/774-expression-keywords.md); performance-only, with no v2.x backport.
- Higher storage/WAL coverage gates and branch-coverage gates (#396; on `v3`: line and branch floors per file in `coverage-thresholds.toml`, checked on every PR)
- Durability, recovery and operations guide (#398; done on `v3`: `docs/DURABILITY.md`, also on the docs site; found #535)

Expand Down
11 changes: 10 additions & 1 deletion docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

This document records reproducible local benchmark snapshots. Completed release-performance history belongs in the [CHANGELOG](../CHANGELOG.md); continuous CI history is available on [Bencher](https://bencher.dev/perf/minigraf/plots).

## Literal keyword/ref expression equality (#774, unreleased)

On 2026-10-29, caching expression literal resolutions reduced the measured
`=` / `!=` keyword/ref queries by **9.0–11.0%** at 100K and 1M facts, in both
operand orders. Four paired runs per shape swapped sockets between baseline
`5f8073250` and the change. The [report and raw results](benchmarks/774-expression-keywords.md)
include the initial measurements, UUID-keyword controls, point/scan/join guards,
a focused repeat of noisy point timings, and the scope of the improvement.

## Selective in-memory ancestors (#723, unreleased)

**Date**: 2026-10-29 · **Baseline**: `v3` at `756ffea20`. Both revisions
Expand Down Expand Up @@ -727,7 +736,7 @@ On the executor, before #544 (the executor's query path is gone since #551). `pe
- **DICT lookup per row (26–35%).** `uuid_of` does a B+tree `get` for every row's entity id (#530).
- **Bindings as `HashMap<String, Value>` (about 25%).** SipHash insert, lookup and drop per row; #473's columnar batches replace them.
- **`net_asserted_facts` over committed facts that `live_facts` already resolved (about 10%)** (#532). Superseded by #551's removal of the executor's query path; see the [source audit and fresh envelope/profiles](benchmarks/532-net-assert-audit.md). Pending records still need fact-level net-assert.
- **Keyword constants hashed to a UUID per fact (about 8% of the old join)** (#531). The streaming engine resolves pattern constants while planning; see the [post-#551 audit and profiles](benchmarks/531-keyword-constants.md). Expression keyword/ref equality retains a separate per-row conversion described there.
- **Keyword constants hashed to a UUID per fact (about 8% of the old join)** (#531). The streaming engine resolves pattern constants while planning; see the [post-#551 audit and profiles](benchmarks/531-keyword-constants.md). The separate expression-literal path is measured and optimized in [#774](benchmarks/774-expression-keywords.md); dynamic expression values can still require per-row conversion.
- `attribute_value` reads the whole attribute instead of seeking AVET (#518). Fixed by the streaming engine's `Scan` (#544): see below.

### Streaming engine (#544)
Expand Down
13 changes: 11 additions & 2 deletions docs/TEST_COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,16 @@

| | Default features | With `--features test-hooks` |
|---|---|---|
| **Passing** | 1,988 | 2,021 |
| **Passing** | 1,989 | 2,022 |
| **Ignored** | 17 | 18 |
| **Total** | 2,005 | 2,039 |
| **Total** | 2,006 | 2,040 |

#774 adds one default-feature unit test (also present with `test-hooks`). On
2026-10-29, after rebasing onto #723, its full default suite passed 1,989 tests
with 17 ignored; the targeted `test-hooks` join-equality and plan-independence
suites passed 31 tests. The
full `test-hooks` total above includes the added test; that full suite was not
rerun for #774. Formatting and strict all-feature clippy passed.

**#785 patch-coverage verification (2026-10-29)**: the bench machine passed the complete CI branch-coverage run (1,973 passing, 13 ignored; this command excludes doctests) and every unchanged module floor. Overall line coverage is 95.54%. The new boundary cases cover five lines previously reported as missing or partial. CI-equivalent all-features clippy and formatting pass.

Expand Down Expand Up @@ -36,6 +43,8 @@ Also outside those counts: three native unit tests in the separate #760 `minigra

## Covered Areas

- **Expression keyword literals** (#774): a test-only conversion counter verifies that increasing input from 8 to 2,048 rows does not increase literal resolution work, across `=` / `!=`, both operand orders, nested expressions, bindings, negation, disjunction and rule bodies. Existing join-equality tests also cover changing prepared keyword slots across executions and unchanged keyword output, in memory, after checkpoint and after reopen. The envelope harness checks all eight new expression/control shapes on regular and irregular fixtures.

- **In-memory ancestor join ordering** (#723; four tests): a 1,999-edge tree returns three ancestors within a 128 KiB query budget, also after staged edge replacement; cursor close releases its metered state. Reordered and text-order plans return identical bags across keyword/ref edges, cycles, multi-valued edges, retractions, transaction/valid-time filters, empty and unbound queries, in memory and with checkpointed facts. A direct comparison against the original delta plan protects keyword/ref output representation in rule heads. The bounded cost count excludes writes after a captured snapshot, includes staged records, respects the cap, and excludes prefix-neighbor attributes.

- **Proven input order** (#641, first slice; `src/query/engine/order_tests.rs`, `order/tests.rs`, `tests/input_order_test.rs`): full result keys and projection slots, constants and repeated slots, integer/Boolean/float edge cases including NaNs and signed zeros, replacements, join fallback orientation and distinct canonicalization. Snapshot tests cover unrelated versus in-range pending/staged records, captured prefixes, temporal filters, prepared queries, checkpoints and large-transaction working readers including rejected statements and rollback. Counters establish early stopping, bounded raw proof reads and no reads without demand; optional proof-read errors leave the original plan. Late callback and `collect-set` errors retain their evaluation. A forced-fallback differential matrix checks identical ordered lists across plan switches. A committed `MemoryBackend` fixture runs on native, WASI and the browser. Storage tests cover every key kind/tag and raw seeks over retracted history.
Expand Down
3 changes: 2 additions & 1 deletion docs/benchmarks/531-keyword-constants.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,8 @@ binding `HashMap` per row, so the UUID computation is only one potential cost.
A follow-up should measure this shape before choosing a milestone or cache
design. Preserve equality/inequality semantics, both operand orders, dynamic
operands and prepared slots. This is performance-only; under the support policy
it does not warrant a v2.x backport. This draft has not been posted as an issue.
it does not warrant a v2.x backport. This follow-up became #774; see its
[subsequent measurements and literal-cache change](774-expression-keywords.md).

## Deviations from the approved design

Expand Down
Loading
Loading