Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 21 additions & 30 deletions .github/workflows/coverage-gates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,41 +17,32 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
# Branch coverage needs nightly. Pinned so a toolchain update cannot move
# the numbers under an unrelated PR; bump it on purpose and re-measure.
- name: Install Rust nightly
uses: dtolnay/rust-toolchain@master
with:
toolchain: nightly-2026-10-01
components: llvm-tools-preview

- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov

- name: Generate coverage (text)
run: cargo llvm-cov --text 2>&1 > cov_text.txt || true
- name: Build the pid_ns_helper example used by lock tests
# cargo-llvm-cov puts test binaries where the lock tests' lookup of
# examples/ next to the test binary fails; same workaround as
# llvm-cov.yml.
run: cargo build --examples --target-dir /tmp/minigraf-lock-test-examples

# A failing test fails the job: a partial report must never be gated.
- name: Run the tests with branch coverage
env:
MINIGRAF_PID_NS_HELPER: /tmp/minigraf-lock-test-examples/debug/examples/pid_ns_helper
run: cargo llvm-cov --branch --no-report

- name: Summarize coverage
run: cargo llvm-cov report --branch --json --summary-only --output-path coverage.json

# Thresholds come from coverage-thresholds.toml (#396).
- name: Check per-module thresholds
run: |
FAIL=0
check_module() {
local file=$1 threshold=$2
local pct
pct=$(grep "$file" cov_text.txt | grep -oP '\d+\.\d+(?=%)' | tail -1)
if [ -z "$pct" ]; then
echo "WARNING: $file not found in coverage report"
return 0
fi
if python3 -c "import sys; sys.exit(0 if float('$pct') >= $threshold else 1)"; then
echo "OK: $file: ${pct}% >= ${threshold}%"
else
echo "FAIL: $file: ${pct}% < ${threshold}%"
return 1
fi
}
# Thresholds from coverage-thresholds.toml
check_module "wal.rs" 75 || FAIL=1
check_module "db.rs" 70 || FAIL=1
check_module "persistent_facts" 70 || FAIL=1
check_module "btree_v6" 65 || FAIL=1
check_module "executor.rs" 75 || FAIL=1
check_module "evaluator.rs" 75 || FAIL=1
check_module "stratification" 75 || FAIL=1
exit $FAIL
run: python3 scripts/coverage_gates.py coverage-thresholds.toml coverage.json
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Fuzzing reaches the v8 decoders, and the corpus is kept between runs (#387, #375).** On `v3`, the `btree_page` and `fact_page` targets still wrote the pre-release single-header layout and stopped at `STG-032` on every input, and `file_header` rarely got past the meta CRC. The page targets now start from a real v8 file built through the public API (rewritten with `LogWriter` at fixed tx times so its bytes are the same in every process). They overwrite one page with the input and recompute that page's checksum. `btree_page` fuzzes any leaf or internal node of the five trees, `file_header` fuzzes either meta slot (with or without a matching CRC, with or without the other slot), and `fact_page` fuzzes the fact page of a v7 golden file through migration and the read-only load. Each run executes one input-chosen probe query or `verify()`. The nightly fuzz workflow now runs every target in its own parallel job, on `main` and `v3` on alternating crons, and keeps each target's corpus in the Actions cache. A new weekly workflow (`fuzz-corpus.yml`) minimizes the cached corpus with `cargo fuzz cmin`, keeps every committed `seed_*` file, and proposes the result as a pull request.
- **Two fuzz targets check results, not just crashes (#387).** `ops_sequence` decodes the input into up to 32 storage operations (transact, retract, a write transaction that commits or rolls back, checkpoint, reopen, and a crash that opens a copy of the file and WAL) and, after each one, compares the full rows now and at an input-chosen `:as-of`, plus one input-chosen query shape, with the reference model of the model-based test. It uses that test's driver and checks, now in `tests/reference_model`, so the proptest test and the fuzzer share one oracle. `corrupt_file` damages a file written over three checkpoints with up to 16 byte flips and an optional truncation, without fixing any checksum, and requires every query to fail or to return exactly the answers of the last checkpoint or the one before it, all from the same one. Its first run found that a file cut to under one page opens as a new, empty database (#506); the target skips those lengths until that is fixed.
- **Weekly soak test at scale (#392).** `tests/soak_test.rs` grows a file-backed database to 10M live facts in batches of 1–2,000 facts (multi-valued tags and shard values in one transaction, strings over 64 bytes, refs), then churns it for the rest of a 5-hour budget at 50 transactions a second: retracts and re-asserts of random facts, and long retract/assert histories on 32 hot entities. It checkpoints every 20–200 transactions and reopens every 2–10 minutes, half from a checkpointed file and half replaying the WAL. Every 10 minutes, after every reopen, and between churn writes it checks samples against a reference that derives each entity's facts from its number and keeps only what was retracted: EAVT per entity, entity and attribute on the tags, AVET by unique name, AEVT over a sparse attribute, `:as-of` on hot entities, and the transaction counter; a full scan runs in a child process at 2M facts (so its memory does not hide a leak), and `verify()` at the end. It fails if RSS after a reopen grows past 1.25× the baseline plus 256 MiB. Metrics (file size, open and checkpoint time, RSS and peak RSS, p50/p99 per query path) go to a JSON-lines file that `.github/workflows/soak.yml` uploads weekly, alternating `main` and `v3`. `soak_short` runs the same driver at 20K facts in the per-PR suite. Found #518: attribute-and-value patterns scan the whole attribute.
- **Coverage gates that gate (#396).** The per-module gate in `coverage-gates.yml` had never checked anything: the instrumented test run failed (the lock tests could not find `pid_ns_helper`), the report it grepped was empty, and every module was skipped with a warning, so the job passed. It also read `cargo llvm-cov --text`, which has no per-file percentages, and named `btree_v6`, which v3 no longer has. The job now runs the whole suite with branch coverage on a pinned nightly, fails if any test fails, and checks each file's line and branch percentage against `coverage-thresholds.toml` with `scripts/coverage_gates.py`; a file missing from the report, or without branch data, fails. The thresholds file covers the WAL, `db.rs`, the fact log and log writer, every storage file, and the evaluator, executor and stratification, with floors at the measured values less one point. It also lists the paths left uncovered on purpose (invariant checks, browser-only and Windows-only paths). New tests cover the WAL's fact-size and fact-count limits (the old fact-size test changed bytes inside the fact and never reached the limit), a WAL opened in a missing directory, a failed cut of a torn WAL tail, short v7 pages and slots past the count, backup meta pages of another generation or length, a torn first meta with data in page 1, `save` with nothing pending, `into_backend` with pending facts, `rebuild_indexes` with nothing checkpointed, unsorted or repeated B+tree input, a stored key inserted with another value, a tree pointer to a page of another type, the read-only `MutexStorageBackend`, and two threads missing the same page in the page cache (before, only a lucky race reached that path). Branch coverage of `wal.rs` went from 83% to 95%, `packed_pages.rs` from 86% to 100%, `persistent_facts.rs` from 82% to 88%, and `btree.rs` from 85% to 85.5%.

### Known issues

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**1409 tests** (1399 passing, 10 ignored; unit + integration + doc).
**1423 tests** (1413 passing, 10 ignored; unit + integration + doc).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

**Testing conventions** — see the Testing Conventions section below before writing any tests.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit

```bash
cargo run # interactive Datalog REPL
cargo test # run 1409 tests
cargo test # run 1423 tests
cargo run < demos/demo_recursive.txt # recursive rules demo
```

Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Scope — release-gate testing and documentation:
- Fault injection across save/WAL/recovery (#390)
- Long-running soak test at scale (#392; done on `v3`: `tests/soak_test.rs`, 10M facts and a 5-hour churn weekly, alternating `main` and `v3`)
- Benchmarks at 100K–10M facts and a published performance envelope (#394)
- Higher storage/WAL coverage gates and branch-coverage gates (#396)
- Higher storage/WAL coverage gates and branch-coverage gates (#396; on `v3`: line and branch floors per file in `coverage-thresholds.toml`, checked on every PR)
- Durability, recovery and operations guide (#398)

---
Expand Down
124 changes: 124 additions & 0 deletions coverage-thresholds.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# Per-file coverage floors, checked on every push and PR by
# .github/workflows/coverage-gates.yml (scripts/coverage_gates.py). The single
# source of truth for the gates (#396).
#
# Percentages of lines and branches covered by `cargo llvm-cov --branch` with
# the nightly pinned in that workflow, test modules included. Each floor is
# the measured value rounded down, minus one point of slack (none at 100%).
# A floor only goes up: raise it in the PR that adds the tests, and lower one
# only with the reason in the PR. Target for the storage and WAL files: 90%
# branches.
#
# Known uncovered paths, left on purpose:
# - INT-049 bails in btree.rs and persistent_facts.rs (a leaf or node that
# cannot fit, an empty level, a tree deeper than MAX_TREE_DEPTH, a rebuilt
# index failing its own check): invariant checks that no input reaches
# without a bug elsewhere.
# - persistent_facts.rs `check_fits` / `check_written` with
# `holds_every_page() == false`: only the browser buffer, covered by the
# wasm tests in wasm.yml, which this job does not measure.
# - wal.rs `delete_file` retry sleep: Windows only (one attempt elsewhere).
# - wal.rs `scan_entries` read error other than EOF: needs a failing read on a
# regular file, which the fault plan (storage/fault.rs) does not inject.

# WAL and the public API

[files."src/wal.rs"]
lines = 97
branches = 94

[files."src/db.rs"]
lines = 95
branches = 87

[files."src/fact_log.rs"]
lines = 91
branches = 97

[files."src/log_writer.rs"]
lines = 97
branches = 100

[files."src/graph/storage.rs"]
lines = 97
branches = 91

# Storage engine

[files."src/storage/backend/file.rs"]
lines = 88
branches = 74

[files."src/storage/btree.rs"]
lines = 94
branches = 84

[files."src/storage/cache.rs"]
lines = 86 # the test-only BarrierBackend's unused trait methods count
branches = 75

[files."src/storage/dict.rs"]
lines = 94
branches = 89

[files."src/storage/freelist.rs"]
lines = 95
branches = 59

[files."src/storage/index.rs"]
lines = 100
branches = 74

[files."src/storage/keys.rs"]
lines = 94
branches = 80

[files."src/storage/meta.rs"]
lines = 96
branches = 74

[files."src/storage/mod.rs"]
lines = 84
branches = 92

[files."src/storage/node.rs"]
lines = 90
branches = 79

[files."src/storage/packed_pages.rs"]
lines = 97
branches = 100

[files."src/storage/page.rs"]
lines = 90
branches = 83

[files."src/storage/persistent_facts.rs"]
lines = 95
branches = 86

[files."src/storage/reader.rs"]
lines = 97
branches = 94

[files."src/storage/value_pages.rs"]
lines = 85
branches = 80

[files."src/storage/verify.rs"]
lines = 79
branches = 70

# Query engine

[files."src/query/datalog/evaluator.rs"]
lines = 88
branches = 84

[files."src/query/datalog/executor.rs"]
lines = 93
branches = 76

[files."src/query/datalog/stratification.rs"]
lines = 100
branches = 100
15 changes: 10 additions & 5 deletions docs/TEST_COVERAGE.md

Large diffs are not rendered by default.

61 changes: 61 additions & 0 deletions scripts/coverage_gates.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""Check per-file coverage floors from coverage-thresholds.toml (#396).

Usage: coverage_gates.py <thresholds.toml> <llvm-cov summary.json>

The JSON is `cargo llvm-cov report --branch --json --summary-only`. Every file
named in the thresholds must be in the report, with branch data when it has a
branch floor; a missing file or missing branch data fails the gate instead of
being skipped, so an empty or partial report can never pass.
"""

import json
import sys
import tomllib


def main() -> int:
thresholds_path, report_path = sys.argv[1], sys.argv[2]
with open(thresholds_path, "rb") as f:
floors = tomllib.load(f)["files"]
with open(report_path) as f:
report = json.load(f)

by_path = {}
for entry in report["data"][0]["files"]:
name = entry["filename"].replace("\\", "/")
for path in floors:
if name.endswith("/" + path):
by_path[path] = entry["summary"]

failures = 0
print(f"{'file':<40} {'metric':<9} {'covered':>8} {'floor':>6}")
for path, floor in floors.items():
summary = by_path.get(path)
if summary is None:
print(f"{path:<40} FAIL: not in the coverage report")
failures += 1
continue
for metric in ("lines", "branches"):
if metric not in floor:
continue
counts = summary.get(metric, {})
if not counts.get("count"):
print(f"{path:<40} {metric:<9} FAIL: no {metric} data")
failures += 1
continue
pct = counts["percent"]
ok = pct >= floor[metric]
failures += not ok
mark = "" if ok else " FAIL"
print(f"{path:<40} {metric:<9} {pct:>7.2f}% {floor[metric]:>5}%{mark}")

if failures:
print(f"\n{failures} coverage gate(s) failed. Add tests, or list a path "
"left uncovered, with its reason, in the thresholds file; lower a "
"floor only with the reason in the PR.")
return 1 if failures else 0


if __name__ == "__main__":
sys.exit(main())
98 changes: 98 additions & 0 deletions src/storage/btree.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1486,4 +1486,102 @@ mod tests {
assert_eq!(alloc.next_append() - next, 3, "leaf + 2 internal nodes");
assert_eq!(freed.len(), 3);
}

fn code(e: anyhow::Error) -> &'static str {
crate::error::MinigrafError::from(e).code()
}

#[test]
fn unsorted_or_repeated_input_is_int_049() {
let cache = PageCache::new(16);
for bad in [
vec![entry(2, 1), entry(1, 1)],
vec![entry(1, 1), entry(1, 1)],
] {
let mut backend = MemoryBackend::new();
let mut alloc = PageAllocator::new(Vec::new(), 2, 1);
let err = build_btree(bad.clone(), &mut backend, &cache, &mut alloc).unwrap_err();
assert_eq!(code(err), "INT-049");
let (root, next) = build_at(vec![entry(5, 1)], &mut backend, &cache, 2);
let mut alloc = PageAllocator::new(Vec::new(), next, 2);
let mut freed = Vec::new();
let err =
cow_insert(root, bad, &mut backend, &cache, &mut alloc, &mut freed).unwrap_err();
assert_eq!(code(err), "INT-049");
}
}

#[test]
fn insert_of_a_stored_key_with_another_value_is_int_049() {
let cache = PageCache::new(16);
let mut backend = MemoryBackend::new();
let (root, next) = build_at(vec![entry(1, 1)], &mut backend, &cache, 2);
let mut alloc = PageAllocator::new(Vec::new(), next, 2);
let mut freed = Vec::new();
let same = cow_insert(
root,
vec![entry(1, 1)],
&mut backend,
&cache,
&mut alloc,
&mut freed,
)
.unwrap();
assert_eq!(stream_all_entries(same, &backend, &cache).unwrap().len(), 1);
let other = vec![(key(1, 1), vec![1u8])];
let err =
cow_insert(root, other, &mut backend, &cache, &mut alloc, &mut freed).unwrap_err();
assert_eq!(code(err), "INT-049");
}

/// A tree pointer to a page of another type (here a free-list page) is
/// STG-013 on every path that follows it.
#[test]
fn node_of_another_page_type_is_stg_013() {
let cache = PageCache::new(16);
let mut backend = MemoryBackend::new();
let id = 2;
let mut page = crate::storage::page::new_page(crate::storage::page::PAGE_TYPE_FREELIST, 0);
crate::storage::page::seal(&mut page, id, 1).unwrap();
backend.write_page(id, &page).unwrap();

assert_eq!(
code(get(id, &key(1, 1), &backend, &cache).unwrap_err()),
"STG-013"
);
assert_eq!(
code(collect_leaf_pages(id, &backend, &cache, None).unwrap_err()),
"STG-013"
);
let mut alloc = PageAllocator::new(Vec::new(), 3, 2);
let mut freed = Vec::new();
let err = cow_insert(
id,
vec![entry(1, 1)],
&mut backend,
&cache,
&mut alloc,
&mut freed,
)
.unwrap_err();
assert_eq!(code(err), "STG-013");
}

#[test]
fn mutex_backend_reads_and_refuses_writes() {
let mut inner = MemoryBackend::new();
inner.write_page(0, &[7u8; PAGE_SIZE]).unwrap();
let mut b = MutexStorageBackend(Arc::new(Mutex::new(inner)));
assert_eq!(b.read_page(0).unwrap()[0], 7);
assert_eq!(b.page_count().unwrap(), 1);
assert!(b.holds_every_page());
assert!(b.partial_page().unwrap().is_none());
assert_eq!(b.backend_name(), "mutex-adapter");
assert_eq!(
code(b.write_page(0, &[0u8; PAGE_SIZE]).unwrap_err()),
"INT-049"
);
assert_eq!(code(b.sync().unwrap_err()), "INT-049");
assert_eq!(code(b.close().unwrap_err()), "INT-049");
}
}
Loading
Loading