Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/envelope.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Performance envelope

# #394: builds 100K and 1M fact fixtures and measures open and checkpoint time,
# RSS, and p50/p99 latency per query shape (examples/perf_envelope.rs). The
# published envelope in docs/BENCHMARKS.md also has 10M facts, measured
# locally. Runs on demand; on pull requests that touch the harness it runs a
# 10K smoke pass so the harness keeps working.

on:
workflow_dispatch:
inputs:
ref:
description: Branch to measure
required: true
default: v3
scales:
description: Comma-separated fact counts
required: true
default: "100k,1m"
pull_request:
paths:
- examples/perf_envelope.rs
- .github/workflows/envelope.yml

permissions:
contents: read

jobs:
envelope:
name: Envelope (${{ github.event_name == 'pull_request' && '10k smoke' || inputs.scales }})
runs-on: ubuntu-latest
timeout-minutes: 120

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.ref || '' }}

- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable

- name: Build
run: cargo build --profile envelope --example perf_envelope

- name: Run
shell: bash
env:
SCALES: ${{ github.event_name == 'pull_request' && '10k' || inputs.scales }}
BUDGET: ${{ github.event_name == 'pull_request' && '1' || '15' }}
run: |
set -o pipefail
./target/envelope/examples/perf_envelope --scales "$SCALES" \
--budget-secs "$BUDGET" --json envelope.json | tee envelope.md
cat envelope.md >> "$GITHUB_STEP_SUMMARY"

- name: Upload results
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: envelope-${{ github.run_id }}
path: |
envelope.json
envelope.md
retention-days: 90
if-no-files-found: warn
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Two fuzz targets check results, not just crashes (#387).** `ops_sequence` decodes the input into up to 32 storage operations (transact, retract, a write transaction that commits or rolls back, checkpoint, reopen, and a crash that opens a copy of the file and WAL) and, after each one, compares the full rows now and at an input-chosen `:as-of`, plus one input-chosen query shape, with the reference model of the model-based test. It uses that test's driver and checks, now in `tests/reference_model`, so the proptest test and the fuzzer share one oracle. `corrupt_file` damages a file written over three checkpoints with up to 16 byte flips and an optional truncation, without fixing any checksum, and requires every query to fail or to return exactly the answers of the last checkpoint or the one before it, all from the same one. Its first run found that a file cut to under one page opens as a new, empty database (#506); the target skips those lengths until that is fixed.
- **Weekly soak test at scale (#392).** `tests/soak_test.rs` grows a file-backed database to 10M live facts in batches of 1–2,000 facts (multi-valued tags and shard values in one transaction, strings over 64 bytes, refs), then churns it for the rest of a 5-hour budget at 50 transactions a second: retracts and re-asserts of random facts, and long retract/assert histories on 32 hot entities. It checkpoints every 20–200 transactions and reopens every 2–10 minutes, half from a checkpointed file and half replaying the WAL. Every 10 minutes, after every reopen, and between churn writes it checks samples against a reference that derives each entity's facts from its number and keeps only what was retracted: EAVT per entity, entity and attribute on the tags, AVET by unique name, AEVT over a sparse attribute, `:as-of` on hot entities, and the transaction counter; a full scan runs in a child process at 2M facts (so its memory does not hide a leak), and `verify()` at the end. It fails if RSS after a reopen grows past 1.25× the baseline plus 256 MiB. Metrics (file size, open and checkpoint time, RSS and peak RSS, p50/p99 per query path) go to a JSON-lines file that `.github/workflows/soak.yml` uploads weekly, alternating `main` and `v3`. `soak_short` runs the same driver at 20K facts in the per-PR suite. Found #518: attribute-and-value patterns scan the whole attribute.
- **Coverage gates that gate (#396).** The per-module gate in `coverage-gates.yml` had never checked anything: the instrumented test run failed (the lock tests could not find `pid_ns_helper`), the report it grepped was empty, and every module was skipped with a warning, so the job passed. It also read `cargo llvm-cov --text`, which has no per-file percentages, and named `btree_v6`, which v3 no longer has. The job now runs the whole suite with branch coverage on a pinned nightly, fails if any test fails, and checks each file's line and branch percentage against `coverage-thresholds.toml` with `scripts/coverage_gates.py`; a file missing from the report, or without branch data, fails. The thresholds file covers the WAL, `db.rs`, the fact log and log writer, every storage file, and the evaluator, executor and stratification, with floors at the measured values less one point. It also lists the paths left uncovered on purpose (invariant checks, browser-only and Windows-only paths). New tests cover the WAL's fact-size and fact-count limits (the old fact-size test changed bytes inside the fact and never reached the limit), a WAL opened in a missing directory, a failed cut of a torn WAL tail, short v7 pages and slots past the count, backup meta pages of another generation or length, a torn first meta with data in page 1, `save` with nothing pending, `into_backend` with pending facts, `rebuild_indexes` with nothing checkpointed, unsorted or repeated B+tree input, a stored key inserted with another value, a tree pointer to a page of another type, the read-only `MutexStorageBackend`, and two threads missing the same page in the page cache (before, only a lucky race reached that path). Branch coverage of `wal.rs` went from 83% to 95%, `packed_pages.rs` from 86% to 100%, `persistent_facts.rs` from 82% to 88%, and `btree.rs` from 85% to 85.5%.
- **Performance envelope at 100K–10M facts (#394).** `examples/perf_envelope.rs` builds and caches fixtures of 100K, 1M and 10M facts (six attributes per entity, a ref tree, two valid-time windows, an entity with 1,000 versions). It reports open and checkpoint times, RSS, and p50/p99 latency for twelve query shapes, each shape in its own process. It builds with a new `envelope` Cargo profile (`release` at opt-level 3, as an application compiles the library; `release` itself is opt-level "z"). `docs/BENCHMARKS.md` and the README publish the results: point lookups and single-entity time travel stay flat (12–14 µs at 10M facts). Scans, aggregations and joins grow linearly with the attributes they touch. Rule queries are supported to 100K facts only. A Python FFI round trip (`bench/envelope.py` in minigraf-python) adds about 70 µs per query. `.github/workflows/envelope.yml` runs 100K and 1M on demand, and a 10K smoke pass on PRs that change the harness. The nightly Criterion `query/*` groups gain 100K and 1M scales. Profiling at 1M facts found #528, #529, #530, #531 and #532.

### Known issues

Expand Down
3 changes: 3 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ cargo test --test concurrency_test
cargo run --example embedded
cargo run --example file_storage

# Performance envelope (#394): latency/RSS at 100K–10M facts; opt-level 3 profile
cargo run --profile envelope --example perf_envelope -- --scales 100k,1m

# Run demo scripts
cargo run < demos/demo_commands.txt
cargo run < demos/demo_recursive.txt
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ Releases are managed by the project maintainer. The process is documented in iss
5. Tag pushed — CI publishes to crates.io, PyPI, npm, and Maven Central automatically
6. Tier 1 bindings (Python) released alongside the core release; Tier 2 bindings follow on a best-effort schedule (see [support tiers](PHILOSOPHY.md#support-tiers))
7. Known-issues issue updated, and `data-integrity`/`corruption`/`durability` issues fixed by this release closed with a link to it (see above)
8. For a minor or major release: the performance envelope re-run and the snapshot in [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) and the README refreshed (`cargo run --profile envelope --example perf_envelope`; see [Reproducing](docs/BENCHMARKS.md#reproducing))

## Code of Conduct

Expand Down
7 changes: 7 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,13 @@ opt-level = "z"

# uniffi-bindgen reads UNIFFI_META symbols from .symtab (not .dynsym).

# examples/perf_envelope.rs (#394): opt-level 3, as an application's release
# build compiles the library; `release` is tuned for the REPL binary's size.
[profile.envelope]
inherits = "release"
opt-level = 3
strip = "none"

[profile.bench]
strip = "none"
debug = true
Expand Down
18 changes: 13 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,12 +247,20 @@ See [ROADMAP.md](ROADMAP.md) for planned work and current direction.

## Performance

See [BENCHMARKS.md](docs/BENCHMARKS.md) for the current reproducible local snapshot and benchmark commands.
| Point query at 1M facts | 4.3–4.5 s (selective B+tree lookup for bounded patterns; O(N) for full-attribute scans) |
| Open time at 1M facts | 1.31 s (2.4× faster than v5 — indexes no longer loaded into RAM) |
| Peak heap at 1M facts | 1.05 GB (~21% less than v5 — indexes paged in on demand) |
Performance envelope on the `v3` branch (format v8), measured up to 10M facts on a laptop (Intel i7-1065G7, SSD, page cache warm; see [BENCHMARKS.md](docs/BENCHMARKS.md#performance-envelope-v300-394) for every shape, RSS and the method):

File-backed databases enforce a maximum fact size of **4 080 serialised bytes** per fact. In-memory databases have no limit.
| Workload | 100K facts | 1M facts | 10M facts |
|---|---:|---:|---:|
| Point lookup (bound entity + attribute, incl. `:as-of` / `:valid-at`), p50 / p99 | 13 µs / 16 µs | 12 µs / 48 µs | 12 µs / 46 µs |
| Entity with 1,000 versions, p50 / p99 | 0.26 / 0.30 ms | 0.26 / 0.29 ms | 0.27 / 0.30 ms |
| Open a file (RSS after open: 3 MiB) | 0.12 ms | 0.14 ms | 0.12 ms |
| Checkpoint after 1 new fact, p50 / p99 | 3.1 / 5.1 ms | 2.9 / 4.3 ms | 3.8 / 21 ms |
| Attribute scan (N/6 rows), p50 | 43 ms | 0.46 s | 4.8 s |
| Selective 2-clause join (1 row), p50 | 54 ms | 0.61 s | 6.5 s |

Point lookups and time travel on one entity stay flat at every tested size. Scans, aggregations and joins grow linearly with the facts of the attributes they touch, because a query materializes them before joining (#473). They are interactive up to about 100K facts. Not supported yet: rule queries on databases with 1M or more facts (#528), and graphs over 10M facts (not tested).

File-backed databases limit string values to 4,068 bytes (`WAL-003`). In-memory databases have no limit.

**Durability tuning:** every write is `fsync`'d immediately by default (`SyncMode::Full`). Bulk loaders/migrations that can safely re-run from a checkpoint watermark on failure can trade that for throughput with `OpenOptions::new().synchronous(SyncMode::Normal)` — `checkpoint()` still fsyncs unconditionally in both modes. See the [Performance Tuning](https://project-minigraf.github.io/minigraf-docs/latest/performance-tuning/#configuration-knobs) for the full tradeoff and the write-batching pattern that pairs with it.

Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ Scope — release-gate testing and documentation:
- Persistent fuzz corpus and an operation-sequence target (#387; done on `v3`: corpus caching, weekly `cmin`, the v8 page targets, and the `ops_sequence` and `corrupt_file` targets)
- Fault injection across save/WAL/recovery (#390)
- Long-running soak test at scale (#392; done on `v3`: `tests/soak_test.rs`, 10M facts and a 5-hour churn weekly, alternating `main` and `v3`)
- Benchmarks at 100K–10M facts and a published performance envelope (#394)
- Benchmarks at 100K–10M facts and a published performance envelope (#394; on `v3`: `examples/perf_envelope.rs`, results in `docs/BENCHMARKS.md`; follow-ups #528–#532)
- Higher storage/WAL coverage gates and branch-coverage gates (#396; on `v3`: line and branch floors per file in `coverage-thresholds.toml`, checked on every PR)
- Durability, recovery and operations guide (#398)

Expand Down
8 changes: 7 additions & 1 deletion benches/minigraf_bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,13 @@ fn bench_insert_file(c: &mut Criterion) {
// ── Task 5: query/ ────────────────────────────────────────────────────────────

fn bench_query(c: &mut Criterion) {
const SCALES: &[(&str, usize)] = &[("1k", 1_000), ("10k", 10_000)];
// 100k and 1m track the performance envelope (#394) in the nightly run.
const SCALES: &[(&str, usize)] = &[
("1k", 1_000),
("10k", 10_000),
("100k", 100_000),
("1m", 1_000_000),
];

// point_entity: EAVT range scan on a known entity
{
Expand Down
Loading
Loading