diff --git a/CHANGELOG.md b/CHANGELOG.md index 91ae0c6403..793748080a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Model-based storage test (#385).** `tests/model_based_test.rs` runs random sequences of `transact` (multi-valued batches, transaction- and fact-level valid-time windows, a triple repeated with another window), `retract`, write transactions (committed or rolled back, retracting their own assertions), checkpoints, reopens (default, a 4-page cache, auto-checkpoint every 2 writes) and simulated crashes (a copy of the `.graph` file and WAL taken while the handle is open, so the WAL is replayed) against a file in a temp dir. After every step, `current_tx_count`, `verify`, full rows with windows and tx counts, and entity-, entity+attribute-, attribute-, attribute+value-, value-bound and full-scan queries, under `:as-of N` and `:valid-at T`, must match a plain reference model (one valid-time window per triple, #435; API-011 for two windows of one triple in one statement and API-019 for an empty or inverted window, from `execute` and `tx.execute`, #436; the last statement that writes a triple decides it at commit, #477). Reads inside a write transaction are checked against what its commit will hold. Normal CI runs 24 cases; the nightly property-test workflow runs 30,000. - **SIGKILL crash test checks committed data (#384).** `tests/crash_kill_test.rs` used to check only that the file reopened and one attribute query ran, so it missed #370 and could not reach #371. The child now runs a seeded workload (multi-valued batches, batched retracts, re-assertions) and logs each transaction after `execute` returns; it checkpoints after every transaction, every third, never (WAL replay), automatically every 2 writes, or every third with `SyncMode::Normal`. After a `SIGKILL` at a random point, the reopened file must hold exactly the model after the last logged transaction or the one in flight, through a full scan, attribute (AEVT), attribute+value (AVET) and entity-bound (EAVT) queries and `:as-of`, with `verify` clean. The checks run again after a second reopen and after a checkpoint plus reopen. Per-PR CI runs 5 rounds (one per mode); the nightly crash-kill workflow runs 100 per OS. `MINIGRAF_CRASH_KILL_SEED` replays a failure. - **Fuzzing reaches the v8 decoders, and the corpus is kept between runs (#387, #375).** On `v3`, the `btree_page` and `fact_page` targets still wrote the pre-release single-header layout and stopped at `STG-032` on every input, and `file_header` rarely got past the meta CRC. The page targets now start from a real v8 file built through the public API (rewritten with `LogWriter` at fixed tx times so its bytes are the same in every process). They overwrite one page with the input and recompute that page's checksum. `btree_page` fuzzes any leaf or internal node of the five trees, `file_header` fuzzes either meta slot (with or without a matching CRC, with or without the other slot), and `fact_page` fuzzes the fact page of a v7 golden file through migration and the read-only load. Each run executes one input-chosen probe query or `verify()`. The nightly fuzz workflow now runs every target in its own parallel job, on `main` and `v3` on alternating crons, and keeps each target's corpus in the Actions cache. A new weekly workflow (`fuzz-corpus.yml`) minimizes the cached corpus with `cargo fuzz cmin`, keeps every committed `seed_*` file, and proposes the result as a pull request. +- **Two fuzz targets check results, not just crashes (#387).** `ops_sequence` decodes the input into up to 32 storage operations (transact, retract, a write transaction that commits or rolls back, checkpoint, reopen, and a crash that opens a copy of the file and WAL) and, after each one, compares the full rows now and at an input-chosen `:as-of`, plus one input-chosen query shape, with the reference model of the model-based test. It uses that test's driver and checks, now in `tests/reference_model`, so the proptest test and the fuzzer share one oracle. `corrupt_file` damages a file written over three checkpoints with up to 16 byte flips and an optional truncation, without fixing any checksum, and requires every query to fail or to return exactly the answers of the last checkpoint or the one before it, all from the same one. Its first run found that a file cut to under one page opens as a new, empty database (#506); the target skips those lengths until that is fixed. ### Known issues diff --git a/ROADMAP.md b/ROADMAP.md index 4e353541f2..3343805dd7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -91,7 +91,7 @@ Scope — release-gate testing and documentation: - SIGKILL crash test that checks committed data through EAVT, AEVT and full scan (#384; done on `v3`: `tests/crash_kill_test.rs`, 100 rounds per OS nightly) - Model-based test of transact/retract/checkpoint/reopen (#385; done on `v3`: `tests/model_based_test.rs`, 30,000 cases nightly) - Wider property tests: file-backed, batched, retracts, temporal, joins (#386) -- Persistent fuzz corpus and an operation-sequence target (#387; corpus caching, weekly `cmin` and the v8 page targets done on `v3`; `ops_sequence` and `corrupt_file` targets next) +- Persistent fuzz corpus and an operation-sequence target (#387; done on `v3`: corpus caching, weekly `cmin`, the v8 page targets, and the `ops_sequence` and `corrupt_file` targets) - Fault injection across save/WAL/recovery (#390) - Long-running soak test at scale (#392) - Benchmarks at 100K–10M facts and a published performance envelope (#394) diff --git a/fuzz/Cargo.toml b/fuzz/Cargo.toml index 7eb259d8e6..bfc1d5535b 100644 --- a/fuzz/Cargo.toml +++ b/fuzz/Cargo.toml @@ -14,6 +14,9 @@ libfuzzer-sys = "0.4" minigraf = { path = ".." } tempfile = "3" crc32fast = "1" +# ops_sequence includes tests/reference_model, shared with the model-based test. +proptest = "1" +uuid = { version = "1.0", features = ["v5"] } [[bin]] name = "wal_entry" @@ -50,3 +53,15 @@ name = "datalog_eval" path = "fuzz_targets/datalog_eval.rs" test = false doc = false + +[[bin]] +name = "ops_sequence" +path = "fuzz_targets/ops_sequence.rs" +test = false +doc = false + +[[bin]] +name = "corrupt_file" +path = "fuzz_targets/corrupt_file.rs" +test = false +doc = false diff --git a/fuzz/corpus/corrupt_file/seed_flip_free_list.bin b/fuzz/corpus/corrupt_file/seed_flip_free_list.bin new file mode 100644 index 0000000000..7506521fa8 Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_free_list.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_flip_internal.bin b/fuzz/corpus/corrupt_file/seed_flip_internal.bin new file mode 100644 index 0000000000..c8e41ad57a Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_internal.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_flip_leaf.bin b/fuzz/corpus/corrupt_file/seed_flip_leaf.bin new file mode 100644 index 0000000000..1cea8b7308 Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_leaf.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_flip_meta_a.bin b/fuzz/corpus/corrupt_file/seed_flip_meta_a.bin new file mode 100644 index 0000000000..45ff07ed9e Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_meta_a.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_flip_meta_b.bin b/fuzz/corpus/corrupt_file/seed_flip_meta_b.bin new file mode 100644 index 0000000000..a716ac1c57 Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_meta_b.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_flip_meta_b_read_only.bin b/fuzz/corpus/corrupt_file/seed_flip_meta_b_read_only.bin new file mode 100644 index 0000000000..4e1c533653 Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_meta_b_read_only.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_flip_value.bin b/fuzz/corpus/corrupt_file/seed_flip_value.bin new file mode 100644 index 0000000000..4e437ae312 Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_flip_value.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_no_edits.bin b/fuzz/corpus/corrupt_file/seed_no_edits.bin new file mode 100644 index 0000000000..40b450dd9d Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_no_edits.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_no_edits_read_only.bin b/fuzz/corpus/corrupt_file/seed_no_edits_read_only.bin new file mode 100644 index 0000000000..51d72fb29f Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_no_edits_read_only.bin differ diff --git a/fuzz/corpus/corrupt_file/seed_truncate_last_page.bin b/fuzz/corpus/corrupt_file/seed_truncate_last_page.bin new file mode 100644 index 0000000000..359b7b82d3 Binary files /dev/null and b/fuzz/corpus/corrupt_file/seed_truncate_last_page.bin differ diff --git a/fuzz/corpus/ops_sequence/seed_reopen_checkpoint_crash_write_tx.bin b/fuzz/corpus/ops_sequence/seed_reopen_checkpoint_crash_write_tx.bin new file mode 100644 index 0000000000..4c6967ed06 --- /dev/null +++ b/fuzz/corpus/ops_sequence/seed_reopen_checkpoint_crash_write_tx.bin @@ -0,0 +1 @@ +AAAAAAAA?vAAAAêA2AãAAAÃë=$Å \ No newline at end of file diff --git a/fuzz/corpus/ops_sequence/seed_retract_checkpoint_reopen_crash.bin b/fuzz/corpus/ops_sequence/seed_retract_checkpoint_reopen_crash.bin new file mode 100644 index 0000000000..30b248069e Binary files /dev/null and b/fuzz/corpus/ops_sequence/seed_retract_checkpoint_reopen_crash.bin differ diff --git a/fuzz/corpus/ops_sequence/seed_transact_reopen_write_tx.bin b/fuzz/corpus/ops_sequence/seed_transact_reopen_write_tx.bin new file mode 100644 index 0000000000..6b7b9e5ca5 --- /dev/null +++ b/fuzz/corpus/ops_sequence/seed_transact_reopen_write_tx.bin @@ -0,0 +1 @@ +ÝÝÝ?ÿZñ·È°£PWžBÀ[0ŒÉNéNq \ No newline at end of file diff --git a/fuzz/corpus/ops_sequence/seed_write_tx_commit_rollback.bin b/fuzz/corpus/ops_sequence/seed_write_tx_commit_rollback.bin new file mode 100644 index 0000000000..c11c729e1e --- /dev/null +++ b/fuzz/corpus/ops_sequence/seed_write_tx_commit_rollback.bin @@ -0,0 +1 @@ +<<>, +} + +struct Template { + bytes: Vec, + /// Answers after each of the three checkpoints, oldest first. + generations: Vec, +} + +fn rows(db: &Minigraf, query: &str) -> Option> { + match db.execute(query).ok()? { + QueryResult::QueryResults { results, .. } => { + let mut out: Vec = results.iter().map(|r| format!("{r:?}")).collect(); + out.sort(); + Some(out) + } + _ => None, + } +} + +fn answers(db: &Minigraf) -> Answers { + Answers { + tx_count: db.current_tx_count(), + rows: QUERIES + .iter() + .map(|q| rows(db, q).expect("template query")) + .collect(), + } +} + +fn template() -> &'static Template { + static T: OnceLock