diff --git a/CHANGELOG.md b/CHANGELOG.md index 68ee32c6ff..a5382c12f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Cursor API: `Minigraf::query()` and `PreparedQuery::query()`** (#432). Both return a `Cursor` that hands a query's rows back in batches: `vars()`, `next_batch(max_rows) -> Result, _>`, `close()`, and an iterator over rows. The answer is fixed when the cursor opens, so writes that commit while it is open do not change it. A cursor borrows nothing from the database handle: it can outlive it and move to another thread. For now the answer is still computed when the cursor opens, under the same limits as `execute()`; the API is shaped so the engine can later produce rows as they are pulled without breaking callers. `query()` rejects `transact`, `retract` and `rule` with the new `API-012`, and a query with bind slots with `API-010`. `execute()` is unchanged. Rust API only for now; the bindings follow. - **`Minigraf::verify()` and `Minigraf::rebuild_indexes()`** (#373). Page checksums turn a torn or rotted page into an error, but they cannot see a page that is valid and says the wrong thing. `verify()` walks every committed page and returns an `IntegrityReport` with one structured error per finding: an index that holds different facts than EAVT or a different count than the meta page (new `STG-038`), keys out of order or a page reached twice (new `STG-039`), dictionary maps that disagree (new `STG-040`), ids with no dictionary entry (`STG-036`), and leaked, doubly used or wrongly free pages (`STG-035`). It keeps no per-fact state: indexes are compared by an order-independent digest. `rebuild_indexes()` rebuilds EAVT, AEVT, AVET and VAET from an intact index that agrees with another one (or the only intact one), derives the free list again, and commits like a checkpoint, including uncheckpointed writes; a crash at any point leaves the previous checkpoint. If no index can serve as the source, or the dictionary is damaged, it fails with new `STG-041` and writes nothing. Both are Rust API only for now; the bindings follow. ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index a8923e6ab4..6e20c7f530 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -138,7 +138,8 @@ cargo run < demos/demo_negation.txt 5. **`src/repl.rs`** — Interactive REPL; TTY-aware (suppresses prompts/banner for piped input) -6. **`src/db.rs`** — Public API: `Minigraf::open/execute/prepare/begin_write/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `IntegrityReport`, `OpenOptions::page_cache_size` +6. **`src/db.rs`** — Public API: `Minigraf::open/execute/query/prepare/begin_write/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `IntegrityReport`, `OpenOptions::page_cache_size` + - `src/cursor.rs`: `Cursor` / `Batch` — owned, `Send` cursor returned by `Minigraf::query` and `PreparedQuery::query` (#432); answer computed at open today, shaped for streaming operators 7. **`src/wal.rs`** — Fact-level sidecar WAL, CRC32-protected entries, crash recovery @@ -194,7 +195,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are ## Test Coverage -**1267 tests** (1258 passing, 9 ignored; unit + integration + doc). +**1285 tests** (1276 passing, 9 ignored; unit + integration + doc). See `docs/TEST_COVERAGE.md` for the full per-file breakdown. **Testing conventions** — see the Testing Conventions section below before writing any tests. diff --git a/README.md b/README.md index aa7336f7c7..b5e997ff06 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,12 @@ let results = db.execute(r#" [?friend :person/name ?friend-name]]) "#)?; +// Cursor — read a large answer in batches; the answer is fixed at open +let mut cursor = db.query("(query [:find ?name :where [?e :person/name ?name]])")?; +while let Some(batch) = cursor.next_batch(1000)? { + for row in batch.rows() { /* row[0] is ?name */ } +} + // Explicit transaction — all-or-nothing let mut tx = db.begin_write()?; tx.execute(r#"(transact [[:alice :person/age 31]])"#)?; @@ -105,7 +111,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit ```bash cargo run # interactive Datalog REPL -cargo test # run 1267 tests +cargo test # run 1285 tests cargo run < demos/demo_recursive.txt # recursive rules demo ``` diff --git a/ROADMAP.md b/ROADMAP.md index 3b80de0838..26e2e3c885 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -65,6 +65,10 @@ Scope — format and storage integrity: - One current valid-time window per `(e, a, v)`: a later assertion replaces the earlier window, and two windows of one fact in one transaction are `API-011` (#435; done on `v3`) - `btree_page` fuzz target that reaches node decoding (#375) +Scope — query API: + +- Cursor API: `Minigraf::query` and `PreparedQuery::query` return an owned `Cursor` that delivers rows in batches, with the answer fixed at open (#432; Rust API done on `v3`, with the current evaluator computing the answer at open). The binding cursor objects follow. Streaming operators replace the evaluator behind the cursor in 3.x without further API changes; how operators bound their memory (spill, scratch pages in the `.graph` file, or a clear error) is still open. + Scope — release-gate testing and documentation: - Golden-file compatibility corpus for every format version (#391; done on `v3`: six v7 files from 2.0.3 checked after migration, seven v8 files frozen at `23aa57c`) diff --git a/docs/ERROR_REFERENCE.md b/docs/ERROR_REFERENCE.md index 26ae510b4f..9926740986 100644 --- a/docs/ERROR_REFERENCE.md +++ b/docs/ERROR_REFERENCE.md @@ -167,6 +167,7 @@ with no `CodedError` anywhere in its chain. | API-009 | WAL not initialized | Database API | | API-010 | Query with bind slots passed to execute() | Database API | | API-011 | Two valid-time windows for one fact in one transaction | Database API | +| API-012 | Non-query command passed to query() | Database API | | INT-000 | Unclassified internal error | Internal | | INT-001 | WriteTransaction already in progress on this thread | Internal | | INT-002 | Invalid entity (API layer) | Internal | @@ -2452,6 +2453,25 @@ let result = pq.execute(&[ **Scenario**: A batch import writes the same fact once per period it was true, all in one transaction. +### API-012 Non-query command passed to query() + +**Error text**: `only (query ...) commands can be opened as a cursor; got {}` + +**Cause**: `db.query()` returns a `Cursor` over the rows of a query. It was given a `transact`, `retract` or `rule` command, which has no rows. The `{}` names the command. Nothing is written or registered. + +**Resolution**: +- Run writes and rule definitions with `db.execute()` (or inside a `WriteTransaction`), and open cursors only for `(query ...)` commands. + +```rust +db.execute(r#"(transact [[:alice :person/name "Alice"]])"#)?; +let mut cursor = db.query("(query [:find ?name :where [?e :person/name ?name]])")?; +while let Some(batch) = cursor.next_batch(1000)? { + for row in batch.rows() { /* ... */ } +} +``` + +**Scenario**: Code that dispatches every command string through one function switches from `execute()` to `query()` for all of them. + --- ## INT — Internal Errors diff --git a/docs/TEST_COVERAGE.md b/docs/TEST_COVERAGE.md index 7820215fd5..20383c3d5c 100644 --- a/docs/TEST_COVERAGE.md +++ b/docs/TEST_COVERAGE.md @@ -2,11 +2,11 @@ ## Coverage Summary -**Verified**: 2026-10-06 with `cargo test --quiet` +**Verified**: 2026-10-07 with `cargo test --quiet` -**Result**: 1,258 passing tests and 9 ignored tests (1,267 total) +**Result**: 1,276 passing tests and 9 ignored tests (1,285 total) -This covers the unreleased file-format-v8 work (#287, #371): the index-key value-collision fix, the v7→v8 migration, and the drop of formats v1–v6. It otherwise still covers the public database API, the Datalog engine, storage and recovery paths, v2.0.0's locking and structured-error-code changes, and v2.0.1's regression test for rebuilding indexes on open after several checkpoints (#370). Merged from the v2.x line: `tests/point_query_history_test.rs` and new storage unit tests check that attribute-narrowed bound-entity lookups return exactly what a full scan returns, and that `net_asserted_facts` matches its previous implementation on randomized histories (#323). Attribute-scan tests check that the AEVT range covers exactly the queried attribute, including non-ASCII names such as `:丿` and prefix siblings such as `:ab` for `:a` (#381). Incremental B+tree rebuild tests check that checkpoints which copy untouched index leaves produce exactly the entries and file checksum a full rebuild would, including over repeated checkpoints, that single-fact checkpoints do not fragment the index, and that a damaged slot directory is rejected rather than copied (#315). B+tree cursor tests check that every scan walks the tree, not the leaf `next_leaf` chain: full scans and start positions match the input, `seek` from the current position lands where a fresh descent from the root would (randomized, plus within a leaf, to a sibling, across subtrees, past the end, into the gap after a leaf, and backwards as a no-op), a seek to the sibling leaf reads exactly one page, reads are unchanged with every `next_leaf` zeroed, a child pointer into a non-B+tree page is STG-013, and a cycle in child pointers is an error, not a hang (v8 storage format PR 1, #374). Directory-sync unit tests check that creating the `.graph` file or the WAL, and deleting the WAL, fsync the parent directory after the file operation, and that reopening an existing file does not (#389). Disjunction tests check that `or-join` returns an empty result, not INT-031, when no rows reach it: on an empty database, after a pattern that matches nothing, after a `not` that removes every row, and in a rule body (#405). A stratification unit test builds the same two-rule negative cycle 64 times and checks that the INT-054 message names the same predicate pair every time (#410). Magic-sets tests check that a recursive rule with a literal start that recurses through another rule returns the same results as without the literal (#297). API tests check that `execute()` and `WriteTransaction::execute()` reject `$slot` bind tokens in every position with API-010 (#407). The REPL `run()` test now runs in a child process with null stdin, so it no longer hangs when the test runner's stdin is an open non-TTY stream (#415). v8 page-format tests (PR 2, #374, #388) check the 24-byte page header (a flipped bit, a wrong page id and a future generation each give their own STG code, legacy page types fail before the CRC, and a failed page is never cached), meta-page encoding, the free-list chain, WAL v2's base generation, and every row of the meta-selection table: both slots valid, a torn newest slot with the WAL's base generation (opens at the older one), a WAL newer than the valid meta, pages of the next generation on the free list or past `page_count` with no WAL (all STG-033, with the file unmodified), an unknown feature bit (STG-034), and a pre-release v8 single header (STG-032). Fault-injection tests crash `save()` at every page write (cleanly and with torn pages, including the meta write) and every sync, and crash a v7 migration at every write; each reopen finds exactly the old or the new checkpoint, and every reachable page plus the free list covers the file exactly. Twenty-five consecutive saves never write a page the previous meta references. Covering-key tests (PR 3, #433) check that byte order equals logical order for every key component and value type on random tuples (negative, zero and large integers, NaN and ±0.0, strings with 0x00, 64- and 65-byte strings, FOREVER, newest-transaction-first) and that every component round-trips; that prefix-compressed leaves and shortest-separator internal nodes round-trip, agree with a linear scan, keep `max(left) < s ≤ min(right)`, and reject an overflow; that the byte-key B+tree's builds, incremental rebuilds, cursor and `seek` match their expected sets; that dictionary ids are assigned once, never reused, shared between a keyword attribute and value, and deterministic; that a re-asserted or retracted long value writes no new value page and a forced hash collision still finds the right value; that one transaction with two timestamps is STG-037 and a missing dictionary entry STG-036; that committed reads by entity, entity and attribute, attribute, and in full equal an in-memory model across eight checkpoints and a reopen; that short-valued reads touch no value page; that a v7 file with every value type, long strings, refs and retractions migrates to exactly the same facts; that a pre-covering-key v8 file is STG-032; and that string values over 4,068 bytes and attributes or keywords over 1,024 bytes are WAL-003. A WAL test checks that reopening next to a fully checkpointed WAL does not reuse transaction numbers (#447). Copy-on-write checkpoint tests (PR 4, #434) check that lazy free-list pops and pushes conserve every page across page boundaries, partial head pages and chain exhaustion, never hand out a page freed in the same generation, and share unread chain pages unchanged; that a copy-on-write insert equals a sorted merge for random batches (including keys below the first and above the last, many into one leaf, root growth to depth 3, and 1 KB separators that split internal nodes), leaves the old tree readable and unchanged, frees exactly the replaced nodes, writes only new pages, writes exactly one leaf and two internal nodes for one entry in a depth-3 tree, and does not fragment leaves under single inserts; that a checkpoint after 1 or 100 right-edge facts writes at most the tree-depth difference more pages at 100,000 facts than at 10,000, and at most one path per new entry per tree for random keys; and that four reader threads querying throughout 40 checkpoints never fail and never see the fact count go backwards. Hardening tests (PR 5) damage an index leaf, an internal node, a dictionary leaf, a value page and a free-list page in turn. Every committed read or checkpoint that reaches the damaged page fails with STG-029, never with wrong or fewer rows. A query through `Minigraf` on a file with damaged leaves fails with STG-029, and the file still opens at its last checkpoint. A size test measures at most 200 bytes per fact (142 measured) on 144k facts in #433's shape; its 1.15M-fact variant is ignored locally and runs in the nightly smoke workflow. A shared dictionary-cache test checks that cached ident and timestamp lookups return the same facts and that the cache stays bounded. Merged from the v2.x line: storage and `edge_cases_test.rs` tests check that more than 65,535 uncheckpointed facts each resolve to their own fact through entity lookups and attribute scans, in memory and file-backed (#445). Merged from the v2.x line: `tests/golden_corpus_test.rs` opens a copy of each committed golden file in `tests/golden/` (v7 files written by minigraf 2.0.3: one checkpoint, several checkpoints, indexes rebuilt on open, a crash-left WAL, an already-checkpointed WAL, same-transaction multi-values) and checks its manifest's CRC32, `tx_count` and query results after open, after a checkpoint and reopen, and after one more write; a second test checks that every golden file has a manifest (#391). On `v3` the same test checks that each v7 file has a v8 meta page after its first open and gives the same results after reopening the migrated file, and adds seven v8 golden files (one checkpoint, seven copy-on-write checkpoints, a crash-left v2 WAL, an already-checkpointed v2 WAL, same-transaction multi-values, a v7 file migrated in place, and long values at the size limits); the #371 queries, skipped on v2.x, run and pass. Integrity tests (#373) check that intact files verify with no problems and an exact fact and page count, and that `verify` reports each kind of damage with its code: an EAVT, AEVT, AVET or VAET leaf missing an entry (STG-038), keys out of order and a page reached from two trees (STG-039), a rotted index leaf (STG-029), a free page still in use and a leaked page (STG-035), a missing transaction entry (STG-036), and an entity id whose UUID maps to another id (STG-040). After each index or free-list damage, `rebuild_indexes` leaves a file that verifies clean, reads back every fact and accounts for every page. The source is an index that agrees with another one, or the only intact one; three intact indexes that all disagree, or a damaged dictionary, give STG-041 with the file unchanged. A rebuild commits pending facts, including new ids and long values. A crash at every write (clean and torn) and every sync of a rebuild reopens either at the old generation with only its old damage or at the new one with none, never losing a fact. Through `Minigraf` (`tests/integrity_test.rs`), a rebuild commits WAL entries and deletes the WAL, and flipping a byte in each leaf of a file in turn is always reported, then either repaired (index leaves) or refused with STG-041 and the file unchanged (dictionary leaves). Live-read tests (#379) check that `triple_len` ends exactly where `tx↓` begins for every value type and index; that on random histories (several valid-time windows, retractions, re-assertions, several values per attribute, long values, 12 transactions per checkpoint) the committed live read for every scan kind and `:as-of` point is exactly the net-asserted set, that it stays equivalent with pending facts on top, and after reopen; and that a point read on a triple asserted and retracted 2,000 times reads under a third of the pages a history read does, with or without `:as-of`. `tests/live_reads_test.rs` runs the same history in a checkpointed file database and in memory and checks that point, attribute, full-scan, join, valid-time, `:as-of` and rule queries return the same rows, before and after reopen. One-window tests (#435) check that `net_asserted_facts` keeps each triple's newest transaction only (a later window replaces an earlier one, closing an open window, a retraction in the newest transaction hides it, records with two windows in one transaction keep both, once each) against a per-triple model on random histories; that the on-disk key walk agrees at every `:as-of` before and after reopen; and that `API-011` rejects two windows of one fact in one transaction without taking a `tx_count`. `tests/valid_time_window_test.rs` checks replacing, closing, extending, reopening and retract-then-reassert through entity-bound and attribute queries in memory, on a checkpointed file and after reopen, and `API-011` from `transact` and from `WriteTransaction::commit`. +This covers the unreleased file-format-v8 work (#287, #371): the index-key value-collision fix, the v7→v8 migration, and the drop of formats v1–v6. It otherwise still covers the public database API, the Datalog engine, storage and recovery paths, v2.0.0's locking and structured-error-code changes, and v2.0.1's regression test for rebuilding indexes on open after several checkpoints (#370). Merged from the v2.x line: `tests/point_query_history_test.rs` and new storage unit tests check that attribute-narrowed bound-entity lookups return exactly what a full scan returns, and that `net_asserted_facts` matches its previous implementation on randomized histories (#323). Attribute-scan tests check that the AEVT range covers exactly the queried attribute, including non-ASCII names such as `:丿` and prefix siblings such as `:ab` for `:a` (#381). Incremental B+tree rebuild tests check that checkpoints which copy untouched index leaves produce exactly the entries and file checksum a full rebuild would, including over repeated checkpoints, that single-fact checkpoints do not fragment the index, and that a damaged slot directory is rejected rather than copied (#315). B+tree cursor tests check that every scan walks the tree, not the leaf `next_leaf` chain: full scans and start positions match the input, `seek` from the current position lands where a fresh descent from the root would (randomized, plus within a leaf, to a sibling, across subtrees, past the end, into the gap after a leaf, and backwards as a no-op), a seek to the sibling leaf reads exactly one page, reads are unchanged with every `next_leaf` zeroed, a child pointer into a non-B+tree page is STG-013, and a cycle in child pointers is an error, not a hang (v8 storage format PR 1, #374). Directory-sync unit tests check that creating the `.graph` file or the WAL, and deleting the WAL, fsync the parent directory after the file operation, and that reopening an existing file does not (#389). Disjunction tests check that `or-join` returns an empty result, not INT-031, when no rows reach it: on an empty database, after a pattern that matches nothing, after a `not` that removes every row, and in a rule body (#405). A stratification unit test builds the same two-rule negative cycle 64 times and checks that the INT-054 message names the same predicate pair every time (#410). Magic-sets tests check that a recursive rule with a literal start that recurses through another rule returns the same results as without the literal (#297). API tests check that `execute()` and `WriteTransaction::execute()` reject `$slot` bind tokens in every position with API-010 (#407). The REPL `run()` test now runs in a child process with null stdin, so it no longer hangs when the test runner's stdin is an open non-TTY stream (#415). v8 page-format tests (PR 2, #374, #388) check the 24-byte page header (a flipped bit, a wrong page id and a future generation each give their own STG code, legacy page types fail before the CRC, and a failed page is never cached), meta-page encoding, the free-list chain, WAL v2's base generation, and every row of the meta-selection table: both slots valid, a torn newest slot with the WAL's base generation (opens at the older one), a WAL newer than the valid meta, pages of the next generation on the free list or past `page_count` with no WAL (all STG-033, with the file unmodified), an unknown feature bit (STG-034), and a pre-release v8 single header (STG-032). Fault-injection tests crash `save()` at every page write (cleanly and with torn pages, including the meta write) and every sync, and crash a v7 migration at every write; each reopen finds exactly the old or the new checkpoint, and every reachable page plus the free list covers the file exactly. Twenty-five consecutive saves never write a page the previous meta references. Covering-key tests (PR 3, #433) check that byte order equals logical order for every key component and value type on random tuples (negative, zero and large integers, NaN and ±0.0, strings with 0x00, 64- and 65-byte strings, FOREVER, newest-transaction-first) and that every component round-trips; that prefix-compressed leaves and shortest-separator internal nodes round-trip, agree with a linear scan, keep `max(left) < s ≤ min(right)`, and reject an overflow; that the byte-key B+tree's builds, incremental rebuilds, cursor and `seek` match their expected sets; that dictionary ids are assigned once, never reused, shared between a keyword attribute and value, and deterministic; that a re-asserted or retracted long value writes no new value page and a forced hash collision still finds the right value; that one transaction with two timestamps is STG-037 and a missing dictionary entry STG-036; that committed reads by entity, entity and attribute, attribute, and in full equal an in-memory model across eight checkpoints and a reopen; that short-valued reads touch no value page; that a v7 file with every value type, long strings, refs and retractions migrates to exactly the same facts; that a pre-covering-key v8 file is STG-032; and that string values over 4,068 bytes and attributes or keywords over 1,024 bytes are WAL-003. A WAL test checks that reopening next to a fully checkpointed WAL does not reuse transaction numbers (#447). Copy-on-write checkpoint tests (PR 4, #434) check that lazy free-list pops and pushes conserve every page across page boundaries, partial head pages and chain exhaustion, never hand out a page freed in the same generation, and share unread chain pages unchanged; that a copy-on-write insert equals a sorted merge for random batches (including keys below the first and above the last, many into one leaf, root growth to depth 3, and 1 KB separators that split internal nodes), leaves the old tree readable and unchanged, frees exactly the replaced nodes, writes only new pages, writes exactly one leaf and two internal nodes for one entry in a depth-3 tree, and does not fragment leaves under single inserts; that a checkpoint after 1 or 100 right-edge facts writes at most the tree-depth difference more pages at 100,000 facts than at 10,000, and at most one path per new entry per tree for random keys; and that four reader threads querying throughout 40 checkpoints never fail and never see the fact count go backwards. Hardening tests (PR 5) damage an index leaf, an internal node, a dictionary leaf, a value page and a free-list page in turn. Every committed read or checkpoint that reaches the damaged page fails with STG-029, never with wrong or fewer rows. A query through `Minigraf` on a file with damaged leaves fails with STG-029, and the file still opens at its last checkpoint. A size test measures at most 200 bytes per fact (142 measured) on 144k facts in #433's shape; its 1.15M-fact variant is ignored locally and runs in the nightly smoke workflow. A shared dictionary-cache test checks that cached ident and timestamp lookups return the same facts and that the cache stays bounded. Merged from the v2.x line: storage and `edge_cases_test.rs` tests check that more than 65,535 uncheckpointed facts each resolve to their own fact through entity lookups and attribute scans, in memory and file-backed (#445). Merged from the v2.x line: `tests/golden_corpus_test.rs` opens a copy of each committed golden file in `tests/golden/` (v7 files written by minigraf 2.0.3: one checkpoint, several checkpoints, indexes rebuilt on open, a crash-left WAL, an already-checkpointed WAL, same-transaction multi-values) and checks its manifest's CRC32, `tx_count` and query results after open, after a checkpoint and reopen, and after one more write; a second test checks that every golden file has a manifest (#391). On `v3` the same test checks that each v7 file has a v8 meta page after its first open and gives the same results after reopening the migrated file, and adds seven v8 golden files (one checkpoint, seven copy-on-write checkpoints, a crash-left v2 WAL, an already-checkpointed v2 WAL, same-transaction multi-values, a v7 file migrated in place, and long values at the size limits); the #371 queries, skipped on v2.x, run and pass. Integrity tests (#373) check that intact files verify with no problems and an exact fact and page count, and that `verify` reports each kind of damage with its code: an EAVT, AEVT, AVET or VAET leaf missing an entry (STG-038), keys out of order and a page reached from two trees (STG-039), a rotted index leaf (STG-029), a free page still in use and a leaked page (STG-035), a missing transaction entry (STG-036), and an entity id whose UUID maps to another id (STG-040). After each index or free-list damage, `rebuild_indexes` leaves a file that verifies clean, reads back every fact and accounts for every page. The source is an index that agrees with another one, or the only intact one; three intact indexes that all disagree, or a damaged dictionary, give STG-041 with the file unchanged. A rebuild commits pending facts, including new ids and long values. A crash at every write (clean and torn) and every sync of a rebuild reopens either at the old generation with only its old damage or at the new one with none, never losing a fact. Through `Minigraf` (`tests/integrity_test.rs`), a rebuild commits WAL entries and deletes the WAL, and flipping a byte in each leaf of a file in turn is always reported, then either repaired (index leaves) or refused with STG-041 and the file unchanged (dictionary leaves). Live-read tests (#379) check that `triple_len` ends exactly where `tx↓` begins for every value type and index; that on random histories (several valid-time windows, retractions, re-assertions, several values per attribute, long values, 12 transactions per checkpoint) the committed live read for every scan kind and `:as-of` point is exactly the net-asserted set, that it stays equivalent with pending facts on top, and after reopen; and that a point read on a triple asserted and retracted 2,000 times reads under a third of the pages a history read does, with or without `:as-of`. `tests/live_reads_test.rs` runs the same history in a checkpointed file database and in memory and checks that point, attribute, full-scan, join, valid-time, `:as-of` and rule queries return the same rows, before and after reopen. One-window tests (#435) check that `net_asserted_facts` keeps each triple's newest transaction only (a later window replaces an earlier one, closing an open window, a retraction in the newest transaction hides it, records with two windows in one transaction keep both, once each) against a per-triple model on random histories; that the on-disk key walk agrees at every `:as-of` before and after reopen; and that `API-011` rejects two windows of one fact in one transaction without taking a `tx_count`. `tests/valid_time_window_test.rs` checks replacing, closing, extending, reopening and retract-then-reassert through entity-bound and attribute queries in memory, on a checkpointed file and after reopen, and `API-011` from `transact` and from `WriteTransaction::commit`. Cursor tests (#432, `tests/cursor_test.rs`) check that `query()` batches partition the `execute()` answer for several batch sizes (never empty, never over `max_rows`, `max_rows == 0` still progresses, `None` stays `None`), that the iterator and aggregates, rules, `:as-of` and `:any-valid-time` match `execute()`, that a cursor opened before a `transact`, a `WriteTransaction` commit, a retract or a checkpoint returns the pre-write answer to the end (in memory and file-backed), that `transact`/`retract`/`rule` are `API-012` and bind slots `API-010`, that a `WriteTransaction` thread gets `INT-001`, that rule limits give `INT-020` as from `execute()`, that `PreparedQuery::query` matches `PreparedQuery::execute`, and that a cursor is `Send` and outlives its database handle. ### Covered Areas diff --git a/docs/superpowers/specs/2026-10-07-cursor-api-design.md b/docs/superpowers/specs/2026-10-07-cursor-api-design.md new file mode 100644 index 0000000000..f91c30aba3 --- /dev/null +++ b/docs/superpowers/specs/2026-10-07-cursor-api-design.md @@ -0,0 +1,89 @@ +# Cursor API (#432, v3.0.0 scope) — design + +**Issue:** #432 (streaming, pull-based query engine). **Branch:** `v3`. +**Scope decision (2026-10-01):** option 2 of the issue — land the public cursor API in +3.0.0 with the current materialising evaluator behind it, then replace operators one by +one in 3.x without further breaking changes. The spill/memory-budget design (goal 1) +is explicitly deferred: the issue comment of 2026-10-03 asks for a revisit (index-ordered +streaming, scratch pages inside the `.graph` file, or a clear error over budget) and the +cursor API does not depend on it. + +## Goal + +Fix the public shape now, so that a client written against 3.0.0 never has to change when +the engine starts streaming: + +- `Minigraf::query(&str) -> Result` +- `PreparedQuery::query(&[(&str, BindValue)]) -> Result` +- `Cursor::vars()`, `Cursor::next_batch(max_rows) -> Result, MinigrafError>`, + `Cursor::close(self)`, and `impl Iterator for Cursor` yielding + `Result, MinigrafError>` rows. +- `Batch`: `len`, `is_empty`, `rows() -> &[Vec]`, `into_rows() -> Vec>`, + `IntoIterator`. + +`execute()` keeps its signature and behaviour; every existing test passes unchanged. + +## Decisions + +1. **`Cursor` is owned, `Send`, and has no lifetime.** A streaming cursor will need read + access to the store after `query()` returns. It gets that through the same `Arc`s + `PreparedQuery` already holds, never a borrow of `&Minigraf` and never a lock held for + the cursor's life: the store is append-only, so a snapshot is a bound on `tx_count`, not + a copy or a lock. Fixing this now is the one thing that would otherwise be a breaking + change later. +2. **`next_batch` returns `Result`.** Today it cannot fail after `query()` succeeds, but a + streaming engine can fail mid-stream (a damaged page, the recursive operator's + `max_derived_facts` limit). Iterator items are `Result` for the same reason. +3. **The public `Batch` is row-major.** Internal operator batches may become columnar; the + conversion happens once, at the cursor boundary, where rows leave the engine anyway. + Fields are private so the representation can change behind `rows()`/`into_rows()`. +4. **`max_rows == 0` is treated as 1**, so a loop of `next_batch(n)` always makes progress + and `None` is the only end-of-results signal. Batches are never empty. +5. **Snapshot.** A cursor's answer is fixed at `query()`: writes that commit while it is + open do not change it. Today this holds because the answer is computed at open; the + streaming engine must keep it by bounding reads at the `tx_count` current at open. + A test pins the guarantee. +6. **Limits.** `query()` uses the same executor and limits as `execute()`. + `max_results` and `max_derived_facts` bound recursive rule evaluation (`INT-020`); they + do not cap the rows of a plain query, for either entry point. The issue's "a cursor has + no result cap" therefore needs no change now. When the recursive operator is rewritten, + `max_derived_facts` stays as its safety limit. `PreparedQuery::query` matches + `PreparedQuery::execute`, which does not apply the `OpenOptions` limits (unchanged here). +7. **Commands other than `(query ...)`.** `query()` rejects `transact`, `retract` and `rule` + with new **`API-012`**: "only (query ...) commands can be opened as a cursor; got {}". + A query with bind slots fails with `API-010`, as `execute()` does. Called on a thread + that holds a `WriteTransaction`, it fails with `INT-001`, as `execute()` does (a cursor + there would silently miss the transaction's own writes). +8. **`close(self)`** drops the cursor. It exists so the API names the early-stop operation + the streaming engine will propagate upstream; dropping without `close` is equivalent. +9. **Out of scope:** `WriteTransaction::query` (additive later), `:limit` (#306, v3.1.0), + operator rewrite, spill/memory budget, FFI. The FFI `Cursor` object (`vars()`, + `next_batch(n) -> String`, `close()`) lives in the binding repos and gets its own issue; + it is additive over the core API. + +## Implementation + +- New module `src/cursor.rs` with `Cursor` and `Batch`; re-exported from `lib.rs`. + `Cursor { vars: Vec, rows: std::vec::IntoIter> }`. +- `Minigraf::query` parses, rejects non-queries (`API-012`) and unbound slots (`API-010`), + runs the read path `execute()` uses, and wraps the `QueryResults`. +- `PreparedQuery::query` calls the existing substitute + execute path and wraps the result. +- Error code `API-012` registered in `src/error.rs` and `docs/ERROR_REFERENCE.md`. + +## Tests (`tests/cursor_test.rs`) + +- Batches partition the `execute()` answer: same vars, same multiset of rows, each batch + `<= max_rows`, no empty batch, then `None` (and `None` again after the end). +- `max_rows == 0` makes progress. +- The iterator yields the same rows as `execute()`. +- Empty result: `vars` set, first `next_batch` is `None`. +- Aggregates, rules and `:as-of` through `query()` match `execute()`. +- Snapshot: a cursor opened before a `transact` (and before a `WriteTransaction` commit) + returns the pre-transaction answer to the end; on a file-backed database too. +- `transact`/`retract`/`rule` → `API-012`; bind slots → `API-010`; inside a + `WriteTransaction` thread → `INT-001`; parse error → the parser's code. +- `PreparedQuery::query` with binds matches `PreparedQuery::execute`. +- A recursive rule over `max_results` → `INT-020` from `query()`, as from `execute()`. +- `Cursor` is `Send` and can be consumed on another thread after `db` is dropped. +- Close after the first batch is accepted (counters for upstream work arrive with the + operator rewrite). diff --git a/src/cursor.rs b/src/cursor.rs new file mode 100644 index 0000000000..2e8aeed161 --- /dev/null +++ b/src/cursor.rs @@ -0,0 +1,135 @@ +//! Incremental delivery of query results (#432). +//! +//! [`crate::Minigraf::query`] and [`crate::PreparedQuery::query`] return a +//! [`Cursor`], which hands the answer back in [`Batch`]es of rows. The answer +//! is fixed when the cursor opens: writes that commit while it is open do not +//! change it. +//! +//! The engine behind the cursor currently computes the whole answer when the +//! cursor opens, so `max_results` and `max_derived_facts` apply exactly as they +//! do to [`crate::Minigraf::execute`]. The API is shaped for a streaming engine +//! that produces rows as they are pulled, which is why a cursor is owned (it +//! borrows nothing from the database handle) and why +//! [`Cursor::next_batch`] can fail. + +use crate::error::{ErrorCode, MinigrafError, err_coded}; +use crate::graph::types::Value; +use crate::query::datalog::executor::QueryResult; + +/// A forward-only cursor over the rows of one query. +/// +/// Pull rows with [`Cursor::next_batch`] or iterate over them; each item is one +/// row aligned with [`Cursor::vars`]. Stop early with [`Cursor::close`] or by +/// dropping the cursor. +/// +/// ``` +/// # use minigraf::Minigraf; +/// let db = Minigraf::in_memory().unwrap(); +/// db.execute(r#"(transact [[:alice :person/name "Alice"] [:bob :person/name "Bob"]])"#) +/// .unwrap(); +/// +/// let mut cursor = db +/// .query("(query [:find ?name :where [?e :person/name ?name]])") +/// .unwrap(); +/// assert_eq!(cursor.vars(), ["?name".to_string()].as_slice()); +/// while let Some(batch) = cursor.next_batch(1000).unwrap() { +/// for row in batch.rows() { +/// println!("{:?}", row[0]); +/// } +/// } +/// ``` +#[derive(Debug)] +pub struct Cursor { + vars: Vec, + rows: std::vec::IntoIter>, +} + +impl Cursor { + /// Wrap the answer of a query. Any other result is a write, which the + /// callers reject before executing. + pub(crate) fn from_result(result: QueryResult) -> anyhow::Result { + match result { + QueryResult::QueryResults { vars, results } => Ok(Cursor { + vars, + rows: results.into_iter(), + }), + _ => Err(err_coded!(ErrorCode::Api002)), + } + } + + /// The variable names of the `:find` clause, in order. Every row has one + /// value per variable. + pub fn vars(&self) -> &[String] { + &self.vars + } + + /// The next batch of at most `max_rows` rows, or `None` once every row has + /// been returned. A batch is never empty; `max_rows == 0` is treated as 1. + /// + /// # Errors + /// + /// Reserved for failures while rows are produced. The current engine + /// computes the answer when the cursor opens and never fails here. + pub fn next_batch(&mut self, max_rows: usize) -> Result, MinigrafError> { + let rows: Vec> = self.rows.by_ref().take(max_rows.max(1)).collect(); + if rows.is_empty() { + Ok(None) + } else { + Ok(Some(Batch { rows })) + } + } + + /// Stop reading. Equivalent to dropping the cursor. + pub fn close(self) {} +} + +impl Iterator for Cursor { + type Item = Result, MinigrafError>; + + fn next(&mut self) -> Option { + self.rows.next().map(Ok) + } + + fn size_hint(&self) -> (usize, Option) { + (0, None) + } +} + +/// One batch of rows from a [`Cursor`]. Each row is aligned with +/// [`Cursor::vars`]. +#[derive(Debug, Clone, PartialEq)] +pub struct Batch { + rows: Vec>, +} + +impl Batch { + /// The number of rows in this batch. + pub fn len(&self) -> usize { + self.rows.len() + } + + /// Whether this batch has no rows. Batches returned by + /// [`Cursor::next_batch`] are never empty. + pub fn is_empty(&self) -> bool { + self.rows.is_empty() + } + + /// The rows of this batch. + pub fn rows(&self) -> &[Vec] { + &self.rows + } + + /// Take the rows of this batch. + pub fn into_rows(self) -> Vec> { + self.rows + } +} + +impl IntoIterator for Batch { + type Item = Vec; + type IntoIter = std::vec::IntoIter>; + + fn into_iter(self) -> Self::IntoIter { + self.rows.into_iter() + } +} diff --git a/src/db.rs b/src/db.rs index d86f51a0fa..28d75a1efa 100644 --- a/src/db.rs +++ b/src/db.rs @@ -15,6 +15,7 @@ use crate::graph::types::{Fact, TxId, VALID_TIME_FOREVER}; /// in any practical context, avoiding the collision that `0` would have with the Unix /// epoch (1970-01-01T00:00:00Z), which is a legitimate `valid_from` value. pub(crate) const VALID_FROM_USE_TX_TIME: i64 = i64::MIN; +use crate::cursor::Cursor; use crate::error::{ErrorCode, MinigrafError, bail_coded, err_coded}; use crate::graph::FactStorage; use crate::graph::types::Value; @@ -674,17 +675,58 @@ impl Minigraf { } } else { // Read-only: no lock needed - let mut executor = DatalogExecutor::new_with_rules_and_functions( - self.inner.fact_storage.clone(), - self.inner.rules.clone(), - self.inner.functions.clone(), - ); - executor.set_limits( - self.inner.options.max_derived_facts, - self.inner.options.max_results, - ); - executor.execute(cmd) + self.read_executor().execute(cmd) + } + } + + /// The executor for a read-only command: no lock, the database's limits. + fn read_executor(&self) -> DatalogExecutor { + let mut executor = DatalogExecutor::new_with_rules_and_functions( + self.inner.fact_storage.clone(), + self.inner.rules.clone(), + self.inner.functions.clone(), + ); + executor.set_limits( + self.inner.options.max_derived_facts, + self.inner.options.max_results, + ); + executor + } + + /// Run a query and return a [`Cursor`] over its rows. + /// + /// The cursor's answer is fixed when this returns: writes that commit + /// while it is open do not change it. The cursor owns what it needs, so it + /// can outlive this handle and move to another thread. While the engine + /// computes the whole answer up front, `max_results` and + /// `max_derived_facts` apply exactly as they do to [`Minigraf::execute`]. + /// + /// # Errors + /// + /// Returns an error if: + /// - A `WriteTransaction` is active on **this thread** (`INT-001`). + /// - Parsing fails. + /// - The command is not a `(query ...)` (`API-012`), or it has bind slots + /// (`API-010`; use [`Minigraf::prepare`] and [`PreparedQuery::query`]). + /// - Execution fails. + /// + /// [`PreparedQuery::query`]: crate::PreparedQuery::query + pub fn query(&self, input: &str) -> Result { + self.query_inner(input).map_err(MinigrafError::from) + } + + fn query_inner(&self, input: &str) -> Result { + if is_write_tx_active() { + bail_coded!(ErrorCode::Int001); + } + let cmd = parse_datalog_command(input)?; + match &cmd { + DatalogCommand::Query(q) => reject_unbound_slots(q)?, + DatalogCommand::Transact(_) => bail_coded!(ErrorCode::Api012, "transact"), + DatalogCommand::Retract(_) => bail_coded!(ErrorCode::Api012, "retract"), + DatalogCommand::Rule(_) => bail_coded!(ErrorCode::Api012, "rule"), } + Cursor::from_result(self.read_executor().execute(cmd)?) } // ── Explicit transaction ────────────────────────────────────────────────── diff --git a/src/error.rs b/src/error.rs index 457bf4163c..eda4828c3f 100644 --- a/src/error.rs +++ b/src/error.rs @@ -199,6 +199,7 @@ pub(crate) enum ErrorCode { Api009, Api010, Api011, + Api012, Int001, Int002, Int003, @@ -1143,6 +1144,12 @@ pub(crate) const REGISTRY: &[(ErrorCode, &str, &str, ErrorCategory)] = &[ "one transaction asserts the same value of {} with two valid-time windows; assert it once with its final window", ErrorCategory::Api, ), + ( + ErrorCode::Api012, + "API-012", + "only (query ...) commands can be opened as a cursor; got {}", + ErrorCategory::Api, + ), ( ErrorCode::Int001, "INT-001", @@ -1892,6 +1899,7 @@ mod tests { | ErrorCode::Api009 | ErrorCode::Api010 | ErrorCode::Api011 + | ErrorCode::Api012 | ErrorCode::Int001 | ErrorCode::Int002 | ErrorCode::Int003 @@ -2098,6 +2106,7 @@ mod tests { ErrorCode::Api009, ErrorCode::Api010, ErrorCode::Api011, + ErrorCode::Api012, ErrorCode::Int001, ErrorCode::Int002, ErrorCode::Int003, diff --git a/src/lib.rs b/src/lib.rs index 2d733257f5..70f51255e4 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -74,6 +74,8 @@ //! - **Browser** (`wasm32-unknown-unknown` + `browser` feature) — `wasm-pack build --target web --features browser` //! - **WASI / server-side** (`wasm32-wasip1`) — `cargo build --target wasm32-wasip1 --release --bin minigraf` +/// Incremental delivery of query results: [`Cursor`] and [`Batch`]. +pub mod cursor; pub mod db; pub mod error; pub(crate) mod graph; @@ -97,7 +99,8 @@ pub use repl::Repl; // EAV value types — users construct and match on these pub use graph::types::{EntityId, Value}; -// Query result type +// Query result types +pub use cursor::{Batch, Cursor}; pub use query::datalog::executor::QueryResult; // Bi-temporal query types diff --git a/src/query/datalog/prepared.rs b/src/query/datalog/prepared.rs index 3be9c4b11d..6f0c29b69b 100644 --- a/src/query/datalog/prepared.rs +++ b/src/query/datalog/prepared.rs @@ -1,3 +1,4 @@ +use crate::cursor::Cursor; use crate::error::{ErrorCode, MinigrafError, bail_coded}; use crate::graph::FactStorage; use crate::graph::types::Value; @@ -103,6 +104,19 @@ impl PreparedQuery { self.execute_inner(bindings).map_err(MinigrafError::from) } + /// Substitute bind values and return a [`Cursor`] over the rows. + /// + /// The cursor's answer is fixed when this returns; see + /// [`crate::Minigraf::query`]. + /// + /// # Errors + /// The same as [`PreparedQuery::execute`]. + pub fn query(&self, bindings: &[(&str, BindValue)]) -> Result { + self.execute_inner(bindings) + .and_then(Cursor::from_result) + .map_err(MinigrafError::from) + } + fn execute_inner(&self, bindings: &[(&str, BindValue)]) -> Result { let binding_map: HashMap<&str, &BindValue> = bindings.iter().map(|(name, val)| (*name, val)).collect(); diff --git a/tests/cursor_test.rs b/tests/cursor_test.rs new file mode 100644 index 0000000000..dcd0661402 --- /dev/null +++ b/tests/cursor_test.rs @@ -0,0 +1,279 @@ +//! `Minigraf::query` and `PreparedQuery::query` return a `Cursor` that delivers +//! the answer in batches (#432). These tests pin the public contract the +//! streaming engine must keep: batches partition the `execute()` answer, the +//! answer is fixed when the cursor opens, and non-queries are rejected. + +use minigraf::{BindValue, Cursor, Minigraf, OpenOptions, QueryResult, Value}; + +fn seeded(n: i64) -> Minigraf { + let db = Minigraf::in_memory().unwrap(); + let facts: String = (0..n) + .map(|i| format!("[:p{i} :person/age {i}] [:p{i} :person/name \"n{i}\"]")) + .collect::>() + .join(" "); + db.execute(&format!("(transact [{facts}])")).unwrap(); + db +} + +fn execute_rows(db: &Minigraf, q: &str) -> (Vec, Vec>) { + match db.execute(q).unwrap() { + QueryResult::QueryResults { vars, results } => (vars, results), + _ => panic!("expected query results"), + } +} + +/// Rows in a stable order, so answers can be compared as multisets. +fn sorted(mut rows: Vec>) -> Vec> { + rows.sort_by_key(|r| format!("{r:?}")); + rows +} + +fn drain(cursor: &mut Cursor, max_rows: usize) -> Vec> { + let mut rows = Vec::new(); + while let Some(batch) = cursor.next_batch(max_rows).unwrap() { + assert!(!batch.is_empty(), "a batch is never empty"); + assert!(batch.len() <= max_rows.max(1), "batch exceeds max_rows"); + assert_eq!(batch.len(), batch.rows().len()); + rows.extend(batch.into_rows()); + } + rows +} + +const AGES: &str = "(query [:find ?e ?age :where [?e :person/age ?age]])"; + +#[test] +fn batches_partition_the_execute_answer() { + let db = seeded(25); + let (vars, expected) = execute_rows(&db, AGES); + for max_rows in [1, 7, 25, 1000] { + let mut cursor = db.query(AGES).unwrap(); + assert_eq!(cursor.vars(), vars.as_slice()); + let rows = drain(&mut cursor, max_rows); + assert_eq!(rows.len(), 25, "every row delivered once"); + assert_eq!(sorted(rows), sorted(expected.clone())); + // The end stays the end. + assert!(cursor.next_batch(max_rows).unwrap().is_none()); + } +} + +#[test] +fn batch_sizes_are_full_until_the_last() { + let db = seeded(25); + let mut cursor = db.query(AGES).unwrap(); + let mut sizes = Vec::new(); + while let Some(batch) = cursor.next_batch(10).unwrap() { + sizes.push(batch.len()); + } + assert_eq!(sizes, vec![10, 10, 5]); +} + +#[test] +fn zero_max_rows_still_makes_progress() { + let db = seeded(3); + let mut cursor = db.query(AGES).unwrap(); + let rows = drain(&mut cursor, 0); + assert_eq!(rows.len(), 3); +} + +#[test] +fn iterator_yields_the_execute_rows() { + let db = seeded(12); + let (_, expected) = execute_rows(&db, AGES); + let rows: Vec> = db.query(AGES).unwrap().collect::>().unwrap(); + assert_eq!(sorted(rows), sorted(expected)); +} + +#[test] +fn batch_into_iterator_yields_rows() { + let db = seeded(4); + let batch = db.query(AGES).unwrap().next_batch(10).unwrap().unwrap(); + let n = batch.into_iter().filter(|row| row.len() == 2).count(); + assert_eq!(n, 4); +} + +#[test] +fn empty_answer_has_vars_and_no_batch() { + let db = seeded(3); + let mut cursor = db + .query("(query [:find ?e :where [?e :person/missing ?x]])") + .unwrap(); + assert_eq!(cursor.vars(), ["?e".to_string()].as_slice()); + assert!(cursor.next_batch(10).unwrap().is_none()); + assert!(cursor.next().is_none()); +} + +#[test] +fn aggregates_rules_and_as_of_match_execute() { + let db = seeded(10); + db.execute("(transact [[:p1 :person/friend :p2] [:p2 :person/friend :p3]])") + .unwrap(); + db.execute("(rule [(reach ?a ?b) [?a :person/friend ?b]])") + .unwrap(); + db.execute("(rule [(reach ?a ?b) [?a :person/friend ?m] (reach ?m ?b)])") + .unwrap(); + let queries = [ + "(query [:find (count ?e) (max ?age) :where [?e :person/age ?age]])", + "(query [:find ?a ?b :where (reach ?a ?b)])", + "(query [:find ?e :as-of 1 :where [?e :person/friend ?f]])", + "(query [:find ?e :any-valid-time :where [?e :person/age ?age]])", + ]; + for q in queries { + let (vars, expected) = execute_rows(&db, q); + let mut cursor = db.query(q).unwrap(); + assert_eq!(cursor.vars(), vars.as_slice()); + assert_eq!(sorted(drain(&mut cursor, 2)), sorted(expected)); + } +} + +#[test] +fn cursor_answer_is_fixed_at_open() { + let db = seeded(5); + let mut cursor = db.query(AGES).unwrap(); + let first = cursor.next_batch(2).unwrap().unwrap(); + assert_eq!(first.len(), 2); + + db.execute("(transact [[:late :person/age 99]])").unwrap(); + let mut tx = db.begin_write().unwrap(); + tx.execute("(transact [[:later :person/age 100]])").unwrap(); + tx.commit().unwrap(); + db.execute("(retract [[:p0 :person/age 0]])").unwrap(); + + let rest = drain(&mut cursor, 2); + assert_eq!( + first.len() + rest.len(), + 5, + "pre-transaction answer to the end" + ); + let ages: Vec = first + .into_rows() + .into_iter() + .chain(rest) + .map(|r| r[1].clone()) + .collect(); + assert!(!ages.contains(&Value::Integer(99)), "later write not seen"); + assert!( + !ages.contains(&Value::Integer(100)), + "later commit not seen" + ); + assert!(ages.contains(&Value::Integer(0)), "later retract not seen"); +} + +#[test] +fn file_backed_cursor_answer_is_fixed_at_open() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("cursor.graph"); + let db = Minigraf::open(&path).unwrap(); + db.execute("(transact [[:a :n 1] [:b :n 2] [:c :n 3]])") + .unwrap(); + db.checkpoint().unwrap(); + db.execute("(transact [[:d :n 4]])").unwrap(); + + let mut cursor = db.query("(query [:find ?v :where [?e :n ?v]])").unwrap(); + db.execute("(transact [[:e :n 5]])").unwrap(); + db.checkpoint().unwrap(); + let rows = drain(&mut cursor, 1); + assert_eq!( + rows.len(), + 4, + "committed and WAL facts at open, nothing later" + ); +} + +#[test] +fn non_queries_are_api_012() { + let db = seeded(1); + for (cmd, kind) in [ + ("(transact [[:x :person/age 1]])", "transact"), + ("(retract [[:p0 :person/age 0]])", "retract"), + ("(rule [(adult ?e) [?e :person/age ?a]])", "rule"), + ] { + let err = db.query(cmd).unwrap_err(); + assert_eq!(err.code(), "API-012"); + assert!(err.to_string().contains(kind), "message names the command"); + } + // Nothing was written or registered. + let (_, rows) = execute_rows(&db, AGES); + assert_eq!(rows.len(), 1); +} + +#[test] +fn bind_slots_are_api_010() { + let db = seeded(1); + let err = db + .query("(query [:find ?e :where [?e :person/age $age]])") + .unwrap_err(); + assert_eq!(err.code(), "API-010"); +} + +#[test] +fn parse_errors_keep_their_code() { + let db = seeded(1); + let err = db.query("(query [:find ?e :where").unwrap_err(); + assert!(err.code().starts_with("PRS-"), "parse error code"); +} + +#[test] +fn query_on_a_write_transaction_thread_is_int_001() { + let db = seeded(1); + let tx = db.begin_write().unwrap(); + let err = db.query(AGES).unwrap_err(); + assert_eq!(err.code(), "INT-001"); + tx.rollback(); +} + +#[test] +fn rule_limits_apply_while_the_engine_materialises() { + let db = OpenOptions::new().max_results(3).open_memory().unwrap(); + db.execute("(transact [[:a :next :b] [:b :next :c] [:c :next :d] [:d :next :e]])") + .unwrap(); + db.execute("(rule [(reach ?x ?y) [?x :next ?y]])").unwrap(); + db.execute("(rule [(reach ?x ?y) [?x :next ?m] (reach ?m ?y)])") + .unwrap(); + let q = "(query [:find ?x ?y :where (reach ?x ?y)])"; + assert_eq!(db.execute(q).unwrap_err().code(), "INT-020"); + assert_eq!(db.query(q).unwrap_err().code(), "INT-020"); +} + +#[test] +fn prepared_query_cursor_matches_prepared_execute() { + let db = seeded(10); + let pq = db + .prepare("(query [:find ?name :where [?e :person/age $age] [?e :person/name ?name]])") + .unwrap(); + for age in [0, 4, 9, 42] { + let binds = [("age", BindValue::Val(Value::Integer(age)))]; + let expected = match pq.execute(&binds).unwrap() { + QueryResult::QueryResults { results, .. } => results, + _ => panic!("expected query results"), + }; + let mut cursor = pq.query(&binds).unwrap(); + assert_eq!(cursor.vars(), ["?name".to_string()].as_slice()); + assert_eq!(drain(&mut cursor, 1), expected); + } + let err = pq.query(&[]).unwrap_err(); + assert_eq!(err.code(), "INT-033", "missing bind value"); +} + +#[test] +fn cursor_is_send_and_outlives_the_database_handle() { + fn assert_send() {} + assert_send::(); + + let db = seeded(6); + let cursor = db.query(AGES).unwrap(); + drop(db); + let n = std::thread::spawn(move || cursor.map(Result::unwrap).count()) + .join() + .unwrap(); + assert_eq!(n, 6); +} + +#[test] +fn close_after_first_batch() { + let db = seeded(10); + let mut cursor = db.query(AGES).unwrap(); + assert!(cursor.next_batch(1).unwrap().is_some()); + cursor.close(); + // The database is unaffected. + assert_eq!(execute_rows(&db, AGES).1.len(), 10); +}