From 97cd4a2a85301ea3478250dd654139d45a1f7cd0 Mon Sep 17 00:00:00 2001 From: Aditya Mukhopadhyay Date: Wed, 7 Oct 2026 10:55:37 +0530 Subject: [PATCH 1/3] =?UTF-8?q?feat:=20binding=20support=20=E2=80=94=20API?= =?UTF-8?q?-017/API-018=20constructors=20and=20BrowserDb=20cursor=20(#462,?= =?UTF-8?q?=20#465,=20#467)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Language bindings can now report what Rust's types rule out with the core's own codes: MinigrafError::invalid_argument (API-017) for an argument a binding cannot convert, and MinigrafError::closed (API-018) for a log writer used after finish or close. BrowserDb.query returns a BrowserCursor (vars, nextBatch with JSON rows, close), matching Minigraf::query (API-012, API-010). The wasm32 test build compiles again: drop a duplicate page_cache_capacity and gate the tempfile-based dir_sync tests off wasm32. The binding shape shared by every binding repo is specified in docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md. Refs #462, #465, #467 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn --- CHANGELOG.md | 1 + CLAUDE.md | 6 +- README.md | 2 +- docs/ERROR_REFERENCE.md | 26 +++ docs/TEST_COVERAGE.md | 4 +- .../2026-10-07-binding-etl-surface-design.md | 113 ++++++++++++ src/browser/mod.rs | 169 ++++++++++++++++++ src/error.rs | 48 +++++ src/storage/dir_sync.rs | 3 +- src/storage/persistent_facts.rs | 11 -- 10 files changed, 365 insertions(+), 18 deletions(-) create mode 100644 docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 624d861bf..61b002a5c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Binding support for cursors, open options, the fact log and the log writer** (#462, #465, #467). Two new codes let language bindings report what Rust's types rule out: `API-017` (an argument the binding cannot convert, such as an entity id that is not a UUID string), from `MinigrafError::invalid_argument`, and `API-018` (a log writer used after `finish` or `close`), from `MinigrafError::closed`. `BrowserDb.query(datalog)` returns a `BrowserCursor` with `vars()`, `nextBatch(maxRows)` (rows as JSON, as in `execute()`, or `undefined` at the end) and `close()`. The shared binding shape is in `docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md`. The wasm32 test build compiles again (a duplicate `page_cache_capacity` and `tempfile` tests on wasm32). - **Fact log: `Minigraf::fact_log()`** (#430). Streams every fact version in the database as `FactRecord`s (entity, attribute, value, `tx_count`, `tx_id`, valid-time bounds, and whether it is an assertion or a retraction), without Datalog. `FactFilter` selects attributes (exact or by prefix), entities and a `tx_count` range, checked on index keys so excluded records are never decoded. `FactOrder::Tx` (the default) returns records in ascending `tx_count`; format v8 has no index in transaction order, so it holds at most `FactFilter::window` records (default 262,144) and scans the selected keys once per window. `FactOrder::Storage` reads in one pass. The log reads the database as it was when it opened; while one is open, checkpoints are deferred (writes stay in the WAL, automatic checkpoints wait, and `checkpoint()` or `rebuild_indexes()` with work to do fail with the new `API-013`). Close, drop or finish the log to release it. Like `Cursor`, a `FactLog` is owned and `Send`. Rust API only for now; the bindings follow. - **Raw log writer: `LogWriter`** (#431). Builds a new database from explicit `FactRecord`s, the load step of an offline migration after `fact_log()` on a read-only source. Each record keeps its `tx_count`, `tx_id`, valid-time window and assertion flag. Records come in transaction order, gaps allowed: leaving a transaction out leaves a hole, and `:as-of` on it shows the state before. `advance_tx_count` keeps the source's counter after trailing purged transactions. The checks are structural only: order (new `API-015`), one `tx_id` per transaction (new `API-016`), one valid-time window per fact per transaction (`API-011`) and size (`WAL-003`); a rejected record changes nothing. There is no WAL: records are committed in batches with the normal copy-on-write checkpoint, never splitting a transaction. The file is built at `.partial` and renamed into place by `finish`, so a crash or an abandoned build never leaves part of a log at `path`; an existing target or WAL is the new `STG-043`. The page-range copy the issue proposed is not provided, since v8 pages share one dictionary and are not self-contained. The bindings follow. - **Read-only open: `OpenOptions::read_only`** (#429). Opens a database so that nothing done through the handle writes to the `.graph` file or its WAL. The file is locked in shared mode: any number of read-only handles, in this process or others, can hold it at once, and they exclude a read-write open (and it excludes them) with the existing STG-025/STG-026. The handle shows the same data as a read-write open: a WAL left by an earlier session is applied in memory but not checkpointed or deleted, and a format v7 file is read into memory instead of migrated. A missing file is the new `STG-042` (nothing is created). `transact`, `retract`, `begin_write`, `checkpoint` and `rebuild_indexes` fail with the new `API-014`; queries, cursors, `fact_log`, `verify` and rule registration work, and dropping the handle does not checkpoint. `page_cache_size` applies as usual, so a tool can keep the whole file resident. Rust API only for now; the bindings follow. diff --git a/CLAUDE.md b/CLAUDE.md index a3cd5f26c..4bd63d651 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -145,9 +145,9 @@ cargo run < demos/demo_negation.txt 7. **`src/wal.rs`** — Fact-level sidecar WAL, CRC32-protected entries, crash recovery -8. **`src/error.rs`** — Structured error codes: `MinigrafError`, `ErrorCategory`, `ErrorCode` registry (PRS/QRY/STG/WAL/API/INT codes); must match `docs/ERROR_REFERENCE.md` +8. **`src/error.rs`** — Structured error codes: `MinigrafError`, `ErrorCategory`, `ErrorCode` registry (PRS/QRY/STG/WAL/API/INT codes); must match `docs/ERROR_REFERENCE.md`. `MinigrafError::invalid_argument` (API-017) and `MinigrafError::closed` (API-018) exist for language bindings -9. **`src/browser/`** — Browser WASM backend (`browser` feature): `buffer.rs` (`BrowserBufferBackend`, in-memory pages with dirty tracking), `indexeddb.rs` (IndexedDB persistence) +9. **`src/browser/`** — Browser WASM backend (`browser` feature): `buffer.rs` (`BrowserBufferBackend`, in-memory pages with dirty tracking), `indexeddb.rs` (IndexedDB persistence); `BrowserDb.query` returns a `BrowserCursor` (JSON row batches) ### Data Model @@ -197,7 +197,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are ## Test Coverage -**1345 tests** (1336 passing, 9 ignored; unit + integration + doc). +**1346 tests** (1337 passing, 9 ignored; unit + integration + doc). See `docs/TEST_COVERAGE.md` for the full per-file breakdown. **Testing conventions** — see the Testing Conventions section below before writing any tests. diff --git a/README.md b/README.md index 769caf354..33509c5e8 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit ```bash cargo run # interactive Datalog REPL -cargo test # run 1345 tests +cargo test # run 1346 tests cargo run < demos/demo_recursive.txt # recursive rules demo ``` diff --git a/docs/ERROR_REFERENCE.md b/docs/ERROR_REFERENCE.md index 9c74c0d38..5d2d64e71 100644 --- a/docs/ERROR_REFERENCE.md +++ b/docs/ERROR_REFERENCE.md @@ -174,6 +174,8 @@ with no `CodedError` anywhere in its chain. | API-014 | Write on a read-only handle | Database API | | API-015 | Log writer record out of transaction order | Database API | | API-016 | Log writer record with a second tx_id for one transaction | Database API | +| API-017 | Invalid argument passed through a language binding | Database API | +| API-018 | Call on a closed binding object | Database API | | INT-000 | Unclassified internal error | Internal | | INT-001 | WriteTransaction already in progress on this thread | Internal | | INT-002 | Invalid entity (API layer) | Internal | @@ -2578,6 +2580,30 @@ let mut log = src.fact_log(&FactFilter::new())?; **Scenario**: A transform sets `tx_id` from a per-record field instead of from the transaction. +### API-017 Invalid argument passed through a language binding + +**Error text**: `invalid argument: {}` + +**Cause**: A language binding (Python, Java, Swift, Node.js, C and the others) received an argument it could not convert to the Rust type, such as an entity id or `Ref` value that is not a UUID string, or a negative or out-of-range number. The `{}` names the argument and the problem. The Rust API takes typed arguments and never returns this code. Nothing was changed. + +**Resolution**: +- Pass entity ids and `Ref` values as the UUID strings that `fact_log` records and query results carry. +- Check the binding's documentation for the argument's type and range. + +**Scenario**: A migration script builds a `FactRecord` with `entity="alice"` instead of the entity's UUID. + +### API-018 Call on a closed binding object + +**Error text**: `{} is closed` + +**Cause**: A language binding object was used after it was closed or finished, for example `append` or `finish` on a log writer after `finish` or `close`. Rust consumes these objects, so the Rust API never returns this code. Bindings cannot consume an object a caller still holds, so they return this error instead. A cursor or fact log reports the end (no more batches) after `close`, not this error. The `{}` names the object. + +**Resolution**: +- Create a new object instead of reusing a finished one. +- In Python, use the writer as a context manager so it is finished once, on exit. + +**Scenario**: A loader calls `finish()` in a `finally` block after it already called it on success. + --- ## INT — Internal Errors diff --git a/docs/TEST_COVERAGE.md b/docs/TEST_COVERAGE.md index 398537a5a..6bfbd330c 100644 --- a/docs/TEST_COVERAGE.md +++ b/docs/TEST_COVERAGE.md @@ -4,9 +4,9 @@ **Verified**: 2026-10-07 with `cargo test --quiet` -**Result**: 1,336 passing tests and 9 ignored tests (1,345 total) +**Result**: 1,337 passing tests and 9 ignored tests (1,346 total) -This covers the unreleased file-format-v8 work (#287, #371): the index-key value-collision fix, the v7→v8 migration, and the drop of formats v1–v6. It otherwise still covers the public database API, the Datalog engine, storage and recovery paths, v2.0.0's locking and structured-error-code changes, and v2.0.1's regression test for rebuilding indexes on open after several checkpoints (#370). Merged from the v2.x line: `tests/point_query_history_test.rs` and new storage unit tests check that attribute-narrowed bound-entity lookups return exactly what a full scan returns, and that `net_asserted_facts` matches its previous implementation on randomized histories (#323). Attribute-scan tests check that the AEVT range covers exactly the queried attribute, including non-ASCII names such as `:丿` and prefix siblings such as `:ab` for `:a` (#381). Incremental B+tree rebuild tests check that checkpoints which copy untouched index leaves produce exactly the entries and file checksum a full rebuild would, including over repeated checkpoints, that single-fact checkpoints do not fragment the index, and that a damaged slot directory is rejected rather than copied (#315). B+tree cursor tests check that every scan walks the tree, not the leaf `next_leaf` chain: full scans and start positions match the input, `seek` from the current position lands where a fresh descent from the root would (randomized, plus within a leaf, to a sibling, across subtrees, past the end, into the gap after a leaf, and backwards as a no-op), a seek to the sibling leaf reads exactly one page, reads are unchanged with every `next_leaf` zeroed, a child pointer into a non-B+tree page is STG-013, and a cycle in child pointers is an error, not a hang (v8 storage format PR 1, #374). Directory-sync unit tests check that creating the `.graph` file or the WAL, and deleting the WAL, fsync the parent directory after the file operation, and that reopening an existing file does not (#389). Disjunction tests check that `or-join` returns an empty result, not INT-031, when no rows reach it: on an empty database, after a pattern that matches nothing, after a `not` that removes every row, and in a rule body (#405). A stratification unit test builds the same two-rule negative cycle 64 times and checks that the INT-054 message names the same predicate pair every time (#410). Magic-sets tests check that a recursive rule with a literal start that recurses through another rule returns the same results as without the literal (#297). API tests check that `execute()` and `WriteTransaction::execute()` reject `$slot` bind tokens in every position with API-010 (#407). The REPL `run()` test now runs in a child process with null stdin, so it no longer hangs when the test runner's stdin is an open non-TTY stream (#415). v8 page-format tests (PR 2, #374, #388) check the 24-byte page header (a flipped bit, a wrong page id and a future generation each give their own STG code, legacy page types fail before the CRC, and a failed page is never cached), meta-page encoding, the free-list chain, WAL v2's base generation, and every row of the meta-selection table: both slots valid, a torn newest slot with the WAL's base generation (opens at the older one), a WAL newer than the valid meta, pages of the next generation on the free list or past `page_count` with no WAL (all STG-033, with the file unmodified), an unknown feature bit (STG-034), and a pre-release v8 single header (STG-032). Fault-injection tests crash `save()` at every page write (cleanly and with torn pages, including the meta write) and every sync, and crash a v7 migration at every write; each reopen finds exactly the old or the new checkpoint, and every reachable page plus the free list covers the file exactly. Twenty-five consecutive saves never write a page the previous meta references. Covering-key tests (PR 3, #433) check that byte order equals logical order for every key component and value type on random tuples (negative, zero and large integers, NaN and ±0.0, strings with 0x00, 64- and 65-byte strings, FOREVER, newest-transaction-first) and that every component round-trips; that prefix-compressed leaves and shortest-separator internal nodes round-trip, agree with a linear scan, keep `max(left) < s ≤ min(right)`, and reject an overflow; that the byte-key B+tree's builds, incremental rebuilds, cursor and `seek` match their expected sets; that dictionary ids are assigned once, never reused, shared between a keyword attribute and value, and deterministic; that a re-asserted or retracted long value writes no new value page and a forced hash collision still finds the right value; that one transaction with two timestamps is STG-037 and a missing dictionary entry STG-036; that committed reads by entity, entity and attribute, attribute, and in full equal an in-memory model across eight checkpoints and a reopen; that short-valued reads touch no value page; that a v7 file with every value type, long strings, refs and retractions migrates to exactly the same facts; that a pre-covering-key v8 file is STG-032; and that string values over 4,068 bytes and attributes or keywords over 1,024 bytes are WAL-003. A WAL test checks that reopening next to a fully checkpointed WAL does not reuse transaction numbers (#447). Copy-on-write checkpoint tests (PR 4, #434) check that lazy free-list pops and pushes conserve every page across page boundaries, partial head pages and chain exhaustion, never hand out a page freed in the same generation, and share unread chain pages unchanged; that a copy-on-write insert equals a sorted merge for random batches (including keys below the first and above the last, many into one leaf, root growth to depth 3, and 1 KB separators that split internal nodes), leaves the old tree readable and unchanged, frees exactly the replaced nodes, writes only new pages, writes exactly one leaf and two internal nodes for one entry in a depth-3 tree, and does not fragment leaves under single inserts; that a checkpoint after 1 or 100 right-edge facts writes at most the tree-depth difference more pages at 100,000 facts than at 10,000, and at most one path per new entry per tree for random keys; and that four reader threads querying throughout 40 checkpoints never fail and never see the fact count go backwards. Hardening tests (PR 5) damage an index leaf, an internal node, a dictionary leaf, a value page and a free-list page in turn. Every committed read or checkpoint that reaches the damaged page fails with STG-029, never with wrong or fewer rows. A query through `Minigraf` on a file with damaged leaves fails with STG-029, and the file still opens at its last checkpoint. A size test measures at most 200 bytes per fact (142 measured) on 144k facts in #433's shape; its 1.15M-fact variant is ignored locally and runs in the nightly smoke workflow. A shared dictionary-cache test checks that cached ident and timestamp lookups return the same facts and that the cache stays bounded. Merged from the v2.x line: storage and `edge_cases_test.rs` tests check that more than 65,535 uncheckpointed facts each resolve to their own fact through entity lookups and attribute scans, in memory and file-backed (#445). Merged from the v2.x line: `tests/golden_corpus_test.rs` opens a copy of each committed golden file in `tests/golden/` (v7 files written by minigraf 2.0.3: one checkpoint, several checkpoints, indexes rebuilt on open, a crash-left WAL, an already-checkpointed WAL, same-transaction multi-values) and checks its manifest's CRC32, `tx_count` and query results after open, after a checkpoint and reopen, and after one more write; a second test checks that every golden file has a manifest (#391). On `v3` the same test checks that each v7 file has a v8 meta page after its first open and gives the same results after reopening the migrated file, and adds seven v8 golden files (one checkpoint, seven copy-on-write checkpoints, a crash-left v2 WAL, an already-checkpointed v2 WAL, same-transaction multi-values, a v7 file migrated in place, and long values at the size limits); the #371 queries, skipped on v2.x, run and pass. Integrity tests (#373) check that intact files verify with no problems and an exact fact and page count, and that `verify` reports each kind of damage with its code: an EAVT, AEVT, AVET or VAET leaf missing an entry (STG-038), keys out of order and a page reached from two trees (STG-039), a rotted index leaf (STG-029), a free page still in use and a leaked page (STG-035), a missing transaction entry (STG-036), and an entity id whose UUID maps to another id (STG-040). After each index or free-list damage, `rebuild_indexes` leaves a file that verifies clean, reads back every fact and accounts for every page. The source is an index that agrees with another one, or the only intact one; three intact indexes that all disagree, or a damaged dictionary, give STG-041 with the file unchanged. A rebuild commits pending facts, including new ids and long values. A crash at every write (clean and torn) and every sync of a rebuild reopens either at the old generation with only its old damage or at the new one with none, never losing a fact. Through `Minigraf` (`tests/integrity_test.rs`), a rebuild commits WAL entries and deletes the WAL, and flipping a byte in each leaf of a file in turn is always reported, then either repaired (index leaves) or refused with STG-041 and the file unchanged (dictionary leaves). Live-read tests (#379) check that `triple_len` ends exactly where `tx↓` begins for every value type and index; that on random histories (several valid-time windows, retractions, re-assertions, several values per attribute, long values, 12 transactions per checkpoint) the committed live read for every scan kind and `:as-of` point is exactly the net-asserted set, that it stays equivalent with pending facts on top, and after reopen; and that a point read on a triple asserted and retracted 2,000 times reads under a third of the pages a history read does, with or without `:as-of`. `tests/live_reads_test.rs` runs the same history in a checkpointed file database and in memory and checks that point, attribute, full-scan, join, valid-time, `:as-of` and rule queries return the same rows, before and after reopen. One-window tests (#435) check that `net_asserted_facts` keeps each triple's newest transaction only (a later window replaces an earlier one, closing an open window, a retraction in the newest transaction hides it, records with two windows in one transaction keep both, once each) against a per-triple model on random histories; that the on-disk key walk agrees at every `:as-of` before and after reopen; and that `API-011` rejects two windows of one fact in one transaction without taking a `tx_count`. `tests/valid_time_window_test.rs` checks replacing, closing, extending, reopening and retract-then-reassert through entity-bound and attribute queries in memory, on a checkpointed file and after reopen, and `API-011` from `transact` and from `WriteTransaction::commit`. Cursor tests (#432, `tests/cursor_test.rs`) check that `query()` batches partition the `execute()` answer for several batch sizes (never empty, never over `max_rows`, `max_rows == 0` still progresses, `None` stays `None`), that the iterator and aggregates, rules, `:as-of` and `:any-valid-time` match `execute()`, that a cursor opened before a `transact`, a `WriteTransaction` commit, a retract or a checkpoint returns the pre-write answer to the end (in memory and file-backed), that `transact`/`retract`/`rule` are `API-012` and bind slots `API-010`, that a `WriteTransaction` thread gets `INT-001`, that rule limits give `INT-020` as from `execute()`, that `PreparedQuery::query` matches `PreparedQuery::execute`, and that a cursor is `Send` and outlives its database handle. Fact-log tests (#430, `tests/fact_log_test.rs`) check that `fact_log()` returns exactly the records written (assertions and retractions, with `tx_count` and `tx_id`) in memory, with everything pending, half checkpointed, fully checkpointed and after reopen, in both orders, with windows of 1 to 1,000 records and batch sizes of 1 to 1,000, in non-decreasing `tx_count` in Tx order; that every layout gives identical records, including valid-time windows and long string values; that a transaction larger than the window is split with no loss or duplicate; that attribute, attribute-prefix, entity and tx-range filters and their combinations match a model, and unknown names match nothing; that writes after open are not in the log; that `checkpoint()` and `rebuild_indexes()` are `API-013` while a log is open (a checkpoint with nothing to write is allowed), that automatic checkpoints from `execute()` and `WriteTransaction::commit` wait for the log, and that closing, dropping or finishing the log releases the database; that a `WriteTransaction` thread gets `INT-001`; and that a log is `Send`, outlives its database handle, and the deferred writes replay on the next open. Read-only open tests (#429, `tests/read_only_open_test.rs`) check that a read-only handle on a file with checkpointed facts and a pending WAL returns the same query results, `tx_count` and fact log as a read-write open; that queries, a cursor, a fact log, `verify`, `prepare`, rules and a UDF leave the `.graph` and WAL bytes and the `.graph` mtime unchanged, and no WAL is created; that `transact`, `retract`, `begin_write`, `checkpoint` and `rebuild_indexes` are `API-014` without taking a `tx_count`, also on a read-only in-memory database and through the builder path; that a missing file is `STG-042` and nothing is created; that an empty file is an empty database and stays 0 bytes; that read-only handles share a file while a read-write one is refused with `STG-025`, both ways; that a file without write permission opens read-only; and that small and whole-file page caches work. File-backend unit tests check that a shared-mode backend refuses page writes and syncs with `INT-056`, that the in-process registry counts readers and admits readers or one writer (also without kernel locks), and that a reader in another process shares the file with a reader here and refuses a writer here with `STG-026`. Storage unit tests check that a read-only open of a v7 file at every point of an interrupted migration (clean and torn writes) sees every fact and writes nothing, and that an empty backend stays empty. `tests/golden_corpus_test.rs` opens a copy of every golden file read-only first and checks its `tx_count` and queries, and that neither the file nor its WAL changes. Log-writer tests (#431, `tests/log_writer_test.rs`) copy a source's fact log (assertions, retractions, an explicit valid-time window, long strings, refs, keywords, checkpointed and pending facts) through `LogWriter` and check that the new file has no `.partial` or WAL, verifies clean, and gives the source's `tx_count`, fact log and query results (current, `:valid-at`, `:any-valid-time`, every `:as-of`, and entity-, attribute- and value-bound lookups) read-write, read-only and after reopen (#370); that it takes new writes at the source's next `tx_count`; that 90,000 facts over two batch commits verify clean and count correctly at `:as-of`; that a purged middle transaction is a hole whose `:as-of` equals the one before, and trailing holes keep the counter through `advance_tx_count`; that an empty log is an empty database with the counter; that `tx_count` 0, a lower or closed transaction and a backwards advance are `API-015`, a second `tx_id` `API-016`, a second window `API-011` and oversize values or idents `WAL-003`, each leaving the writer usable and the output exact, with repeats written once; that an existing target or WAL is `STG-043`, a second writer `STG-025` and `read_only` `API-014`; that a stale `.partial` is reused; that dropping an unfinished writer leaves nothing; that `finish` refuses a target created during the build and leaves it untouched; and that `LogWriter` is `Send`. Unit tests check that a batch commits only when a transaction opens past the batch size, never splitting one, and that `FileBackend::truncate` empties a file. +This covers the unreleased file-format-v8 work (#287, #371): the index-key value-collision fix, the v7→v8 migration, and the drop of formats v1–v6. It otherwise still covers the public database API, the Datalog engine, storage and recovery paths, v2.0.0's locking and structured-error-code changes, and v2.0.1's regression test for rebuilding indexes on open after several checkpoints (#370). Merged from the v2.x line: `tests/point_query_history_test.rs` and new storage unit tests check that attribute-narrowed bound-entity lookups return exactly what a full scan returns, and that `net_asserted_facts` matches its previous implementation on randomized histories (#323). Attribute-scan tests check that the AEVT range covers exactly the queried attribute, including non-ASCII names such as `:丿` and prefix siblings such as `:ab` for `:a` (#381). Incremental B+tree rebuild tests check that checkpoints which copy untouched index leaves produce exactly the entries and file checksum a full rebuild would, including over repeated checkpoints, that single-fact checkpoints do not fragment the index, and that a damaged slot directory is rejected rather than copied (#315). B+tree cursor tests check that every scan walks the tree, not the leaf `next_leaf` chain: full scans and start positions match the input, `seek` from the current position lands where a fresh descent from the root would (randomized, plus within a leaf, to a sibling, across subtrees, past the end, into the gap after a leaf, and backwards as a no-op), a seek to the sibling leaf reads exactly one page, reads are unchanged with every `next_leaf` zeroed, a child pointer into a non-B+tree page is STG-013, and a cycle in child pointers is an error, not a hang (v8 storage format PR 1, #374). Directory-sync unit tests check that creating the `.graph` file or the WAL, and deleting the WAL, fsync the parent directory after the file operation, and that reopening an existing file does not (#389). Disjunction tests check that `or-join` returns an empty result, not INT-031, when no rows reach it: on an empty database, after a pattern that matches nothing, after a `not` that removes every row, and in a rule body (#405). A stratification unit test builds the same two-rule negative cycle 64 times and checks that the INT-054 message names the same predicate pair every time (#410). Magic-sets tests check that a recursive rule with a literal start that recurses through another rule returns the same results as without the literal (#297). API tests check that `execute()` and `WriteTransaction::execute()` reject `$slot` bind tokens in every position with API-010 (#407). The REPL `run()` test now runs in a child process with null stdin, so it no longer hangs when the test runner's stdin is an open non-TTY stream (#415). v8 page-format tests (PR 2, #374, #388) check the 24-byte page header (a flipped bit, a wrong page id and a future generation each give their own STG code, legacy page types fail before the CRC, and a failed page is never cached), meta-page encoding, the free-list chain, WAL v2's base generation, and every row of the meta-selection table: both slots valid, a torn newest slot with the WAL's base generation (opens at the older one), a WAL newer than the valid meta, pages of the next generation on the free list or past `page_count` with no WAL (all STG-033, with the file unmodified), an unknown feature bit (STG-034), and a pre-release v8 single header (STG-032). Fault-injection tests crash `save()` at every page write (cleanly and with torn pages, including the meta write) and every sync, and crash a v7 migration at every write; each reopen finds exactly the old or the new checkpoint, and every reachable page plus the free list covers the file exactly. Twenty-five consecutive saves never write a page the previous meta references. Covering-key tests (PR 3, #433) check that byte order equals logical order for every key component and value type on random tuples (negative, zero and large integers, NaN and ±0.0, strings with 0x00, 64- and 65-byte strings, FOREVER, newest-transaction-first) and that every component round-trips; that prefix-compressed leaves and shortest-separator internal nodes round-trip, agree with a linear scan, keep `max(left) < s ≤ min(right)`, and reject an overflow; that the byte-key B+tree's builds, incremental rebuilds, cursor and `seek` match their expected sets; that dictionary ids are assigned once, never reused, shared between a keyword attribute and value, and deterministic; that a re-asserted or retracted long value writes no new value page and a forced hash collision still finds the right value; that one transaction with two timestamps is STG-037 and a missing dictionary entry STG-036; that committed reads by entity, entity and attribute, attribute, and in full equal an in-memory model across eight checkpoints and a reopen; that short-valued reads touch no value page; that a v7 file with every value type, long strings, refs and retractions migrates to exactly the same facts; that a pre-covering-key v8 file is STG-032; and that string values over 4,068 bytes and attributes or keywords over 1,024 bytes are WAL-003. A WAL test checks that reopening next to a fully checkpointed WAL does not reuse transaction numbers (#447). Copy-on-write checkpoint tests (PR 4, #434) check that lazy free-list pops and pushes conserve every page across page boundaries, partial head pages and chain exhaustion, never hand out a page freed in the same generation, and share unread chain pages unchanged; that a copy-on-write insert equals a sorted merge for random batches (including keys below the first and above the last, many into one leaf, root growth to depth 3, and 1 KB separators that split internal nodes), leaves the old tree readable and unchanged, frees exactly the replaced nodes, writes only new pages, writes exactly one leaf and two internal nodes for one entry in a depth-3 tree, and does not fragment leaves under single inserts; that a checkpoint after 1 or 100 right-edge facts writes at most the tree-depth difference more pages at 100,000 facts than at 10,000, and at most one path per new entry per tree for random keys; and that four reader threads querying throughout 40 checkpoints never fail and never see the fact count go backwards. Hardening tests (PR 5) damage an index leaf, an internal node, a dictionary leaf, a value page and a free-list page in turn. Every committed read or checkpoint that reaches the damaged page fails with STG-029, never with wrong or fewer rows. A query through `Minigraf` on a file with damaged leaves fails with STG-029, and the file still opens at its last checkpoint. A size test measures at most 200 bytes per fact (142 measured) on 144k facts in #433's shape; its 1.15M-fact variant is ignored locally and runs in the nightly smoke workflow. A shared dictionary-cache test checks that cached ident and timestamp lookups return the same facts and that the cache stays bounded. Merged from the v2.x line: storage and `edge_cases_test.rs` tests check that more than 65,535 uncheckpointed facts each resolve to their own fact through entity lookups and attribute scans, in memory and file-backed (#445). Merged from the v2.x line: `tests/golden_corpus_test.rs` opens a copy of each committed golden file in `tests/golden/` (v7 files written by minigraf 2.0.3: one checkpoint, several checkpoints, indexes rebuilt on open, a crash-left WAL, an already-checkpointed WAL, same-transaction multi-values) and checks its manifest's CRC32, `tx_count` and query results after open, after a checkpoint and reopen, and after one more write; a second test checks that every golden file has a manifest (#391). On `v3` the same test checks that each v7 file has a v8 meta page after its first open and gives the same results after reopening the migrated file, and adds seven v8 golden files (one checkpoint, seven copy-on-write checkpoints, a crash-left v2 WAL, an already-checkpointed v2 WAL, same-transaction multi-values, a v7 file migrated in place, and long values at the size limits); the #371 queries, skipped on v2.x, run and pass. Integrity tests (#373) check that intact files verify with no problems and an exact fact and page count, and that `verify` reports each kind of damage with its code: an EAVT, AEVT, AVET or VAET leaf missing an entry (STG-038), keys out of order and a page reached from two trees (STG-039), a rotted index leaf (STG-029), a free page still in use and a leaked page (STG-035), a missing transaction entry (STG-036), and an entity id whose UUID maps to another id (STG-040). After each index or free-list damage, `rebuild_indexes` leaves a file that verifies clean, reads back every fact and accounts for every page. The source is an index that agrees with another one, or the only intact one; three intact indexes that all disagree, or a damaged dictionary, give STG-041 with the file unchanged. A rebuild commits pending facts, including new ids and long values. A crash at every write (clean and torn) and every sync of a rebuild reopens either at the old generation with only its old damage or at the new one with none, never losing a fact. Through `Minigraf` (`tests/integrity_test.rs`), a rebuild commits WAL entries and deletes the WAL, and flipping a byte in each leaf of a file in turn is always reported, then either repaired (index leaves) or refused with STG-041 and the file unchanged (dictionary leaves). Live-read tests (#379) check that `triple_len` ends exactly where `tx↓` begins for every value type and index; that on random histories (several valid-time windows, retractions, re-assertions, several values per attribute, long values, 12 transactions per checkpoint) the committed live read for every scan kind and `:as-of` point is exactly the net-asserted set, that it stays equivalent with pending facts on top, and after reopen; and that a point read on a triple asserted and retracted 2,000 times reads under a third of the pages a history read does, with or without `:as-of`. `tests/live_reads_test.rs` runs the same history in a checkpointed file database and in memory and checks that point, attribute, full-scan, join, valid-time, `:as-of` and rule queries return the same rows, before and after reopen. One-window tests (#435) check that `net_asserted_facts` keeps each triple's newest transaction only (a later window replaces an earlier one, closing an open window, a retraction in the newest transaction hides it, records with two windows in one transaction keep both, once each) against a per-triple model on random histories; that the on-disk key walk agrees at every `:as-of` before and after reopen; and that `API-011` rejects two windows of one fact in one transaction without taking a `tx_count`. `tests/valid_time_window_test.rs` checks replacing, closing, extending, reopening and retract-then-reassert through entity-bound and attribute queries in memory, on a checkpointed file and after reopen, and `API-011` from `transact` and from `WriteTransaction::commit`. Cursor tests (#432, `tests/cursor_test.rs`) check that `query()` batches partition the `execute()` answer for several batch sizes (never empty, never over `max_rows`, `max_rows == 0` still progresses, `None` stays `None`), that the iterator and aggregates, rules, `:as-of` and `:any-valid-time` match `execute()`, that a cursor opened before a `transact`, a `WriteTransaction` commit, a retract or a checkpoint returns the pre-write answer to the end (in memory and file-backed), that `transact`/`retract`/`rule` are `API-012` and bind slots `API-010`, that a `WriteTransaction` thread gets `INT-001`, that rule limits give `INT-020` as from `execute()`, that `PreparedQuery::query` matches `PreparedQuery::execute`, and that a cursor is `Send` and outlives its database handle. Fact-log tests (#430, `tests/fact_log_test.rs`) check that `fact_log()` returns exactly the records written (assertions and retractions, with `tx_count` and `tx_id`) in memory, with everything pending, half checkpointed, fully checkpointed and after reopen, in both orders, with windows of 1 to 1,000 records and batch sizes of 1 to 1,000, in non-decreasing `tx_count` in Tx order; that every layout gives identical records, including valid-time windows and long string values; that a transaction larger than the window is split with no loss or duplicate; that attribute, attribute-prefix, entity and tx-range filters and their combinations match a model, and unknown names match nothing; that writes after open are not in the log; that `checkpoint()` and `rebuild_indexes()` are `API-013` while a log is open (a checkpoint with nothing to write is allowed), that automatic checkpoints from `execute()` and `WriteTransaction::commit` wait for the log, and that closing, dropping or finishing the log releases the database; that a `WriteTransaction` thread gets `INT-001`; and that a log is `Send`, outlives its database handle, and the deferred writes replay on the next open. Read-only open tests (#429, `tests/read_only_open_test.rs`) check that a read-only handle on a file with checkpointed facts and a pending WAL returns the same query results, `tx_count` and fact log as a read-write open; that queries, a cursor, a fact log, `verify`, `prepare`, rules and a UDF leave the `.graph` and WAL bytes and the `.graph` mtime unchanged, and no WAL is created; that `transact`, `retract`, `begin_write`, `checkpoint` and `rebuild_indexes` are `API-014` without taking a `tx_count`, also on a read-only in-memory database and through the builder path; that a missing file is `STG-042` and nothing is created; that an empty file is an empty database and stays 0 bytes; that read-only handles share a file while a read-write one is refused with `STG-025`, both ways; that a file without write permission opens read-only; and that small and whole-file page caches work. File-backend unit tests check that a shared-mode backend refuses page writes and syncs with `INT-056`, that the in-process registry counts readers and admits readers or one writer (also without kernel locks), and that a reader in another process shares the file with a reader here and refuses a writer here with `STG-026`. Storage unit tests check that a read-only open of a v7 file at every point of an interrupted migration (clean and torn writes) sees every fact and writes nothing, and that an empty backend stays empty. `tests/golden_corpus_test.rs` opens a copy of every golden file read-only first and checks its `tx_count` and queries, and that neither the file nor its WAL changes. Log-writer tests (#431, `tests/log_writer_test.rs`) copy a source's fact log (assertions, retractions, an explicit valid-time window, long strings, refs, keywords, checkpointed and pending facts) through `LogWriter` and check that the new file has no `.partial` or WAL, verifies clean, and gives the source's `tx_count`, fact log and query results (current, `:valid-at`, `:any-valid-time`, every `:as-of`, and entity-, attribute- and value-bound lookups) read-write, read-only and after reopen (#370); that it takes new writes at the source's next `tx_count`; that 90,000 facts over two batch commits verify clean and count correctly at `:as-of`; that a purged middle transaction is a hole whose `:as-of` equals the one before, and trailing holes keep the counter through `advance_tx_count`; that an empty log is an empty database with the counter; that `tx_count` 0, a lower or closed transaction and a backwards advance are `API-015`, a second `tx_id` `API-016`, a second window `API-011` and oversize values or idents `WAL-003`, each leaving the writer usable and the output exact, with repeats written once; that an existing target or WAL is `STG-043`, a second writer `STG-025` and `read_only` `API-014`; that a stale `.partial` is reused; that dropping an unfinished writer leaves nothing; that `finish` refuses a target created during the build and leaves it untouched; and that `LogWriter` is `Send`. Unit tests check that a batch commits only when a transaction opens past the batch size, never splitting one, and that `FileBackend::truncate` empties a file. A unit test in `error.rs` checks that `MinigrafError::invalid_argument` and `MinigrafError::closed`, the constructors for language bindings, give API-017 and API-018 with their text. Three `BrowserDb` tests (wasm32, run with `wasm-bindgen-test` in a browser, not counted above) check that `query()` batches of 1, 7 and 1,000 give the rows of `execute()`, that a cursor keeps the answer from when it opened and returns the end after `close()`, and that a `transact` is API-012. ### Covered Areas diff --git a/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md b/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md new file mode 100644 index 000000000..d0cfc7d4c --- /dev/null +++ b/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md @@ -0,0 +1,113 @@ +# Binding surface for cursors, open options, the fact log and the log writer — design + +**Issues:** #462 (binding cursors), #465 (`open_with_options`), #467 (`LogWriter` and +`FactRecord`). **Core:** #432 (PR #461), #430 (PR #463), #429 (PR #464), #431 (PR #466). +**Branch:** `v3` in core and in every binding repo. + +## Goal + +Every binding can do what the offline migration in temporal_reasoning#378 needs: +open a source read-only, stream its query answers and its full fact log, and write a +new file from the records. The shape is the same in every binding, with each +language's idiom on top. + +## Branching + +The bindings pin minigraf from crates.io, but these APIs exist only on the +unreleased `v3` branch. Each binding repo gets a `v3` branch that depends on core by +git revision: + +```toml +minigraf = { git = "https://github.com/project-minigraf/minigraf", rev = "" } +``` + +Feature PRs target `v3`, and each repo's CI runs on PRs to `v3`. `main` keeps serving +v2.x patch releases. At the v3.0.0 release the dependency switches to +`minigraf = "=3.0.0"` and `v3` becomes `main`, as in core. + +## Core additions (this PR) + +- **`API-017`** `invalid argument: {}`, from `MinigrafError::invalid_argument(detail)`. + A binding returns it for an argument it cannot convert, such as an entity id that is + not a UUID string or a number out of range. The Rust API takes typed arguments and + never returns it. +- **`API-018`** `{} is closed`, from `MinigrafError::closed(object)`. A binding + cannot consume an object its caller still holds, so a log writer used after `finish` + or `close` returns it. Rust consumes these objects and never returns it. +- **`BrowserDb.query(datalog)`** returns a `BrowserCursor` (`vars()`, + `nextBatch(maxRows)` with JSON rows or `undefined`, `close()`). `transact`, `retract` + and `rule` are `API-012`, and bind slots are `API-010`, as in `Minigraf::query`. + +Bindings build these errors with the core constructors, so every binding reports the +same code and text, and `docs/ERROR_REFERENCE.md` stays the one registry. + +## Shared shape + +| Concept | UniFFI (Python, Java, Android, Swift) | Node (napi) | C | Browser | +|---|---|---|---|---| +| Open options | `OpenOptions` record, every field optional | options object | JSON object string | not applicable | +| Open | `MiniGrafDb.open_with_options(path, options)` | `MiniGrafDb.openWithOptions(path, options)` | `minigraf_open_with_options(path, json)` | — | +| Cursor | `db.query(datalog) -> MiniGrafCursor` | `db.query(datalog) -> Cursor` | `minigraf_query` | `BrowserDb.query` | +| Cursor rows | JSON string of rows, as in `execute()` | JSON string of rows | JSON string of rows | JSON string of rows | +| Fact log | `db.fact_log(filter) -> MiniGrafFactLog` | `db.factLog(filter)` | `minigraf_fact_log(db, json)` | not exposed | +| Fact record value | `MiniGrafValue` enum | `{ type, value }` object | `{"type", "value"}` JSON | — | +| Log writer | `MiniGrafLogWriter.create(path, options)` | `LogWriter.create(path, options)` | `minigraf_log_writer_create` | not built on wasm32 | + +### Decisions + +1. **Cursor rows stay JSON.** A batch is a JSON array of rows encoded exactly like the + `results` of `execute()`, so a cursor and `execute()` give the same rows and every + binding reuses its existing decoding. `max_rows == 0` counts as 1, a batch is never + empty, and the end is `None`/`null`/`undefined`. After `close()` a cursor returns + the end, not an error. +2. **Fact records are lossless.** A migration must write back exactly what it read, so + `FactRecord.value` keeps the value's type, and a `Ref` and a string with the same + text stay different. UniFFI bindings get an enum with the variants `Text`, `Int64`, + `Float64`, `Bool`, `Ref`, `Keyword` and `Null`; these names avoid `String`, `Int`, + `Float` and `Boolean`, which would shadow built-in types in Kotlin and Swift. JSON + and Node use `{ "type": "string" | "integer" | "float" | "boolean" | "ref" | + "keyword" | "null", "value": ... }`. Entity ids and `Ref` values are UUID strings; + anything else is `API-017`. +3. **64-bit fields stay exact.** `valid_to` is `i64::MAX` for forever. Node uses + `BigInt` for `txCount`, `txId`, `validFrom` and `validTo`, because a JS number + cannot hold `i64::MAX`. JSON writes them as integers. +4. **Fact filter.** A record with optional `attributes`, `attribute_prefixes`, + `entities`, `tx_from`, `tx_to` (inclusive), `order` (`tx` or `storage`) and + `window`. An absent field does not filter. +5. **`append_batch(records)`** appends in order and stops at the first rejected + record. The error is that record's coded error with ` (batch index N)` appended, + and the records before it stay appended, as if appended one by one. +6. **Writer lifetime.** `finish()` consumes the core writer behind a mutex. `close()` + drops it unfinished, which deletes `.partial`, and is a no-op when already + finished or closed. Every other call after `finish` or `close` is `API-018`. + Python's context manager finishes on a clean exit and closes on an exception; Java's + `close()` (`AutoCloseable`) closes without finishing. +7. **Options map onto `OpenOptions` builders.** Counts are 64-bit unsigned in the + bindings and convert to `usize`; a value that does not fit is `API-017`. An absent + field keeps the Rust default. +8. **Errors keep their code in the message.** Every binding already surfaces + `[CODE] message` (the `Display` of `MinigrafError`), so API-014, API-015, API-017, + API-018, STG-042 and STG-043 reach callers like every other code. + +### Not applicable + +- **Browser (`BrowserDb`, minigraf-wasm):** `read_only`, `allow_unlocked`, + `page_cache_size`, the WAL options and `LogWriter` are file-backed only. BrowserDb + keeps its pages in memory, has no WAL and no file lock, and `LogWriter` is not built + on wasm32. The fact log is not exposed in the browser; the cursor is. +- **WASI package:** it ships the REPL binary, not a library API, so there is nothing to + expose. +- **`PreparedQuery.query`** follows #181 (v3.1.0). + +## Tests + +Each binding tests, in its own language: + +- cursor batches of 1, 7 and 1,000 equal `execute()`, `close()` after the first batch, + a cursor opened before a `transact` keeps the earlier answer, and a non-query is + `API-012`; +- a read-only open with queries working, a write giving `API-014`, a missing file + `STG-042`, and two read-only handles on one file; +- a round trip through `fact_log` and the log writer that gives the same records and + `current_tx_count`, a hole from a skipped transaction, `API-015` and `STG-043`, and a + writer closed without `finish` leaving no file. diff --git a/src/browser/mod.rs b/src/browser/mod.rs index 8b1c61159..0a2625290 100644 --- a/src/browser/mod.rs +++ b/src/browser/mod.rs @@ -139,6 +139,22 @@ impl BrowserDb { } } + /// Open a cursor over a `(query ...)` and return its rows in batches. + /// + /// The answer is fixed when the cursor opens: later writes do not change + /// it. Synchronous, since reading never touches IndexedDB. + /// + /// Throws `[API-012]` for a `transact`, `retract` or `rule`, and + /// `[API-010]` for a query with `$slot` bind slots. + #[wasm_bindgen(js_name = query)] + pub fn query(&self, datalog: &str) -> Result { + let cursor = self.query_inner(datalog).map_err(to_js_error)?; + Ok(BrowserCursor { + vars: cursor.vars().to_vec(), + cursor: Some(cursor), + }) + } + /// Flush all dirty pages to IndexedDB. /// /// Write-through means individual `execute()` calls already flush dirty pages, @@ -249,6 +265,28 @@ impl BrowserDb { } impl BrowserDb { + fn query_inner(&self, datalog: &str) -> anyhow::Result { + use crate::error::{ErrorCode, bail_coded}; + + let cmd = parse_datalog_command(datalog)?; + match &cmd { + DatalogCommand::Query(q) => crate::query::datalog::prepared::reject_unbound_slots(q)?, + DatalogCommand::Transact(_) => bail_coded!(ErrorCode::Api012, "transact"), + DatalogCommand::Retract(_) => bail_coded!(ErrorCode::Api012, "retract"), + DatalogCommand::Rule(_) => bail_coded!(ErrorCode::Api012, "rule"), + } + let result = { + let inner = self.inner.borrow(); + DatalogExecutor::new_with_rules_and_functions( + inner.fact_storage.clone(), + inner.rules.clone(), + inner.functions.clone(), + ) + .execute(cmd)? + }; + crate::Cursor::from_result(result) + } + /// Apply a batch of pre-materialized facts to the in-memory store and /// flush dirty pages to IndexedDB (if present). /// @@ -331,6 +369,60 @@ impl BrowserDb { } } +// ── BrowserCursor ─────────────────────────────────────────────────────────── + +/// A cursor from `BrowserDb.query()`: the rows of one query answer, in batches. +/// +/// ```js +/// const cursor = db.query("(query [:find ?n :where [?e :name ?n]])"); +/// try { +/// let batch; +/// while ((batch = cursor.nextBatch(1000)) !== undefined) { +/// for (const row of JSON.parse(batch)) console.log(row); +/// } +/// } finally { +/// cursor.close(); +/// } +/// ``` +#[wasm_bindgen] +pub struct BrowserCursor { + vars: Vec, + /// `None` after `close()`. + cursor: Option, +} + +#[wasm_bindgen] +impl BrowserCursor { + /// The query's `:find` variables, in column order. + pub fn vars(&self) -> Vec { + self.vars.clone() + } + + /// The next batch of at most `maxRows` rows (`0` counts as 1), as a JSON + /// array of rows encoded like `execute()`'s `results`, or `undefined` at + /// the end. A batch is never empty. After `close()` it returns + /// `undefined`. + #[wasm_bindgen(js_name = nextBatch)] + pub fn next_batch(&mut self, max_rows: u32) -> Result, JsValue> { + let Some(cursor) = self.cursor.as_mut() else { + return Ok(None); + }; + let batch = cursor + .next_batch(max_rows as usize) + .map_err(|e| JsValue::from_str(&e.to_string()))?; + Ok(batch.map(|b| rows_to_json(b.rows()))) + } + + /// Release the cursor's rows. Later `nextBatch()` calls return `undefined`. + pub fn close(&mut self) { + self.cursor = None; + } +} + +fn to_js_error(e: anyhow::Error) -> JsValue { + JsValue::from_str(&crate::MinigrafError::from(e).to_string()) +} + // ── JSON serialisation helpers (free functions, not exported to WASM) ──────── fn query_result_to_json(result: QueryResult) -> String { @@ -355,6 +447,14 @@ fn query_result_to_json(result: QueryResult) -> String { val.to_string() } +fn rows_to_json(rows: &[Vec]) -> String { + let rows: Vec> = rows + .iter() + .map(|row| row.iter().map(value_to_json).collect()) + .collect(); + serde_json::Value::from(rows).to_string() +} + fn value_to_json(v: &crate::graph::types::Value) -> serde_json::Value { use crate::graph::types::Value; use serde_json::Value as JVal; @@ -398,6 +498,75 @@ mod tests { assert_eq!(results[0][0], serde_json::Value::String("Alice".into())); } + #[wasm_bindgen_test] + async fn cursor_batches_match_execute() { + let db = BrowserDb::open_in_memory().expect("open_in_memory"); + for i in 0..20 { + db.execute(format!("(transact [[:e{i} :n {i}]])")) + .await + .expect("transact"); + } + let q = "(query [:find ?n :where [?e :n ?n]])"; + let all: serde_json::Value = + serde_json::from_str(&db.execute(q.to_string()).await.expect("execute")).unwrap(); + let mut expected: Vec = all["results"] + .as_array() + .unwrap() + .iter() + .map(|r| r[0].as_i64().unwrap()) + .collect(); + expected.sort_unstable(); + for size in [1u32, 7, 1000] { + let mut cursor = db.query(q).expect("query"); + assert_eq!(cursor.vars(), vec!["?n".to_string()]); + let mut got = Vec::new(); + while let Some(batch) = cursor.next_batch(size).expect("next_batch") { + let rows: Vec> = serde_json::from_str(&batch).unwrap(); + assert!(!rows.is_empty() && rows.len() <= size as usize); + got.extend(rows.into_iter().map(|r| r[0])); + } + got.sort_unstable(); + assert_eq!(got, expected, "batch size {size}"); + } + } + + #[wasm_bindgen_test] + async fn cursor_is_fixed_at_open_and_closes() { + let db = BrowserDb::open_in_memory().expect("open_in_memory"); + db.execute(r#"(transact [[:a :n 1]])"#.to_string()) + .await + .expect("transact"); + let mut cursor = db + .query("(query [:find ?n :where [?e :n ?n]])") + .expect("query"); + db.execute(r#"(transact [[:b :n 2]])"#.to_string()) + .await + .expect("transact"); + assert_eq!( + cursor.next_batch(10).expect("batch").as_deref(), + Some("[[1]]") + ); + assert_eq!(cursor.next_batch(10).expect("end"), None); + + let mut cursor = db + .query("(query [:find ?n :where [?e :n ?n]])") + .expect("query"); + assert!(cursor.next_batch(1).expect("batch").is_some()); + cursor.close(); + assert_eq!(cursor.next_batch(1).expect("closed"), None); + } + + #[wasm_bindgen_test] + fn cursor_rejects_non_queries() { + let db = BrowserDb::open_in_memory().expect("open_in_memory"); + let err = db + .query(r#"(transact [[:a :n 1]])"#) + .err() + .and_then(|e| e.as_string()) + .expect("error string"); + assert!(err.starts_with("[API-012]"), "expected API-012"); + } + #[wasm_bindgen_test] async fn empty_query_returns_empty_results() { let db = BrowserDb::open_in_memory().expect("open_in_memory"); diff --git a/src/error.rs b/src/error.rs index b09874ce3..d82afed4e 100644 --- a/src/error.rs +++ b/src/error.rs @@ -206,6 +206,8 @@ pub(crate) enum ErrorCode { Api014, Api015, Api016, + Api017, + Api018, Int001, Int002, Int003, @@ -1193,6 +1195,18 @@ pub(crate) const REGISTRY: &[(ErrorCode, &str, &str, ErrorCategory)] = &[ "transaction {} has tx_id {}; a record with tx_id {} cannot join it", ErrorCategory::Api, ), + ( + ErrorCode::Api017, + "API-017", + "invalid argument: {}", + ErrorCategory::Api, + ), + ( + ErrorCode::Api018, + "API-018", + "{} is closed", + ErrorCategory::Api, + ), ( ErrorCode::Int001, "INT-001", @@ -1700,6 +1714,20 @@ impl MinigrafError { pub fn code(&self) -> &'static str { self.code } + + /// An argument a language binding could not convert, such as a malformed + /// entity UUID string (`API-017`). For binding authors; the Rust API + /// takes typed arguments and never returns it. + pub fn invalid_argument(detail: impl fmt::Display) -> Self { + MinigrafError::from(err_coded!(ErrorCode::Api017, detail)) + } + + /// A call on a binding object that was already closed or finished, such + /// as a log writer after `finish` (`API-018`). `object` names it. For + /// binding authors; the Rust API consumes such objects instead. + pub fn closed(object: impl fmt::Display) -> Self { + MinigrafError::from(err_coded!(ErrorCode::Api018, object)) + } } impl fmt::Display for MinigrafError { @@ -1955,6 +1983,8 @@ mod tests { | ErrorCode::Api014 | ErrorCode::Api015 | ErrorCode::Api016 + | ErrorCode::Api017 + | ErrorCode::Api018 | ErrorCode::Int001 | ErrorCode::Int002 | ErrorCode::Int003 @@ -2169,6 +2199,8 @@ mod tests { ErrorCode::Api014, ErrorCode::Api015, ErrorCode::Api016, + ErrorCode::Api017, + ErrorCode::Api018, ErrorCode::Int001, ErrorCode::Int002, ErrorCode::Int003, @@ -2230,6 +2262,22 @@ mod tests { } } + #[test] + fn binding_constructors_carry_their_codes() { + let e = MinigrafError::invalid_argument("entity \"alice\" is not a UUID"); + assert_eq!(e.code(), "API-017"); + assert_eq!(e.category(), ErrorCategory::Api); + assert_eq!( + e.to_string(), + "[API-017] invalid argument: entity \"alice\" is not a UUID" + ); + + let e = MinigrafError::closed("log writer"); + assert_eq!(e.code(), "API-018"); + assert_eq!(e.category(), ErrorCategory::Api); + assert_eq!(e.to_string(), "[API-018] log writer is closed"); + } + #[test] fn registry_entry_resolves_int000() { let (code_str, template, category) = registry_entry(ErrorCode::Int000); diff --git a/src/storage/dir_sync.rs b/src/storage/dir_sync.rs index 21f9d8d9c..65edc6fae 100644 --- a/src/storage/dir_sync.rs +++ b/src/storage/dir_sync.rs @@ -84,7 +84,8 @@ pub(crate) fn take_sync_log() -> Vec { SYNC_LOG.with(|log| std::mem::take(&mut *log.borrow_mut())) } -#[cfg(test)] +// The tests use `tempfile`, a native-only dev-dependency. +#[cfg(all(test, not(target_arch = "wasm32")))] mod tests { use super::*; diff --git a/src/storage/persistent_facts.rs b/src/storage/persistent_facts.rs index 9e9f3fe47..6cca5cda9 100644 --- a/src/storage/persistent_facts.rs +++ b/src/storage/persistent_facts.rs @@ -669,17 +669,6 @@ impl PersistentFactStorage { self.dirty = false; } - /// Capacity (in pages) of the internal LRU page cache passed to `new()`. - /// - /// Exposed only for the browser WASM layer's tests, which verify that - /// `BrowserDb` constructors pass `0` (cache disabled — see #275) since - /// `BrowserBufferBackend` is already an in-memory page store and the LRU - /// layer on top of it would be redundant. - #[cfg(all(target_arch = "wasm32", feature = "browser", test))] - pub(crate) fn page_cache_capacity(&self) -> usize { - self.page_cache.capacity() - } - /// Run a closure with read access to the underlying storage backend. /// /// Used by the browser WASM layer to read pages after `save()` without From 0170a59205f783a09e7d85dc3888b5040069b9dd Mon Sep 17 00:00:00 2001 From: Aditya Mukhopadhyay Date: Wed, 7 Oct 2026 11:25:16 +0530 Subject: [PATCH 2/3] docs(spec): signed counts in the UniFFI shim, Kotlin close renames Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn --- .../2026-10-07-binding-etl-surface-design.md | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md b/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md index d0cfc7d4c..fa78807ec 100644 --- a/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md +++ b/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md @@ -81,13 +81,21 @@ same code and text, and `docs/ERROR_REFERENCE.md` stays the one registry. drops it unfinished, which deletes `.partial`, and is a no-op when already finished or closed. Every other call after `finish` or `close` is `API-018`. Python's context manager finishes on a clean exit and closes on an exception; Java's - `close()` (`AutoCloseable`) closes without finishing. -7. **Options map onto `OpenOptions` builders.** Counts are 64-bit unsigned in the - bindings and convert to `usize`; a value that does not fit is `API-017`. An absent - field keeps the Rust default. + `close()` (`AutoCloseable`) frees the writer, which abandons an unfinished build. +7. **Options map onto `OpenOptions` builders.** In the UniFFI shim every count + (options, `tx_count`, `tx_id`, batch sizes, filter bounds) is a signed integer + (`i64`, `i32` for batch sizes): Kotlin's `ULong`/`UInt` are name-mangled and cannot + be called from Java. A negative or oversized value is `API-017`. C and JSON use + unsigned integers and `size_t`. An absent field keeps the Rust default. +9. **Kotlin renames `close`.** UniFFI's Kotlin objects are `AutoCloseable`, and + `close()` frees the Rust object, which releases a cursor or fact log and abandons an + unfinished writer. The shim's own `close` methods clash with it, so the Kotlin + bindings rename them in `uniffi.toml`: `release()` (cursor, fact log) and `abandon()` + (writer). 8. **Errors keep their code in the message.** Every binding already surfaces `[CODE] message` (the `Display` of `MinigrafError`), so API-014, API-015, API-017, - API-018, STG-042 and STG-043 reach callers like every other code. + API-018, STG-042 and STG-043 reach callers like every other code. (UniFFI wraps + it: Python's `e.msg`, Kotlin's `Minigraf.errorMessage(e)`, Swift's `e.message`.) ### Not applicable From 9c83fe45480a405a759f350f1340d2c2631a3ffa Mon Sep 17 00:00:00 2001 From: Aditya Mukhopadhyay Date: Wed, 7 Oct 2026 11:26:13 +0530 Subject: [PATCH 3/3] docs(spec): fix decision numbering Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn --- .../specs/2026-10-07-binding-etl-surface-design.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md b/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md index fa78807ec..2ac8083c0 100644 --- a/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md +++ b/docs/superpowers/specs/2026-10-07-binding-etl-surface-design.md @@ -87,12 +87,12 @@ same code and text, and `docs/ERROR_REFERENCE.md` stays the one registry. (`i64`, `i32` for batch sizes): Kotlin's `ULong`/`UInt` are name-mangled and cannot be called from Java. A negative or oversized value is `API-017`. C and JSON use unsigned integers and `size_t`. An absent field keeps the Rust default. -9. **Kotlin renames `close`.** UniFFI's Kotlin objects are `AutoCloseable`, and +8. **Kotlin renames `close`.** UniFFI's Kotlin objects are `AutoCloseable`, and `close()` frees the Rust object, which releases a cursor or fact log and abandons an unfinished writer. The shim's own `close` methods clash with it, so the Kotlin bindings rename them in `uniffi.toml`: `release()` (cursor, fact log) and `abandon()` (writer). -8. **Errors keep their code in the message.** Every binding already surfaces +9. **Errors keep their code in the message.** Every binding already surfaces `[CODE] message` (the `Display` of `MinigrafError`), so API-014, API-015, API-017, API-018, STG-042 and STG-043 reach callers like every other code. (UniFFI wraps it: Python's `e.msg`, Kotlin's `Minigraf.errorMessage(e)`, Swift's `e.message`.)