Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- **Configurable recursion round limit** (#861, part of #862). `OpenOptions::max_iterations` sets the most rounds one stratum of a rule query may run (default 1,000,000, see Breaking changes); `:max-iterations N` or `$slot` replaces it for one query, up or down (INT-012/013/014/015 as for `:max-derived-facts`), gated by the new `Overridable::MaxIterations`, which **is** in `Overridable::DEFAULTS`; `PreparedQuery::with_max_iterations` replaces it for one statement, ungated, and an allowed keyword still overrides it. Rounds are still counted per stratum. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md).

- **Owned write transactions and browser transactions** (#824). `Minigraf::begin_write_owned()` and `begin_write_owned_with(TxOptions)` return an `OwnedWriteTransaction`: `execute`, `commit` and `rollback` with the borrowed `WriteTransaction`'s results and errors, but it is `Send` and `'static`, keeps the database open, and may begin on one thread and run, commit, roll back or be dropped on another. Dropping it open rolls it back. While it is open the calls a `WriteTransaction` refuses are `INT-001` on the thread that began it (per database, whichever thread uses it meanwhile); other threads' writes, checkpoints and begins wait until it finishes, and their queries read the committed state without waiting. A user function in one of its statements that calls the database gets `INT-001` on whichever thread runs the statement. Every language binding's transaction is built on it. `BrowserDb.beginWrite()` returns a `BrowserWriteTransaction` with a synchronous `execute`, a `commit()` that applies the staged facts as one transaction and resolves after the IndexedDB flush, and a `rollback()`; freeing it open rolls it back. While it is open, `BrowserDb`'s `execute`, `query`, `prepare`, `checkpoint`, `importGraph`, a second `beginWrite` and a prepared query's `execute`/`query` throw or reject with `[INT-001]`; a finished one's `execute` and `commit` are `[API-018]`. No new error codes; no file format change.

### Changed

- **INT-001 text** (#824): `a WriteTransaction is already in progress on this thread; use tx.execute() instead` now reads `a write transaction is already in progress on this thread; use tx.execute() instead`, since owned and browser transactions raise it too. The code and the calls that raise it for a borrowed transaction are unchanged.

- **INT-015 text** (#870): `{} must be a positive integer` is now `{} must be a non-negative integer`, because `:query-memory-limit` accepts 0. The code and when `:max-derived-facts`/`:max-results` raise it are unchanged.

- **INT-020 round-limit text** (#861): `Max iterations (N) exceeded. Possible infinite recursion or cycle in rules.` now reads `Max iterations (N) exceeded. A stratum's rules were still deriving new facts after N rounds; raise the limit with :max-iterations N or OpenOptions::max_iterations.` The code is unchanged.
Expand All @@ -38,6 +42,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Testing

- **Owned and browser transaction tests** (#824): `tests/owned_tx_test.rs` checks that an owned transaction leaves the database as a borrowed one given the same statements and options (commit, rollback, drop; small and large; file and in-memory, after reopening), the threads rules and the begin refusals; the fault matrix runs its large-transaction and mixed-sync-mode cases through the owned transaction too. `src/browser/tx_tests.rs` (`wasm-bindgen`) checks the browser transaction against the native one over fixed statement sequences, its IndexedDB commit, rollback, drop and failed-flush paths, and the `[INT-001]`/`[API-018]` rules.

- **Acceptance execution gaps** (#837): nonlinear and multistratum recursive
scaling, custom transaction chunk/clamp bounds, mandatory Linux RSS assertions,
and exact named-test/output verification in the nightly smoke route. The storage
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ cargo run < demos/demo_negation.txt

5. **`src/repl.rs`** — Interactive REPL; TTY-aware (suppresses prompts/banner for piped input)

6. **`src/db.rs`** — Public API: `Minigraf::open/execute/query/fact_log/prepare/begin_write/begin_write_with/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `TxOptions` (#871: one transaction's `synchronous` and `chunk_facts`; `None` = the database value; the WAL writer keeps no mode, each append gets its transaction's; no effect on a large transaction, whose WAL guard is always synced), `IntegrityReport`, `OpenOptions::page_cache_size`, `OpenOptions::read_only` (#429: shared lock, nothing written, writes are `API-014`), `OpenOptions::query_memory_limit` (#549: `QRY-012`, off by default), `OpenOptions::transaction_chunk_facts` (#550), `OpenOptions::max_iterations` (#861: rounds per stratum, default 1,000,000), `OpenOptions::allow_overrides` (#869: which query keywords may override settings, `API-025`; see `docs/CONFIGURATION.md`)
6. **`src/db.rs`** — Public API: `Minigraf::open/execute/query/fact_log/prepare/begin_write/begin_write_with/begin_write_owned/begin_write_owned_with/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `OwnedWriteTransaction` (#824: `Send + 'static`, holds a `Minigraf` clone; the write lock is `Mutex<WriteSlot>` + `Condvar`, `WriteSlot::owned` makes every other writer wait in `Inner::lock_write`; `TxOwner` records its begin thread per database, and `refuse_reentry` gives `INT-001` there as the thread-local flag does for `WriteTransaction`, and each owned statement sets that flag on the thread running it (`RunningHere`); both kinds share `TxState`, and small staging is `Staged`, which `BrowserWriteTransaction` uses too), `TxOptions` (#871: one transaction's `synchronous` and `chunk_facts`; `None` = the database value; the WAL writer keeps no mode, each append gets its transaction's; no effect on a large transaction, whose WAL guard is always synced), `IntegrityReport`, `OpenOptions::page_cache_size`, `OpenOptions::read_only` (#429: shared lock, nothing written, writes are `API-014`), `OpenOptions::query_memory_limit` (#549: `QRY-012`, off by default), `OpenOptions::transaction_chunk_facts` (#550), `OpenOptions::max_iterations` (#861: rounds per stratum, default 1,000,000), `OpenOptions::allow_overrides` (#869: which query keywords may override settings, `API-025`; see `docs/CONFIGURATION.md`)
- `src/overrides.rs`: `Overridable` (`DEFAULTS`, `keyword`; `QueryMemoryLimit` not in `DEFAULTS`, #870; `MaxIterations` in it, #861) and `check_overrides`, called after parsing at every query entry point (`BrowserDb` uses `DEFAULTS`)
- `src/cursor.rs`: `Cursor` / `Batch` — owned, `Send` cursor returned by `Minigraf::query` and `PreparedQuery::query` (#432); pulls engine queries one internal batch at a time (`close`/drop stops the work; a read error after some rows comes on the next call), other queries computed at open; holds a `Snapshot` (one state of the store, its committed generation pinned, #543) until dropped
- `src/fact_log.rs`: `FactLog` / `FactFilter` / `FactRecord` / `FactOrder` — every fact version streamed by `Minigraf::fact_log` (#430): tx order (multi-pass, bounded window) or storage order, key-level filters; an open log pins the committed generation (`storage::pins`), so checkpoints run but do not reuse its pages (`API-013` deprecated, #543)
Expand All @@ -197,7 +197,7 @@ cargo run < demos/demo_negation.txt

8. **`src/error.rs`** — Structured error codes: `MinigrafError`, `ErrorCategory`, `ErrorCode` registry (PRS/QRY/STG/WAL/API/INT codes); must match `docs/ERROR_REFERENCE.md`. `MinigrafError::invalid_argument` (API-017) and `MinigrafError::closed` (API-018) exist for language bindings

9. **`src/browser/`** — Browser WASM backend (`browser` feature): `buffer.rs` (`BrowserBufferBackend`, in-memory pages with dirty tracking; free pages a pinned generation may read are withheld from release, `pins::split_released`), `indexeddb.rs` (IndexedDB persistence); `BrowserDb.query` returns a `BrowserCursor` (JSON row batches); `BrowserDb.prepare` returns a `BrowserPreparedQuery` (`execute`/`query` with a bindings object; `withMaxDerivedFacts`/`withQueryMemoryLimit`/`withMaxIterations` return a new one, #872; read by `binds.rs`: plain JS values or `{type, value}` forms, #181; a set as `{type: 'set', value: [..]}` or a JS `Set`, #316); `udfs.rs`: `registerPredicate`/`registerAggregate` (#180; values as `{type, value}` objects, a throw is `QRY-014`; JS values held in a thread-local table, the registry keeps ids). Tests here must be `#[wasm_bindgen_test]` (the module only builds for wasm32); `.github/workflows/wasm.yml` runs them in headless browsers, runs the WASI tests under Wasmtime, and lints both wasm targets. New file-backed integration test files need `#![cfg(not(target_arch = "wasm32"))]`
9. **`src/browser/`** — Browser WASM backend (`browser` feature): `buffer.rs` (`BrowserBufferBackend`, in-memory pages with dirty tracking; free pages a pinned generation may read are withheld from release, `pins::split_released`), `indexeddb.rs` (IndexedDB persistence); `BrowserDb.query` returns a `BrowserCursor` (JSON row batches); `BrowserDb.prepare` returns a `BrowserPreparedQuery` (`execute`/`query` with a bindings object; `withMaxDerivedFacts`/`withQueryMemoryLimit`/`withMaxIterations` return a new one, #872; read by `binds.rs`: plain JS values or `{type, value}` forms, #181; a set as `{type: 'set', value: [..]}` or a JS `Set`, #316); `BrowserDb.beginWrite` returns a `BrowserWriteTransaction` (#824: sync `execute` staging through `db::Staged`, `commit()` a `Promise` resolved after the IndexedDB flush, `rollback()`; `BrowserDbInner::tx_open` makes every other read/write call `INT-001`; finished = `API-018`; tests in `tx_tests.rs`); `udfs.rs`: `registerPredicate`/`registerAggregate` (#180; values as `{type, value}` objects, a throw is `QRY-014`; JS values held in a thread-local table, the registry keeps ids). Tests here must be `#[wasm_bindgen_test]` (the module only builds for wasm32); `.github/workflows/wasm.yml` runs them in headless browsers, runs the WASI tests under Wasmtime, and lints both wasm targets. New file-backed integration test files need `#![cfg(not(target_arch = "wasm32"))]`

### Data Model

Expand Down Expand Up @@ -250,7 +250,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**2133 tests** (2112 active, 21 ignored; unit + integration + doc). Engine answers once checked by the executor are frozen in `tests/answers/` (#551).
**2159 tests** (2138 active, 21 ignored; unit + integration + doc). Engine answers once checked by the executor are frozen in `tests/answers/` (#551).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

The #760 deterministic storage suite adds three native unit tests in a separate `--cfg minigraf_shuttle` build, outside the default count. Run only `graph::storage::deterministic::` with that cfg; ordinary unit tests do not enter its scheduler. `deterministic-schedules.yml` runs relevant v3 PRs on Linux/macOS/Windows plus Linux MSRV, and larger nightly/manual searches with replay artifacts. The identical main workflow activates the v3 schedule after both PRs merge. See `docs/testing/deterministic-schedules.md` for scope, controls and findings.
Expand Down
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,12 @@ let mut tx = db.begin_write()?;
tx.execute(r#"(transact [[:alice :person/age 31]])"#)?;
tx.commit()?;

// Owned transaction — Send + 'static, so it can live in an object or move to
// another thread; every language binding's transaction is one
let mut tx = db.begin_write_owned()?;
tx.execute(r#"(transact [[:alice :person/age 32]])"#)?;
std::thread::spawn(move || tx.commit()).join().unwrap()?;

// Time travel — query as of past transaction counter
db.execute("(query [:find ?age :as-of 1 :where [:alice :person/age ?age]])")?;

Expand Down Expand Up @@ -271,7 +277,7 @@ See the [Mobile Integration](https://project-minigraf.github.io/minigraf-docs/la

### For WASM / Browser

Published as [`@minigraf/browser`](https://www.npmjs.com/package/@minigraf/browser) on npm (IndexedDB-backed, `wasm-pack`). WASI build (`wasm32-wasip1`) available as [`@minigraf/wasi`](https://www.npmjs.com/package/@minigraf/wasi) on npm and as a GitHub Releases artifact (Wasmtime / Wasmer). See the [Use Cases](https://project-minigraf.github.io/minigraf-docs/latest/use-cases/). The [playground](https://minigraf-playground.vercel.app/) and the [time travel visualizer](https://project-minigraf.github.io/minigraf-visualizer/) are both built on `@minigraf/browser`.
Published as [`@minigraf/browser`](https://www.npmjs.com/package/@minigraf/browser) on npm (IndexedDB-backed, `wasm-pack`). `BrowserDb.beginWrite()` opens a write transaction (from v3.0.0): `execute` stages and reads back, `await tx.commit()` stores it in IndexedDB, `rollback()` discards it, and the database's other calls throw `[INT-001]` until it ends. The WASI package is the REPL: each command commits on its own, with no explicit transactions. WASI build (`wasm32-wasip1`) available as [`@minigraf/wasi`](https://www.npmjs.com/package/@minigraf/wasi) on npm and as a GitHub Releases artifact (Wasmtime / Wasmer). See the [Use Cases](https://project-minigraf.github.io/minigraf-docs/latest/use-cases/). The [playground](https://minigraf-playground.vercel.app/) and the [time travel visualizer](https://project-minigraf.github.io/minigraf-visualizer/) are both built on `@minigraf/browser`.

### For Python / Node.js / Java / C

Expand Down
30 changes: 22 additions & 8 deletions docs/ERROR_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ with no `CodedError` anywhere in its chain.
| API-024 | Bind slot type mismatch | Database API |
| API-025 | Query option override not allowed | Database API |
| INT-000 | Unclassified internal error | Internal |
| INT-001 | WriteTransaction already in progress on this thread | Internal |
| INT-001 | Write transaction already in progress on this thread | Internal |
| INT-002 | Invalid entity (API layer) | Internal |
| INT-003 | Invalid value (API layer) | Internal |
| INT-004 | Invalid timestamp | Internal |
Expand Down Expand Up @@ -3557,7 +3557,7 @@ let mut log = src.fact_log(&FactFilter::new())?;

**Error text**: `{} is closed`

**Cause**: A language binding object was used after it was closed or finished, for example `append` or `finish` on a log writer after `finish` or `close`. Rust consumes these objects, so the Rust API never returns this code. Bindings cannot consume an object a caller still holds, so they return this error instead. A cursor or fact log reports the end (no more batches) after `close`, not this error. The `{}` names the object.
**Cause**: A language binding object was used after it was closed or finished, for example `append` or `finish` on a log writer after `finish` or `close`, or `execute` or `commit` on a write transaction after it committed (or failed to), rolled back, or its database was closed (#824). A finished transaction's `rollback` does nothing instead. Rust consumes these objects, so the Rust API never returns this code. Bindings cannot consume an object a caller still holds, so they return this error instead. A cursor or fact log reports the end (no more batches) after `close`, not this error. The `{}` names the object.

**Resolution**:
- Create a new object instead of reusing a finished one.
Expand Down Expand Up @@ -3704,22 +3704,36 @@ specific code, so this should now be rare.
**Scenario**: An I/O error from the underlying filesystem propagates through
`?` without ever being wrapped in a `CodedError`.

### INT-001 WriteTransaction already in progress on this thread
### INT-001 Write transaction already in progress on this thread

**Error text**: `a WriteTransaction is already in progress on this thread; use tx.execute() instead`
**Error text**: `a write transaction is already in progress on this thread; use tx.execute() instead`

**Cause**: on the same thread that already holds an active
`WriteTransaction`, one of these was called: the `Minigraf` methods
`execute()`, `query()`, `fact_log()`, `begin_write()`, `checkpoint()`,
`verify()`, `rebuild_indexes()` or `prepare()`, or a `PreparedQuery`'s
`execute()` or `query()`. Acquiring the write
`execute()`, `query()`, `fact_log()`, `begin_write()`, `begin_write_owned()`,
`checkpoint()`, `verify()`, `rebuild_indexes()` or `prepare()`, or a
`PreparedQuery`'s `execute()` or `query()`. Acquiring the write
lock again on that thread would deadlock, so this is detected and rejected
instead. (Before v3.0.0, `checkpoint()`, `verify()` and `rebuild_indexes()`
did not check and deadlocked, #734.)

The same calls are refused on the thread that began an open
`OwnedWriteTransaction` of that database (#824), whichever thread uses the
transaction meanwhile, and on the thread running one of its statements (a
user function calling back into the database); other threads' writes and begins wait for it
instead, and other databases are not affected. A binding transaction is an
owned one, so the same rule applies to every language binding. In the
browser, every `BrowserDb` call that reads or writes (`execute`, `query`,
`prepare`, `checkpoint`, `importGraph`, a second `beginWrite`, and a
prepared query's `execute`/`query`) is refused while a
`BrowserWriteTransaction` is open, since other asynchronous code could
otherwise mistake the database for the transaction.

**Resolution**:
- Use the existing `WriteTransaction`'s `tx.execute()` for further writes
- Use the existing transaction's `tx.execute()` for further writes
instead of calling `db.execute()` again on the same thread.
- End the transaction (`commit()` or `rollback()`) before using the
database directly again.

**Scenario**: Code inside a `WriteTransaction` write closure calls
`db.execute("(transact ...)")` on the outer `db` handle instead of
Expand Down
Loading
Loading