Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Documentation

- **Generated grammar page (#643).** `docs/GRAMMAR.md`, published on the docs site as the Grammar page, is EBNF rendered from `tests/grammar/grammar.pest`, the grammar the parser verifier checks the parser against, so the documented syntax is the tested one. `tests/grammar_doc.rs` renders it with `pest_meta` (a new dev-dependency, already built for `pest_derive`) and fails when the file is stale; `UPDATE_GRAMMAR_DOC=1 cargo test --test grammar_doc` rewrites it. A pest lookahead is shown as an EBNF comment, and rules read without skipping whitespace are marked `(* token *)`. The hand-written EBNF in the Datalog reference stays for v2.x only.
- **Durability, recovery and operations guide** (#398). `docs/DURABILITY.md`, also published on the docs site, covers what survives a process crash and a power loss under `SyncMode::Full` and `SyncMode::Normal`, what opening after a crash does and what it costs, which filesystems and container setups are supported, how to back up a live database (offline copy, volume snapshot, or `fact_log` into `LogWriter` from the running process) and restore it, how to check and repair a file with `verify()` and `rebuild_indexes()`, how to upgrade from v2.x and roll back, which storage errors mean retry, reopen or restore, and the known limitations. Writing it found #535: a `.wal` from a later checkpoint is replayed onto an older `.graph` without an error, so the restore steps remove any `.wal` first. A log-writer test runs the documented online backup while another thread writes past the automatic checkpoint threshold.

### Known issues
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**1650 tests** (1640 passing, 10 ignored; unit + integration + doc).
**1659 tests** (1649 passing, 10 ignored; unit + integration + doc).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

**Testing conventions** — see the Testing Conventions section below before writing any tests.
Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ web-sys = { version = "0.3", optional = true, features = [
serde_json = "1.0"
pest = "2"
pest_derive = "2"
pest_meta = "2"

[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
tempfile = "3"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit

```bash
cargo run # interactive Datalog REPL
cargo test # run 1650 tests
cargo test # run 1659 tests
cargo run < demos/demo_recursive.txt # recursive rules demo
```

Expand Down
127 changes: 127 additions & 0 deletions docs/GRAMMAR.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Datalog Grammar

<!-- Generated from tests/grammar/grammar.pest by tests/grammar_doc.rs. Do not edit: change the grammar, then run UPDATE_GRAMMAR_DOC=1 cargo test --test grammar_doc -->

This EBNF is the structural syntax of Minigraf's Datalog commands. It is generated from `tests/grammar/grammar.pest`, the grammar the parser is tested against: for every input up to a size bound, a test checks that the parser rejects what the grammar rejects, and accepts what it accepts unless a semantic rule rejects it. To change this page, edit the grammar and run `UPDATE_GRAMMAR_DOC=1 cargo test --test grammar_doc`.

Between tokens the parser skips spaces, tabs, newlines, carriage returns and commas. Inside a rule marked `(* token *)` nothing is skipped, and the rules it uses are read the same way.

EBNF has no lookahead, so a pest lookahead appears as a comment: `(* not followed by x *)` means the next characters must not match `x`, and nothing is read. Literals have no escapes: `"\"` is one backslash and `'"'` one double quote.

```ebnf
(* ── WORD-BOUNDARY HELPER ── *)
word-cont ::= letter | digit | "?" | "_" | "-" | "/"
kw-find ::= ":find" (* not followed by word-cont *) (* token *)
kw-where ::= ":where" (* not followed by word-cont *) (* token *)
kw-with ::= ":with" (* not followed by word-cont *) (* token *)
kw-as-of ::= ":as-of" (* not followed by word-cont *) (* token *)
kw-valid-at ::= ":valid-at" (* not followed by word-cont *) (* token *)
kw-any-valid-time ::= ":any-valid-time" (* not followed by word-cont *) (* token *)
kw-max-derived-facts ::= ":max-derived-facts" (* not followed by word-cont *) (* token *)
kw-max-results ::= ":max-results" (* not followed by word-cont *) (* token *)
kw-over ::= ":over" (* not followed by word-cont *) (* token *)
kw-partition-by ::= ":partition-by" (* not followed by word-cont *) (* token *)
kw-order-by ::= ":order-by" (* not followed by word-cont *) (* token *)
kw-desc ::= ":desc" (* not followed by word-cont *) (* token *)
kw-asc ::= ":asc" (* not followed by word-cont *) (* token *)
kw-valid-from ::= ":valid-from" (* not followed by word-cont *) (* token *)
kw-valid-to ::= ":valid-to" (* not followed by word-cont *) (* token *)
reserved-head ::= ("not-join" | "not" | "or-join" | "or") (* not followed by word-cont *) (* token *)
and-head ::= "and" (* not followed by word-cont *) (* token *)

(* ── EDN PRIMITIVES ── *)
boolean-lit ::= ("true" | "false") (* not followed by word-cont *) (* token *)
nil-lit ::= "nil" (* not followed by word-cont *) (* token *)
integer-lit ::= "-"? digit+ (* not followed by "." digit *) (* token *)
float-lit ::= "-"? digit+ "." digit* (* token *)
str-char ::= "\" ("n" | "t" | "r" | '"' | "\" | any-char)
| (* not followed by '"' | "\" *) any-char
string-lit ::= '"' str-char* '"' (* token *)
uuid-lit ::= "#uuid" string-lit
bind-slot ::= "$" (letter | digit | "_" | "-")+ (* token *)
keyword ::= ":" word-cont+ (* token *)
variable ::= "?" word-cont* (* token *)
op-sym ::= "<=" | ">=" | "!=" | "<" | ">" | "=" | "+" | "*" | "/" (* token *)
plain-sym ::= (* not followed by boolean-lit *) (* not followed by nil-lit *) (letter | "_") word-cont*
| "-" word-cont+ (* token *)

(* ── EDN VALUE (any EDN atom or container) ── *)
edn-value ::= uuid-lit | boolean-lit | nil-lit | float-lit | integer-lit | string-lit
| bind-slot | keyword | op-sym | variable | plain-sym | edn-list
| edn-vector | edn-map
edn-list ::= "(" edn-value* ")"
edn-vector ::= "[" edn-value* "]"
edn-map ::= "{" (edn-value edn-value)* "}"

(* ── TOP-LEVEL COMMAND ── *)
command ::= transact-cmd | retract-cmd | query-cmd | rule-cmd

(* ── TRANSACT ── *)
transact-cmd ::= "(" "transact" (valid-time-map fact-vector | fact-vector) ")"

(* ── RETRACT ── *)
retract-cmd ::= "(" "retract" "[" retract-fact* "]" ")"
fact-vector ::= "[" fact* "]"
fact ::= "[" edn-value edn-value edn-value valid-time-map? "]"
retract-fact ::= "[" edn-value edn-value edn-value "]"
valid-time-map ::= "{" valid-time-pair* "}"
valid-time-pair ::= (kw-valid-from | kw-valid-to) string-lit

(* ── QUERY ── *)
query-cmd ::= "(" "query" query-vector ")"
query-vector ::= "[" query-section+ "]"
query-section ::= find-section | where-section | as-of-section | valid-at-section
| any-valid-time-section | with-section | max-derived-facts-section
| max-results-section

(* ── FIND SECTION ── *)
find-section ::= kw-find find-spec+
find-spec ::= window-expr | aggregate-expr | variable
aggregate-expr ::= "(" (op-sym | plain-sym) variable ")"
window-expr ::= "(" (op-sym | plain-sym) (variable kw-over | kw-over) over-clause ")"
over-clause ::= "(" over-option* ")"
over-option ::= kw-partition-by variable | kw-order-by variable | kw-desc | kw-asc

(* ── WHERE SECTION ── *)
where-section ::= kw-where where-clause+
where-clause ::= expr-clause | not-clause | not-join-clause | or-clause
| or-join-clause | pattern-clause | rule-invocation
pattern-clause ::= "[" edn-value edn-value edn-value "]"
expr-clause ::= "[" expr variable? "]"
not-clause ::= "(" "not" where-clause+ ")"
not-join-clause ::= "(" "not-join" join-vars where-clause+ ")"
or-clause ::= "(" "or" or-branch+ ")"
or-join-clause ::= "(" "or-join" join-vars or-branch+ ")"
join-vars ::= "[" variable* "]"
or-branch ::= and-branch | (* not followed by "(" and-head *) where-clause
and-branch ::= "(" "and" where-clause+ ")"
rule-invocation ::= "(" (* not followed by reserved-head *) plain-sym edn-value* ")"

(* ── EXPRESSIONS ── *)
expr ::= "(" expr-body ")"
expr-body ::= binary-form | unary-form
unary-form ::= (unary-op | op-sym | plain-sym) expr-arg
binary-form ::= binary-op expr-arg expr-arg
unary-op ::= "string?" | "integer?" | "float?" | "boolean?" | "nil?" (* token *)
binary-op ::= "<=" | ">=" | "!=" | "starts-with?" | "ends-with?" | "contains?"
| "matches?" | "<" | ">" | "=" | "+" | "-" | "*" | "/" (* token *)
expr-arg ::= expr | boolean-lit | nil-lit | float-lit | integer-lit | string-lit
| bind-slot | keyword | variable

(* ── TEMPORAL / WITH SECTIONS ── *)
as-of-section ::= kw-as-of (integer-lit | string-lit | bind-slot)
valid-at-section ::= kw-valid-at (string-lit | kw-any-valid-time | bind-slot)
any-valid-time-section ::= kw-any-valid-time
with-section ::= kw-with variable+
max-derived-facts-section ::= kw-max-derived-facts integer-lit
max-results-section ::= kw-max-results integer-lit

(* ── RULE ── *)
rule-cmd ::= "(" "rule" rule-vector ")"
rule-vector ::= "[" rule-head where-clause* "]"
rule-head ::= "(" plain-sym variable* ")"

(* letter = "a".."z" | "A".."Z"; digit = "0".."9"; any-char = any one character *)
```

Input that matches this grammar can still be rejected: the parser also enforces the rules in the Semantic Constraints section of the Datalog reference (variable safety, binding, aggregate and window compatibility, UUID and timestamp formats).
3 changes: 2 additions & 1 deletion docs/TEST_COVERAGE.md

Large diffs are not rendered by default.

Loading
Loading