bug: EavtKey and AevtKey missing value bytes causes silent data loss when batching multi-valued facts

Bug Description

EavtKey (in src/storage/index.rs:100-108) and AevtKey (lines 110-118) use (Entity, Attribute, ValidFrom, ValidTo, TxCount) as the key — they do not include the encoded value bytes. AvetKey (lines 120-131) correctly includes value_bytes.

Consequence

When a single (transact [...] ...) call includes multiple facts for the same entity+attribute (e.g., [:module/A :contains :fn1], [:module/A :contains :fn2], [:module/A :contains :fn3]), all facts share the same tx_count (the batch increments tx_count once). Since they also share entity, attribute, valid_from, and valid_to, their EavtKey is identical.

The BTreeMap::insert() in Indexes::insert() (lines 169-217) overwrites the previous entry with the same key, so only the last fact survives in the EAVT and AEVT pending indexes. The facts DO reach d.facts (the canonical fact list), but only one is indexed.

Query Impact

Queries like [:find ?v :where [:module/A :contains ?v]] use the EAVT index to find facts by entity+attribute. With only one EAVT entry, only one contained entity is returned even though N were transacted.

Queries via the AVET index work correctly (since AVET includes value_bytes), but the standard entity-attribute lookup path uses EAVT.

Workaround

Transact each multi-valued fact individually so each gets a unique tx_count. This is the current workaround in the temporal-reasoning mcp_server ingestion code.

Reproducible Test

// In a test, transact two facts same entity+attribute in one batch:
let triples = "[[:test/entity :multi-val :val1] [:test/entity :multi-val :val2]]";
db.execute(&format!("(transact {triples})")).unwrap();
// Query should return both values
let result = db.execute("(query [:find ?v :where [:test/entity :multi-val ?v]])").unwrap();
// Bug: result only contains one of the two values

Proposed Fix

Add value_bytes: Vec<u8> (encoded via encode_value) to EavtKey and AevtKey.
Update Indexes::insert() accordingly.
Update range query lower/upper bounds — use vec![] as the minimum value_bytes bound.
Assess whether EavtKey/AevtKey are stored on disk in the B+tree; if so, this may require a file format version bump (v7 → v8).

Impact

Silent data loss when batching multi-valued facts for the same entity+attribute in a single transact call. No error or warning is raised — only one value is queryable via EAVT-backed paths, making this a correctness bug.

Independent confirmation on v1.1.1 (crates.io).

We hit this while evaluating minigraf as a bitemporal belief-ledger substrate, and can confirm the silent collapse with a minimal repro:

use minigraf::{Minigraf, QueryResult};

fn main() -> anyhow::Result<()> {
    let db = Minigraf::in_memory()?;
    db.execute(r#"(transact {:valid-from "2025-01-01T00:00:00Z" :valid-to "2026-01-01T00:00:00Z"}
        [[:probe/A :contains :v1] [:probe/A :contains :v2]])"#)?;
    let rows = match db.execute(r#"(query [:find ?v :any-valid-time :where [:probe/A :contains ?v]])"#)? {
        QueryResult::QueryResults { results, .. } => results.len(),
        other => anyhow::bail!("unexpected: {other:?}"),
    };
    println!("rows = {rows}"); // rows = 1 (only :v2) — expected 2
    Ok(())
}

Observations from a slightly wider battery (same entity+attribute in one transact: N=2, N=3, N=10; mixed value types keyword/string/integer; per-fact distinct valid windows), all on v1.1.1:

Entity-bound ([:probe/A :contains ?v]) and attribute-bound ([?e :contains ?v]) queries return one row per (entity, attribute) — only the last value in the transaction batch is visible (last-write-wins display).
The loss appears to be query-surface, not storage: :as-of full scans still return all inserted values, and the collapsed behavior persists unchanged across file close/reopen.
A user can (retract ...) the invisible value: the call succeeds and the visible result stays unchanged — silent in both directions (the lost fact can't be seen, and touching it gives no signal).
Control: the same facts inserted one-fact-per-transaction are all visible, consistent with the key-shape analysis in the report (distinct tx_count disambiguates EAVT/AEVT keys).

Happy to share the full reproduction harness if useful.

Thanks for your independent assessment, and for giving minigraf a try!

This is a known bug, and its fix will almost certainly require a file format change. I have, therefore, marked this for the 2.0 release. In the meantime, as you have already figured out, one workaround is to transact one fact at a time so that their distinct TX counts would act as the discriminator.

The fix will, no doubt, include test cases to catch regressions of this sort in the future; so if you think your harness contains something usable for that purpose, you're welcome to share it 🙂

Contributions in the form of bug reports, PRs, feature requests (discretionary) or discussions are always welcome!

fix: work around minigraf 1.2.0 regressions in lessons and e2e tests

fix: keep STG-003/STG-004 registered as deprecated instead of removin…

fix!: file format v8 — keep same-transaction multi-valued facts (#371, …

Where this fix lives, and when it ships

The fix (#377) changes the index keys and so moves the file format from v7 to v8. That is a one-way migration and a breaking change. To keep main releasable as v2.x, the work is being moved to a long-lived v3 branch:

v3 contains the fix unchanged, at b8a9762. It ships in v3.0.0.
main is the v2.x line. revert: move file format v8 (#377) from main to the v3 branch #378 reverts fix!: file format v8 — keep same-transaction multi-valued facts (#371, #287) #377 there, so v2.1–v2.3 and the bug: after a kill during an active run the EAVT index silently loses most entities' entries while AEVT stays intact — entity-bound lookups return [] and no rebuild path exists #370 fix can ship without forcing the v8 migration.
v3 merges main regularly and merges into main when v3.0.0 is cut.

Until v3.0.0: v2.x still has this bug. Workaround: write each value of a multi-valued attribute in its own transact. Don't batch several values of the same entity + attribute into one call. The same applies to retracting several values of one attribute.

The issue stays closed because the fix is merged, on v3. It will appear in the v3.0.0 CHANGELOG.

revert: move file format v8 (#377) from main to the v3 branch (#378)

release: v2.0.1 — fix index rebuild on open (#370)

Reopened under the known-issues rule in #399: a data-integrity bug stays open until its fix is in a published release. The fix (#377) was moved from main to the v3 branch (#378), so every release through v2.0.1 still has this bug. Tracked with #371; the fix ships in v3.0.0. Affected versions, workaround and fix version: #421.

Fixed on v3 for v3.0.0 by file format v8: index keys carry the encoded value and the assert/retract flag (PR #377, carried through the v8 storage format in #449), so same-transaction multi-valued facts and batched retracts read back correctly on every query path. Regression tests: tests/multi_value_test.rs. A v7 file is re-indexed on its first open under v3.0.0. v2.x, including v2.0.3, is still affected and stays listed in #421 until v3.0.0 ships.

Navigation Menu

Repository navigation

bug: EavtKey and AevtKey missing value bytes causes silent data loss when batching multi-valued facts #287

Bug Description

Consequence

Query Impact

Workaround

Reproducible Test

Proposed Fix

Impact

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

bug: EavtKey and AevtKey missing value bytes causes silent data loss when batching multi-valued facts #287

Description

Bug Description

Consequence

Query Impact

Workaround

Reproducible Test

Proposed Fix

Impact

Activity

etc-sw commented on Jun 4, 2026

adityamukho commented on Jun 7, 2026

adityamukho commented on Sep 25, 2026

adityamukho commented on Sep 27, 2026

adityamukho commented on Oct 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions