Skip to content

test: model-based (stateful) test of transact/retract/checkpoint/reopen against a reference model #385

Closed

Description

@adityamukho

Problem

No test runs random sequences of operations against a file-backed database and compares every result with a simple reference model. The storage and index bugs of the last three months only appear after sequences like these:

Ask

Add a stateful property test, for example with proptest-state-machine. It keeps an in-memory Vec<Fact> as the reference model.

  • Operations: transact (including multi-valued batches), retract (including batched retracts), WriteTransaction with several valid-time windows, checkpoint, drop and reopen, and reopen with a small page cache.
  • Checks after each step:
    • entity-bound, attribute-bound, value-bound and full-scan queries
    • :as-of N for random N
    • :valid-at T for random T
    • All of these must match the model.
  • Backend: file-backed, in a temp dir, so the on-disk B+tree and the packed pages are exercised.
  • Nightly job: add it to the property-test job with a high case count, and keep a small case count in normal CI.

The minimized cases this test finds should become regression tests.

Activity

  1. added this to the v3.0.0 milestone on Sep 26, 2026
  2. added a commit that references this issue on Oct 7, 2026
  3. adityamukho commented on Oct 7, 2026

    @adityamukho
    CollaboratorAuthor

    Done on v3 by #475 (merge commit 19b0765): tests/model_based_test.rs runs 24 cases in normal CI. The nightly property-tests.yml runs 30,000 cases once v3 becomes main; until then, start it with gh workflow run property-tests.yml --ref v3. The test found no failures in its target paths. It did surface two behaviours, filed as #476 and #477.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions