Embedded bi-temporal graph database for Java/Kotlin — Datalog queries, time travel, fat JAR with embedded natives

Minigraf for Java and Kotlin on the desktop JVM. Fat JAR with embedded native libraries. No Rust toolchain required.

On any other platform, the first call fails with an UnsupportedOperationException naming the detected OS and architecture.

dependencies {
    implementation("io.github.project-minigraf:minigraf-jvm:2.0.8")
}

<dependency>
    <groupId>io.github.project-minigraf</groupId>
    <artifactId>minigraf-jvm</artifactId>
    <version>2.0.8</version>
</dependency>

import io.github.project_minigraf.minigraf.MiniGrafDb
import org.json.JSONObject

// File-backed database
val db = MiniGrafDb.open("/path/to/myapp.graph")

// In-memory database (ephemeral / testing)
val mem = MiniGrafDb.openInMemory()

// Transact facts
db.execute("""(transact [[:alice :person/name "Alice"] [:alice :person/age 30]])""")

// Query with Datalog
val json = JSONObject(db.execute(
    "(query [:find ?name ?age :where [?e :person/name ?name] [?e :person/age ?age]])"
))
// json.getJSONArray("results").getJSONArray(0).getString(0) == "Alice"

// Time travel — state as of transaction 1
val snap = db.execute("(query [:find ?age :as-of 1 :where [:alice :person/age ?age]])")

// Flush dirty pages to disk
db.checkpoint()

Use the Minigraf facade for static factory methods. Without it, Java has to go through the Kotlin companion object (MiniGrafDb.Companion.open(path)).

import io.github.project_minigraf.minigraf.Minigraf;
import io.github.project_minigraf.minigraf.MiniGrafDb;

MiniGrafDb db = Minigraf.open("/path/to/myapp.graph");
String result = db.execute("(transact [[:alice :person/name \"Alice\"]])");
db.checkpoint();

The bindings are generated Kotlin, so kotlin-stdlib is a transitive runtime dependency and ends up on Java classpaths too. JPMS users can require the automatic module io.github.project_minigraf.minigraf.

import io.github.project_minigraf.minigraf.*;

// Read-only: shared lock, nothing written; writes fail with API-014.
MiniGrafDb src = Minigraf.openWithOptions("old.graph",
        Minigraf.options().readOnly(true).pageCacheSize(4096).build());

// A cursor's answer is fixed when it opens. Each batch is a JSON array of rows,
// encoded like execute()'s "results".
try (MiniGrafCursor cursor = src.query("(query [:find ?n :where [?e :name ?n]])")) {
    for (String batch : Minigraf.batches(cursor, 1000)) { /* parse rows */ }
}

// Copy every fact version, keeping tx and valid-time bounds, into a new file.
try (MiniGrafFactLog log = src.factLog(Minigraf.filter().build());
     MiniGrafLogWriter out = Minigraf.createLogWriter("new.graph", Minigraf.options().build())) {
    List<FactRecord> batch = new ArrayList<>();
    for (FactRecord rec : Minigraf.records(log, 1000)) {
        if (!rec.getAttribute().startsWith(":secret/")) batch.add(rec);
    }
    out.appendBatch(batch);
    out.advanceTxCount(src.currentTxCount());
    out.finish();
}   // close() without finish() abandons the build and leaves no file

Minigraf.options().queryMemoryLimit(bytes) caps the bytes one query may hold, counted per entry by the operators that keep state (join build sides, negation and or tables, distinct sets, aggregate groups, window rows, rule-derived facts). A query that would hold more fails with QRY-012 and frees what it held. The default, and Long.MAX_VALUE, is no limit. Rows a cursor returns are not counted, so read a large answer with query().

Query text may override some settings with keywords such as :max-derived-facts 1000. allowOverrides(...) lists the Overridable keywords it may use (MAX_DERIVED_FACTS, MAX_RESULTS, QUERY_MEMORY_LIMIT, MAX_ITERATIONS). Not calling it keeps the defaults, Minigraf.OVERRIDABLE_DEFAULTS, read from the core library (today MAX_DERIVED_FACTS, MAX_RESULTS and MAX_ITERATIONS). A list replaces the defaults, it does not add to them, and allowOverrides() with no arguments refuses every keyword: a query that uses one throws MiniGrafException.Api with [API-025] from execute, query and prepare.

List<Overridable> allowed = new ArrayList<>(Minigraf.OVERRIDABLE_DEFAULTS);
allowed.add(Overridable.QUERY_MEMORY_LIMIT);   // the defaults plus :query-memory-limit
MiniGrafDb db = Minigraf.openWithOptions("app.graph", Minigraf.options()
        .allowOverrides(allowed)
        .maxIterations(10_000)            // rounds per stratum of a rule query, else INT-020
        .transactionChunkFacts(16_384)    // facts staged before a large transaction is chunked
        .build());

maxIterations defaults to 1,000,000 and Long.MAX_VALUE is no limit; transactionChunkFacts defaults to 65,536 and is clamped to 1..=1,000,000. A negative count is [API-017].

Kotlin callers can use cursor.batches() and log.records() directly, and the generated constructors with named arguments (OpenOptions(readOnly = true)).

All UniFFI objects are AutoCloseable: close() frees the native object. The shim's own close methods are named release() (cursor, fact log) and abandon() (log writer) here, because close() is taken.

A record's value is a MiniGrafValue (Text, Int64, Float64, Bool, Ref, Keyword, Null), so a ref and a string stay different. Minigraf.VALID_TIME_FOREVER is the validTo of a fact valid forever. Errors are MiniGrafException; Minigraf.errorMessage(e) gives the text starting with its code, such as [API-015].

db.prepare(text) parses a query once; run it with its $slots bound (names without the $). Each run reads the database as it is then, and a prepared query keeps its database open until both are closed.

MiniGrafPreparedQuery pq = db.prepare("(query [:find ?n :where [?e :age $age] [?e :name ?n]])");
String thirty = Minigraf.execute(pq, Map.of("age", 30));
try (MiniGrafCursor cursor = Minigraf.query(pq, Map.of("age", 40))) { /* batches */ }

val thirty = pq.execute("age" to 30)
pq.query("age" to 40).use { cursor -> cursor.batches().forEach { /* rows */ } }

Boolean, Byte/Short/Int/Long, Float/Double, String, null and java.util.UUID (an entity, which also fits value slots as a ref) bind as they are. For the other kinds pass a MiniGrafValue (Keyword, Ref, a whole-number Float64), or Minigraf.entity(uuid), Minigraf.txCount(n) / Minigraf.timestamp(ms) for :as-of, and Minigraf.ANY_VALID_TIME for :valid-at. Any other type throws IllegalArgumentException. A missing slot is MiniGrafException.Api with [API-022], a slot in an attribute position [API-023], and a value of the wrong kind [API-024].

A Set binds a set value, for in? and the other set predicates or a set-valued attribute: pq.execute("ids" to setOf(1, 2, 3)), or Set.of(1, 2, 3) from Java. Its elements are the plain values above or a MiniGrafValue (MiniGrafValue.Keyword(":role/admin")), and a UUID element is a ref. A set inside a set is MiniGrafException.Api with [API-021]; another kind of element, or a List, throws IllegalArgumentException. Set values come back as JSON arrays from execute and as MiniGrafValue.Set in fact records.

Errors are MiniGrafException.Parse, .Query, .Storage, .Api (a bad argument, a closed object, a bind mistake) or .Other.

withMaxDerivedFacts(n), withQueryMemoryLimit(bytes) and withMaxIterations(n) each return a new prepared query with that limit; the one they are called on is unchanged. They are application code, so allowOverrides does not apply to them, and an allowed keyword in the query text still overrides them. Long.MAX_VALUE means no limit and a negative value is [API-017]. A derived query keeps its database open, as any prepared query does.

MiniGrafPreparedQuery reach = db.prepare("(query [:find ?x ?y :where (reach ?x ?y)])");
MiniGrafPreparedQuery small = reach.withMaxDerivedFacts(10_000).withQueryMemoryLimit(16 << 20);
Minigraf.execute(small, Map.of());   // INT-020 or QRY-012 past a limit; reach is unchanged

A predicate implements MiniGrafPredicate (test(value): Boolean) and is called in Datalog as [(name ?v)]. An aggregate implements MiniGrafAggregate; its accumulator() returns a new MiniGrafAccumulator (step(value), finish(count)) for each group or window partition, and it is used as (name ?v) in :find or with :over. Values arrive as MiniGrafValue (Int64, Text, …, with the value in value), and finish returns one.

object Adult : MiniGrafPredicate {
    override fun test(value: MiniGrafValue): Boolean = when (value) {
        is MiniGrafValue.Int64 -> value.value >= 18
        else -> throw MiniGrafUdfException.Failed("age is not an integer")
    }
}

object Product : MiniGrafAggregate {
    override fun accumulator(): MiniGrafAccumulator = object : MiniGrafAccumulator {
        private var product = 1L
        override fun step(value: MiniGrafValue) { product *= (value as MiniGrafValue.Int64).value }
        override fun finish(count: Long): MiniGrafValue = MiniGrafValue.Int64(product) // count: values stepped
    }
}

db.registerPredicate("adult?", Adult)
db.registerAggregate("product", Product)
db.execute("(query [:find ?n :where [?e :age ?a] [(adult? ?a)] [?e :name ?n]])")
db.execute("(query [:find ?g (product ?a) :where [?e :group ?g] [?e :age ?a]])")
db.execute("(query [:find ?a (product ?a :over (:order-by ?a)) :where [?e :age ?a]])")

From Java the interfaces take lambdas where they have one method:

db.registerPredicate("adult?", v -> {
    if (!(v instanceof MiniGrafValue.Int64)) throw new IllegalArgumentException("age is not an integer");
    return ((MiniGrafValue.Int64) v).getValue() >= 18;
});
db.registerAggregate("product", () -> new MiniGrafAccumulator() {
    private long product = 1;
    @Override public void step(MiniGrafValue v) { product *= ((MiniGrafValue.Int64) v).getValue(); }
    @Override public MiniGrafValue finish(long count) { return new MiniGrafValue.Int64(product); }
});

• Failures. A callback that throws stops the query with QRY-014, raised as MiniGrafException.Query; it is never read as false or a missing value. From Kotlin, throw MiniGrafUdfException.Failed(msg) to choose the message. Any other exception also fails the call, with its first line (class and message, such as java.lang.IllegalArgumentException: bad) as the message. The interfaces declare no checked exception, so Java throws an unchecked one. A MiniGrafValue.Ref from finish whose value is not a UUID is a failing call too.
• Lifecycle. The database keeps a strong reference to each registered object until the database and every cursor and prepared query opened from it are closed. There is no unregister. Predicates and aggregates have separate names, and registering a name twice in either is an error (INT-040). A query sees the functions registered when it opened; a prepared query, those registered when each run starts.
• Threading. A callback runs on the thread that calls execute (of the database or a prepared query) or a cursor's nextBatch, including a prepared query's cursor. Queries on different threads may call the same object at once, so it must be thread-safe. One accumulator is used by one query on one thread at a time.
• Cost. Each call crosses the FFI once: one JVM call per value.
• Re-entry. A callback may query the same database, open a cursor or register another function. Inside a write transaction, the calls the core refuses there still return INT-001.

A write transaction stages several transact, retract and query statements, with queries seeing what it staged, and then commits them as one transaction or rolls them back (minigraf#824). First in the release built on core v3.0.0 (core commit 32d71ce0).

val db = MiniGrafDb.open("bank.graph")
db.execute("(transact [[:alice :balance 100] [:bob :balance 50]])")

// use { } closes the transaction on exit, which rolls it back unless it was
// committed, also when an exception leaves the block.
db.beginWrite(TxOptions(chunkFacts = 10_000)).use { tx ->
    tx.execute("(transact [[:alice :balance 70] [:bob :balance 80]])")
    tx.execute("(retract [[:alice :balance 100] [:bob :balance 50]])")
    val staged = tx.execute("(query [:find ?b :where [:alice :balance ?b]])")
    if (!staged.contains("-")) tx.commit()
}

From Java, beginWriteWith takes the options from a builder, and try-with-resources rolls back unless committed:

try (MiniGrafWriteTransaction tx = db.beginWriteWith(Minigraf.txOptions().chunkFacts(10_000).build())) {
    tx.execute("(transact [[:alice :balance 70]])");
    tx.commit();
}

• db.beginWrite() uses the database's options; db.beginWrite(options) (Kotlin) or db.beginWriteWith(options) (Java, with Minigraf.txOptions()) takes TxOptions. An option left null is the database's: synchronous (SyncMode.FULL / NORMAL) is this commit's WAL sync, chunkFacts the facts staged before a file database writes the transaction in chunks (negative is API-017). Read-only databases throw API-014.
• tx.execute(datalog) returns what db.execute returns; a staged write returns {"ok":true}. A statement that fails stages nothing and leaves the transaction open. There are no cursors or prepared queries inside a transaction.
• tx.commit() applies everything as one transaction; a commit that fails rolls back. tx.rollback() discards it. Either way it is finished: execute and commit then throw API-018, and rollback does nothing.
• Commit is always explicit. close() (and use { }, try-with-resources) frees the transaction and rolls it back unless it committed; after close() the object cannot be used. The garbage collector may never free an abandoned one, so do not rely on it.
• An open transaction holds the database's write lock until it finishes, so end every transaction explicitly or through use { } or try-with-resources. On the thread that began it, database calls (execute, query, prepare, factLog, checkpoint, another beginWrite, prepared queries) fail with INT-001, and so do user functions its statements run. On other threads, writes and begins wait until it ends; queries read the committed state. Use a transaction on the thread that began it: a thread that uses one it did not begin and then writes through the database waits on that transaction.
• Rules registered inside a transaction stay registered after a rollback.
• The transaction keeps its database open until it is closed.

• Full Java/JVM integration guide
• Repository
• Datalog Reference

MIT OR Apache-2.0