Skip to content

Use case: decision telemetry (diff two runs to find where they diverged) #26

Closed

Description

@adityamukho

Summary

Turn the decision telemetry project (project-minigraf/decision_telemetry) into a use-case write-up in content/use-cases.md.

The project may not grow past its proof of concept. The write-up should stand on its own, so others can build the same idea in their own language or stack.

Note: the repo is private today. Either make it public before the doc links to it, or describe the idea without linking.

The idea

Decision telemetry records which way code went at each if, ternary and switch, and the values that drove each choice. Each run is stored in a Minigraf file. A diff of two runs shows the first point where they parted ways, and which input changes came before it.

It is aimed at code nobody on the team knows well, such as large amounts of AI-generated code. An SRE should be able to find the decision that went wrong without reading the code.

The README demo shows the value well: the same checkout runs twice, only a background lookup's latency differs, nothing fails, and the totals differ. trail diff points to the cache lookup that missed and to the latency change as the likely cause.

Why Minigraf fits

  • One file per run matches Minigraf's one-file-per-unit model. Runs are portable and easy to attach to a bug report.
  • Graph structure: call paths, decision nodes and the values they read form a graph. "Why did this branch go this way?" is a traversal.
  • Datalog makes run diffs and "why" slices queryable instead of hard-coded.
  • Bi-temporal: the code version (trail.codeVersion) and run time can be stored with each run, so you can ask how a decision behaved across releases.
  • Language-neutral: runs follow a shared schema (docs/SCHEMA.md), and any language with Minigraf bindings could produce them.

What the doc should cover

  • The problem: silent wrong results in unfamiliar code, where nothing throws
  • The core concept: record decisions and their inputs, then diff two runs
  • The demo output from the README, trimmed
  • A short sketch of the fact schema (runs, decision nodes, values, call path), based on docs/SCHEMA.md
  • One or two Datalog queries over a stored run (for example, first divergence, or all decisions that read a given value)
  • Design points worth copying: keep a run only on exception, anomaly or sample; redact sensitive values; carry the run across threads; cap nodes per run
  • Status: Java proof of concept only, idea open for others to build
  • Where it goes: the same new section proposed in Use case: history store for formal verification results #25 for developer tooling ideas

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions