diff --git a/OPTIONS.md b/OPTIONS.md new file mode 100644 index 0000000..15a4908 --- /dev/null +++ b/OPTIONS.md @@ -0,0 +1,90 @@ +# jev-router reference + +Commands, options, presets and privacy. For the overview, benchmarks and install, see the [README](README.md). + +## Presets + +| Preset | ceiling | effort cap | mode | for | +| --- | --- | --- | --- | --- | +| `lean` (default) | sonnet (Opus only at 90% confidence) | medium | balanced | most work: small tasks on Haiku, the rest on Sonnet | +| `balanced` | none | high | balanced | the 0.2 and 0.3 behaviour | +| `max` | none | none | efficient | when quality matters more than cost | + + +`/jev preset ` switches for the session; the `preset` option sets it permanently. `/jev ceiling`, `/jev cap` and the other commands still override one setting. The default is `lean` (the shipped option values are the lean ones; CI checks that they match). + +## The band + +| Part | Meaning | +| --- | --- | +| `balanced` | Routing mode: `efficient`, `balanced` or `cheap` | +| `sonnet ▂▄__` | The model the task runs on, with its tier. Green haiku, blue sonnet, purple opus, gold fable | +| `medium ▰▰▱▱▱` | Reasoning effort, `low` to `max` | +| `conf ▮▮▮▮▯ 88%` | How sure Jev is. Green from 70%, amber from 50%, red below | +| `cache 🔒 warm 42k` | Whether the prompt cache is still worth protecting, and how big it is | + + +When the cache wins, the band says so: `opus (kept 🔒 cache warm; wanted haiku)`. A model that differs from the pick reads `sonnet (picked opus)`; `⤓xhigh` means the effort cap lowered Jev's effort; `🔓` means the prompt ran uncapped; `↑` escalation; `📌` pinned; `⤒opus` clamped by the ceiling. + +## Commands + +| Command | What it does | +| --- | --- | +| `/jev` or `/jev status` | Settings, cache state, last decision and per-model usage | +| `/jev efficient \| balanced \| cheap` | Set the routing mode, from the next task | +| `/jev sticky off \| auto \| strict` | Set cache stickiness | +| `/jev cap low \| medium \| high \| xhigh \| max \| none` | Set the effort cap | +| `/jev preset lean\|balanced\|max` | Switch the ceiling, effort cap and mode together | +| `/jev ceiling ` | Never go above this model unless Jev is very sure | +| `/jev floor ` | Never go below this model | +| `/jev full` | Lift the effort cap for the next prompt only | +| `/jev stats clear` | Reset the savings tally | +| `/jev on` / `/jev off` | Enable routing / use the session model | +| `/jev key ` | Store the TypeSafe API key | + +``` +┌ Jev router ────────────────────────────┐ +│ state ● on mode balanced │ +│ pool haiku · sonnet · opus · fable │ +│ sticky auto cache ● warm │ +│ last sonnet / medium ▮▮▮▮▯ 0.88 │ +│ usage haiku ░░░░░░░░░░ 0 0% │ +│ sonnet ██████████ 1 100% │ +│ opus ░░░░░░░░░░ 0 0% │ +│ fable ░░░░░░░░░░ 0 0% │ +│ key …fd1c (TYPESAFE_API_KEY) │ +└────────────────────────────────────────┘ +``` + + +## Options + +| Option | Default | Meaning | +| --- | --- | --- | +| `mode` | `balanced` | Routing mode | +| `models` | all four | Aliases Jev may choose from | +| `preset` | `none` | `lean`, `balanced` or `max` overrides the three options below. `none`: the options apply as set (the shipped defaults are the `lean` values) | +| `effortCap` | `medium` | Highest effort the router applies. `none` follows Jev. Per prompt: start with `!full` or run `/jev full` | +| `ceiling` | `sonnet` | Highest model Jev may pick unless it is `ceilingBreak` sure (`haiku`, `sonnet`, `opus`, `fable`) | +| `floor` | `none` | Lowest model Jev may pick | +| `ceilingBreak` | `0.9` | Confidence needed to exceed the ceiling | +| `pauseMs` | `60000` | Stop asking Jev for this long after 3 failed calls in a row | +| `escalateAfter` | `3` | Failed tool calls in a row before the task moves up one model. `0` is off | +| `baselineModel` | `opus` | The model the savings estimate compares against | +| `sendHistory` | `true` | Send recent messages to Jev. Off sends only the task text | +| `redact` | `true` | Replace API keys, tokens, private keys and `KEY=value` lines with `[redacted]` before sending | +| `maxTaskChars` | `8000` | How much of the prompt is sent | +| `fallback` | `session` | When Jev is down: `session` keeps the session model, `heuristic` guesses locally | +| `routeSteps` | `true` | Ask Jev again before every step after the first. Off: one call per prompt | +| `routeSubagents` | `true` | Pick a model and effort for each subagent from its description | +| `stickiness` | `auto` | `off`: always follow Jev. `auto`: while the cache is warm, only upgrade on high confidence. `strict`: never switch while warm | +| `minConfidence` | `0.7` | Confidence needed for a warm-cache upgrade | +| `minContextTokens` | `8000` | Below this a switch is free and stickiness is skipped | +| `cacheTtlMs` | `300000` | Time after the last request when the cache counts as cold | +| `timeoutMs` | `4000` | Keep the current model when Jev takes longer | + + +## Privacy + +Sent to `api.typesafe.ai`: the prompt text (up to `maxTaskChars`), recent messages if `sendHistory` is on, and subagent descriptions. Before sending, `redact` replaces API keys, GitHub tokens, AWS keys, bearer tokens, private key blocks and `NAME_KEY=value` style lines with `[redacted]`. That is pattern matching, not a guarantee, so keep `sendHistory` off if your conversations carry things it would not catch. Your TypeSafe key is stored in the plugin store in plain text if you use `/jev key`, and the command line lands in the transcript; prefer the `TYPESAFE_API_KEY` environment variable. + diff --git a/README.md b/README.md index a593459..d683f66 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@

jev-router: automatic Claude model routing for Claude Code

- Switch between Opus, Sonnet and Haiku automatically. He pulls one lever; your task rides the right track. + Stop paying Opus prices for rename-a-variable work. Right model, right effort, every prompt, step and subagent.

@@ -22,328 +22,87 @@

Website · Install · - Benchmarks · + Proof · FAQ · + Reference · Changelog

---- +**jev-router is a free Claude Code plugin that sends each prompt, step and subagent to the cheapest Claude model that can do the job** (Haiku, Sonnet, Opus, Fable), caps runaway reasoning effort, and shows you every pick in a live band above your prompt. No config needed. -**jev-router is a free, open-source Claude Code plugin that routes every prompt, step and subagent to the right Claude model (Haiku, Sonnet, Opus, Fable) and the right reasoning effort.** It asks [TypeSafe Jev](#how-it-works) which model a task needs, caps the effort so hard tasks do not burn tokens, protects your prompt cache from needless model switches, and shows the choice in a live status band above the prompt. Use it to cut Claude Code cost on easy work without giving up Opus when a task needs it. - -## Where jev-router shines - -If your default is Opus, most of your prompts pay for more model than they need. A rename does not need the model that designs your queue. Measured with real Claude Code runs, against **always using Opus**: - -| Your work | What jev-router does | Result | -| --- | --- | --- | -| Small tasks: renames, typos, lookups, version bumps | Haiku at low effort | **−97% cost** | -| Everyday features, tests and refactors | Sonnet | **−65% cost** | -| Multi-step agent runs with tools and subagents | Re-routes each step, picks a model per subagent | **18/18 done at 47% of the cost** | -| Hard tasks, graded by hidden tests (the default `lean` preset) | Sonnet, effort `medium`, Opus only when Jev is 90% sure | **39/40 done at $0.064 per run vs $0.163 (−61%)** | -| A mixed bag of 30 tasks (single prompts, `balanced` preset) | Effort capped at `high` | **−1% to −64%** by mode, never above Opus | - -**Out of the box (0.4.0)** jev-router runs the `lean` preset: a Sonnet ceiling and effort capped at `medium`. Want the old behaviour? `/jev preset balanced` or set the `preset` option. [Presets](#presets). - -**Best for:** Opus-by-default users, mixed workloads full of small prompts, agent runs with subagents, and anyone who wants to *see and steer* which model runs (the band, `!opus`, `!full`, `/jev ceiling`). - -**Not for:** long sessions on one big warm context. There, `lean` averaged the same cost as Opus (some sessions as cheap as Sonnet, some as dear as Opus), and plain Sonnet was 43% cheaper than both; switching models re-writes the cache and the guard only halves that. And if plain Sonnet already does all your work, it is cheaper than routing. Every figure above comes from the [benchmarks](#does-it-reduce-claude-code-costs), including the ones that went against it. - -### Install in 30 seconds +## Install the Claude Code plugin ``` /plugin marketplace add dominicrico/jev-router /plugin install jev-router@jev-router ``` -Then `/jev key ` and `/jev status`. Full setup in [Install](#install-the-claude-code-plugin). - -You know the problem: you rename one variable and your most expensive model thinks about it for a minute. jev-router puts a switchman in front of your session. Before every task he reads it, asks Jev, pulls the lever, and the turn runs on the model and effort that fit. - -**Contents:** [Where it shines](#where-jev-router-shines) · [Status band](#the-status-band) · [Features](#features-model-routing-effort-cap-cache-guard-subagents) · [Does it reduce costs?](#does-it-reduce-claude-code-costs) · [How it works](#how-it-works) · [Install](#install-the-claude-code-plugin) · [Commands](#commands) · [Options](#options) · [Privacy](#privacy) · [FAQ](#faq) - -## The status band - -A live Claude Code status band above your prompt shows which model and effort were picked. Colour coded, with a spinner while a task runs. - -

- The band in all its states, one slide at a time: idle, running, haiku, sonnet, capped opus, !full unlock, cache kept, pinned, escalated, low confidence, cheap mode, paused, off. -

+Then `/jev key ` (or set `TYPESAFE_API_KEY`) and run `/jev status`. That is the whole setup: the `lean` preset is on by default.

- The band above the prompt: JEV, balanced mode, sonnet, medium effort, 88% confidence, warm cache of 42k tokens. Four colour-coded tiers below: haiku trivial, sonnet standard, opus hard, fable hardest. + Measured against always using Opus: minus 97 percent on small tasks, minus 65 percent on everyday work, minus 61 percent on hard tasks with 39 of 40 done, minus 56 percent on agent runs with subagents with 18 of 18 done.

-| Part | Meaning | -| --- | --- | -| `balanced` | Routing mode: `efficient`, `balanced` or `cheap` | -| `sonnet ▂▄__` | The model the task runs on, with its tier. Green haiku, blue sonnet, purple opus, gold fable | -| `medium ▰▰▱▱▱` | Reasoning effort, `low` to `max` | -| `conf ▮▮▮▮▯ 88%` | How sure Jev is. Green from 70%, amber from 50%, red below | -| `cache 🔒 warm 42k` | Whether the prompt cache is still worth protecting, and how big it is | - -When the cache wins, the band says so: `opus (kept 🔒 cache warm; wanted haiku)`. - -## Features: model routing, effort cap, cache guard, subagents - -- **Picks the model and the effort**, before every task, from haiku, sonnet, opus and fable. You can narrow the pool. -- **Three routing modes.** `efficient` for the best result, `balanced` for quality and cost evenly, `cheap` for the cheapest model that can plausibly succeed. -- **Guards the prompt cache.** Switching models throws the cache away. While it is warm and big enough to matter, he stays put unless Jev confidently asks for something stronger. -- **Caps the effort.** Jev's effort pick is limited to `medium` by default (the `lean` preset), because effort drives token use far more than the model does. Start a prompt with `!full` (or run `/jev full`) to lift the cap for that one prompt. -- **Shows what is really running.** The band follows the model that ran the last step, not only the pick: if a `/model` switch or a fallback ran something else it says `sonnet (picked opus)`, and after a Jev failure it shows the session model instead of waiting. -- **Ceiling and floor.** `ceiling: sonnet` keeps tasks off opus unless Jev is at least `ceilingBreak` (0.9) sure; `floor` sets the cheapest allowed model. `/jev ceiling opus`, `/jev floor haiku`. On at `sonnet` by default (the `lean` preset). -- **Escalates when a task struggles.** After `escalateAfter` (default 3) failed tool calls in a row, the rest of the task moves up one model. The band shows `↑`. -- **Pins a model for one prompt.** Start a prompt with `!opus`, `!sonnet`, `!haiku` or `!fable` to skip Jev and use that model (at the effort cap). `!cheap` and `!efficient` set the routing mode for that prompt. Markers combine, like `!opus !full`. -- **Keeps a savings tally.** `/jev status` shows an estimated saving against the model you would otherwise use (`baselineModel`, default opus), kept across sessions. It is a model-price estimate in relative units, blind to effort. -- **Never blocks you.** After 3 failed Jev calls in a row it pauses for a minute instead of paying a timeout on every step. Optional `fallback: heuristic` guesses locally meanwhile. Jev slow or down? The session model keeps working. -- **Routes every step.** Before each step after the first, Jev is asked again, so a task that turned out easier or harder moves to a fitting model (the cache guard still applies). Turn it off with `routeSteps`. -- **Routes subagents.** Each subagent is routed when it is spawned, from its full task prompt, and the pick shows in the subagent list as a tag on its description (`find importers · haiku/medium`). Its steps then run on that model and effort. An explicit `model` on the Agent call wins. Turn it off with `routeSubagents`. -- **Keeps score.** `/jev status` shows how often each model was used this session. - -## Does it reduce Claude Code costs? - -Three benchmarks. First, 30 single-prompt tasks: 270 calls to Jev for the picks, then 147 real runs for token usage and cost, with the effort cap at its default (`high`) and uncapped. Second, 6 multi-step tasks with tools on a fixture repo, 72 real runs, checking per prompt against every-step-and-subagent routing. Third, 4 harder tasks graded by hidden tests, 100 real runs, where quality is measured. +## See every pick, live

- Routing fit: efficient 100%, balanced 87%, cheap 66% of picks fit the task. 253 ms added per task, 97 to 100% same pick on repeat. + The band above the prompt stepping through its states: idle, running, haiku, sonnet, capped opus, ceiling, pinned, escalated, paused, off.

-

- Cost as a percentage of always opus with the effort capped at high. Trivial tasks 3%, standard 32 to 35%, hard 143%, 124% and 44% for efficient, balanced and cheap. Overall 99%, 87% and 36%. Uncapped hard tasks cost 342%, 307% and 61%. -

- -| Cost vs always opus (no plugin) | efficient | balanced | cheap | -| --- | --- | --- | --- | -| Trivial tasks | -97% | -97% | -97% | -| Standard tasks | -65% | -65% | -68% | -| Hard tasks, cap high (the `balanced` preset) | +43% | +24% | -56% | -| Hard tasks, uncapped | +242% | +207% | -39% | -| **All 30, cap high (default)** | **-1%** | **-13%** | **-64%** | -| All 30, uncapped | +123% | +101% | -53% | - -The honest read: the model switch saves a lot on easy and mid tasks, and the default cap of `high` brings every mode to or below always-opus overall. Hard tasks still cost 24% to 43% more in `efficient` and `balanced`, because Jev picks opus there and asks for deeper reasoning. `!full` lifts the cap for one prompt when you want that. The cap changes tokens spent, not answer quality, and that is not measured. +## What you get

- Measured long sessions: always opus costs 51 cents, always sonnet 27, jev-router with the cache guard 63 with 1 model switch, without the guard 70 with 6 switches and 194k cache tokens re-written. + Nine features: right model per prompt, re-routes every step, routes subagents, effort cap, Sonnet ceiling, cache guard, live band, per-prompt control, privacy controls.

-### Multi-step tasks, with tools -

- Multi-step tasks with tools, cost as a percentage of always opus: always sonnet 52%, jev-router per prompt 52%, jev-router every step and subagents 47%. All strategies finished 18 of 18 tasks. + How it works in six steps: a task arrives, ask Jev, check the cache, cap the effort, re-ask each step and route subagents, show it above the prompt.

-| 6 tasks × 3 runs, real runs with tools | tasks done | cost | vs always opus | vs always sonnet | -| --- | --- | --- | --- | --- | -| no plugin: always opus | 18/18 | $2.46 | | | -| no plugin: always sonnet | 18/18 | $1.29 | -48% | | -| jev-router, per prompt | 18/18 | $1.27 | -48% | -2% | -| jev-router, every step + subagents | 18/18 | $1.15 | **-53%** | **-11%** | - -Routing every step and subagent saved 11% on top of per-prompt routing. Most of that comes from one task: asked to use a subagent, it paired a sonnet main thread with a haiku subagent where per-prompt routing sometimes left the subagent on opus. On the other five tasks it matched per-prompt routing. Honest limits: these are easy tasks on a small repo, three runs each, and every strategy finished all of them, so this shows nothing was lost here, not that nothing is lost on hard work. Jev picked sonnet for nearly every one of these tasks, so against always-sonnet the gain is small. +## Does it reduce Claude Code costs? -### Harder tasks, graded by hidden tests +Yes, on most work, and the benchmarks say where it does not. Every number is a real Claude Code run (hard tasks are graded by hidden tests the agent never sees), and the raw runs are in the repo.

- Harder tasks graded by hidden tests, cost as a percentage of always opus: always sonnet 39%, jev-router per prompt 86%, every step and subagents 67%, with !full 60%. Tasks done out of 20: opus 19, sonnet 19, per prompt 19, every step 18, !full 20. + Harder tasks graded by hidden tests, mean cost per run as a percentage of always opus: always sonnet 38 percent, jev-router lean 39 percent with 39 of 40 done.

-| 4 hard tasks, hidden tests | runs | tasks done | mean cost per run | vs always opus | -| --- | --- | --- | --- | --- | -| no plugin: always opus | 40 | 38/40 | $0.163 | | -| no plugin: always sonnet | 40 | 38/40 | $0.062 | -62% | -| jev-router, per prompt | 20 | 19/20 | $0.133 | -18% | -| jev-router, every step + subagents | 20 | 18/20 | $0.104 | -36% | -| jev-router, `!full` | 20 | 20/20 | $0.093 | -43% | -| jev-router, `ceiling: sonnet` (cap high) | 40 | 39/40 | $0.075 | -54% | -| **jev-router, `lean` (the default)** | 40 | **39/40** | **$0.064** | **-61%** | - -The honest read: **on these tasks plain sonnet did as well as opus (38 of 40 each), and `lean` did the same (39 of 40) at 3% above Sonnet's cost.** `lean` spent everything on Sonnet here, so on this kind of work it is Sonnet with a safety valve; its edge over plain Sonnet shows on small tasks (Haiku) and with subagents. On the multi-step set `lean` finished 18/18 at $0.061 per run, the cheapest of the routed strategies. - -The tasks (a concurrency bug, an interval merger with open and closed bounds, a refactor that must keep an invariant, a flaky test with three causes) are harder than the first set, but not hard enough to need opus. jev-router lands between the two on cost because Jev sent about half of the spend to opus. Pass counts differ by at most one run, so none of the quality differences are significant at 5 runs. If your work looks like this, routing buys you less than just using sonnet; it earns its keep when tasks vary, or when your default is opus. - -### Long sessions: the cache guard, measured - -Switching models throws away the warm prompt cache, so the mod holds the current model while the cache is warm and large. I measured this with real long sessions: one Claude Code process per session, about 20k tokens of source pasted first, then 8 mixed tasks, 5 sessions per strategy (re-measured in 0.4.1 with more sessions and the new `lean` default). +## Is it for you?

- Measured long sessions, 5 per strategy: always opus 48 cents, always sonnet 27, jev-router balanced with the cache guard 67 and without 68, lean 48 with a range from 26 to 63 cents. + Great for Opus-by-default users, many small tasks, agent runs with subagents and anyone who wants to see which model runs. Skip it if plain Sonnet already does all your work or you live in one huge warm session.

-| Mean per session (5 sessions each) | cost | range | vs always opus | switches | cache tokens re-written | -| --- | --- | --- | --- | --- | --- | -| always opus | $0.48 | $0.29-$0.57 | | 0 | 36k | -| always sonnet | $0.27 | $0.26-$0.28 | -43% | 0 | 42k | -| jev-router `balanced`, cache guard on (`auto`) | $0.67 | $0.60-$0.77 | +42% | 1.0 | 81k | -| jev-router `balanced`, cache guard off | $0.68 | $0.58-$0.79 | +42% | 6.4 | 168k | -| **jev-router `lean` (the default)** | **$0.48** | $0.26-$0.63 | **0%** | 0.8 | 53k | - -What the 5-session numbers say, including what changed from my first 3-session run: - -- **The guard does what it says on switches and cache writes**: 6 to 7 model switches become 1, and cache re-writes halve (168k to 81k tokens). **It did not lower the bill**: guard on and off both cost +42% against Opus. My first run (3 sessions) showed +24% against +38% and I read that as a guard benefit; with more sessions it was noise. Corrected. -- **`lean` averaged the same as Opus (+0%), but the average hides two kinds of session.** In 3 of 5 it stayed on Sonnet and cost $0.26 to $0.45. In 2 of 5 Jev was at least 90% sure about a hard task, the ceiling let Opus through, the guard then held Opus for the rest of the session, and the session cost about $0.60. Always-Sonnet cost $0.27 every time. -- **So in a long warm session `lean` does not beat staying on one model.** If you work like this, plain Sonnet is the cheapest safe choice, or raise `ceilingBreak` to 1.0 so the ceiling is never broken (not measured). -- Short sessions with small contexts and the cold-cache case look different (see the table above and the benchmarks folder); routing's wins there come from easy tasks. - -### Presets - -How I picked the default. After the first hard-task run showed Opus taking half the spend for no gain, I tested a Sonnet ceiling (`ceiling`, off in 0.3.0) and then added the effort cap at `medium`. The rule, set before running: adopt it as the default only if its pass rate is within one run in 20 of always-Opus's and at least always-Sonnet's, and it costs less than the ceiling alone. It did (39/40 against 38/40 and 38/40; $0.064 against $0.075 per run), so it is the default. - -| Preset | ceiling | effort cap | mode | for | -| --- | --- | --- | --- | --- | -| `lean` (default) | sonnet (Opus only at 90% confidence) | medium | balanced | most work: small tasks on Haiku, the rest on Sonnet | -| `balanced` | none | high | balanced | the 0.2 and 0.3 behaviour | -| `max` | none | none | efficient | when quality matters more than cost | - -`/jev preset ` switches for the session; the `preset` option sets it permanently. `/jev ceiling`, `/jev cap` and the other commands still override one setting. - -Method, per-tier tables and all caveats: [benchmarks/](benchmarks/README.md). - -## How it works +Long warm sessions are the honest weak spot: the cache guard cuts model switches from 6 to 1 and halves cache re-writes, but the bill stayed the same, and `lean` averaged Opus cost there while plain Sonnet was 43% cheaper. See the chart:

- How it works in five steps: a task arrives, ask Jev which model and effort, check whether the prompt cache is warm, run the turn on the pick, show it above the prompt. + Measured long sessions, 5 per strategy: always opus 48 cents, always sonnet 27, jev-router balanced with and without the cache guard 67 and 68, lean 48 with a range of 26 to 63 cents.

-Per prompt, the mod sends Jev the routing mode, the current model, the cache state, the last few messages and the task text, and asks two choice questions: which model, and how much effort. The answer is applied to every step of that task. After each step the mod notes which model the API cached and how many tokens, so the next task knows whether a switch is worth losing the cache. - -Sent to `api.typesafe.ai`: the task text, recent conversation, and subagent descriptions. Each step and subagent adds one call (about 250 ms). Your API key travels in the request header and nowhere else. - -## Install the Claude Code plugin - -Needs a Claude Code version with mods (hooks modules) and a TypeSafe API key. - -``` -/plugin marketplace add dominicrico/jev-router -``` -``` -/plugin install jev-router@jev-router -``` - -Update: - -``` -claude plugin marketplace update jev-router -claude plugin update jev-router@jev-router -``` - -### API key - -Any one of these, checked in this order: - -1. The plugin option `apiKey` (`/config`) -2. `/jev key `, stored by the mod across sessions (`/jev key clear` removes it) -3. The `TYPESAFE_API_KEY` environment variable - -That was it. He'd be proud. He won't say it. - ## Commands -| Command | What it does | -| --- | --- | -| `/jev` or `/jev status` | Settings, cache state, last decision and per-model usage | -| `/jev efficient \| balanced \| cheap` | Set the routing mode, from the next task | -| `/jev sticky off \| auto \| strict` | Set cache stickiness | -| `/jev cap low \| medium \| high \| xhigh \| max \| none` | Set the effort cap | -| `/jev preset lean\|balanced\|max` | Switch the ceiling, effort cap and mode together | -| `/jev ceiling ` | Never go above this model unless Jev is very sure | -| `/jev floor ` | Never go below this model | -| `/jev full` | Lift the effort cap for the next prompt only | -| `/jev stats clear` | Reset the savings tally | -| `/jev on` / `/jev off` | Enable routing / use the session model | -| `/jev key ` | Store the TypeSafe API key | - -``` -┌ Jev router ────────────────────────────┐ -│ state ● on mode balanced │ -│ pool haiku · sonnet · opus · fable │ -│ sticky auto cache ● warm │ -│ last sonnet / medium ▮▮▮▮▯ 0.88 │ -│ usage haiku ░░░░░░░░░░ 0 0% │ -│ sonnet ██████████ 1 100% │ -│ opus ░░░░░░░░░░ 0 0% │ -│ fable ░░░░░░░░░░ 0 0% │ -│ key …fd1c (TYPESAFE_API_KEY) │ -└────────────────────────────────────────┘ -``` +

+ Cheat sheet: /jev status, /jev preset, /jev ceiling, /jev cap, /jev full, /jev key, and the prompt markers !opus, !sonnet, !haiku, !full, !cheap. +

-## Options - -| Option | Default | Meaning | -| --- | --- | --- | -| `mode` | `balanced` | Routing mode | -| `models` | all four | Aliases Jev may choose from | -| `preset` | `none` | `lean`, `balanced` or `max` overrides the three options below. `none`: the options apply as set (the shipped defaults are the `lean` values) | -| `effortCap` | `medium` | Highest effort the router applies. `none` follows Jev. Per prompt: start with `!full` or run `/jev full` | -| `ceiling` | `sonnet` | Highest model Jev may pick unless it is `ceilingBreak` sure (`haiku`, `sonnet`, `opus`, `fable`) | -| `floor` | `none` | Lowest model Jev may pick | -| `ceilingBreak` | `0.9` | Confidence needed to exceed the ceiling | -| `pauseMs` | `60000` | Stop asking Jev for this long after 3 failed calls in a row | -| `escalateAfter` | `3` | Failed tool calls in a row before the task moves up one model. `0` is off | -| `baselineModel` | `opus` | The model the savings estimate compares against | -| `sendHistory` | `true` | Send recent messages to Jev. Off sends only the task text | -| `redact` | `true` | Replace API keys, tokens, private keys and `KEY=value` lines with `[redacted]` before sending | -| `maxTaskChars` | `8000` | How much of the prompt is sent | -| `fallback` | `session` | When Jev is down: `session` keeps the session model, `heuristic` guesses locally | -| `routeSteps` | `true` | Ask Jev again before every step after the first. Off: one call per prompt | -| `routeSubagents` | `true` | Pick a model and effort for each subagent from its description | -| `stickiness` | `auto` | `off`: always follow Jev. `auto`: while the cache is warm, only upgrade on high confidence. `strict`: never switch while warm | -| `minConfidence` | `0.7` | Confidence needed for a warm-cache upgrade | -| `minContextTokens` | `8000` | Below this a switch is free and stickiness is skipped | -| `cacheTtlMs` | `300000` | Time after the last request when the cache counts as cold | -| `timeoutMs` | `4000` | Keep the current model when Jev takes longer | - -## Privacy - -Sent to `api.typesafe.ai`: the prompt text (up to `maxTaskChars`), recent messages if `sendHistory` is on, and subagent descriptions. Before sending, `redact` replaces API keys, GitHub tokens, AWS keys, bearer tokens, private key blocks and `NAME_KEY=value` style lines with `[redacted]`. That is pattern matching, not a guarantee, so keep `sendHistory` off if your conversations carry things it would not catch. Your TypeSafe key is stored in the plugin store in plain text if you use `/jev key`, and the command line lands in the transcript; prefer the `TYPESAFE_API_KEY` environment variable. +Every option, preset, command and the privacy details are in [OPTIONS.md](OPTIONS.md). Method, per-benchmark results and caveats are in [benchmarks/](benchmarks/README.md). ## FAQ -**How do I switch between Opus, Sonnet and Haiku automatically in Claude Code?** -Install jev-router. It picks the model per prompt, again before each step, and once per subagent. Pin a model for one prompt with `!opus`, `!sonnet` or `!haiku`. - -**How do I reduce Claude Code token cost?** -Send easy tasks to cheaper models and cap the reasoning effort. jev-router does both; the [benchmarks](#does-it-reduce-claude-code-costs) show where it saves (trivial tasks -97%, standard -65% against always Opus) and where it does not (on harder tasks plain Sonnet was as good and cheaper). - -**What is the Claude Code effort cap?** -Jev often asks for `xhigh` reasoning on hard tasks, which can use several times the tokens. jev-router limits it to `medium` by default (`high` in the `balanced` preset); start a prompt with `!full` to lift it once. +**How do I switch between Opus, Sonnet and Haiku automatically in Claude Code?** Install jev-router; it picks per prompt, again before each step, and once per subagent. Pin one prompt with `!opus`, `!sonnet` or `!haiku`. -**Does jev-router send my code to a third party?** -It sends the prompt, optionally the last messages, and subagent task text to `api.typesafe.ai`. Keys, tokens, private keys and `password=` style values are redacted first, and `sendHistory` can turn the history off. See [Privacy](#privacy). +**How do I reduce Claude Code token cost?** Send easy tasks to cheaper models and cap the reasoning effort. jev-router does both (small tasks −97%, everyday −65% against always Opus). If plain Sonnet already does your work, use that: it was cheapest on the hard set. -**Will it slow Claude Code down?** -Each Jev call adds about 250 ms. After three failed calls in a row it pauses for a minute and keeps your current model. +**What is the effort cap?** Jev often asks for `xhigh` reasoning on hard tasks, which can triple the tokens. The default `lean` preset limits effort to `medium`; `!full` lifts it for one prompt. +**Does it send my code to a third party?** The prompt, optionally recent messages, and subagent task text go to `api.typesafe.ai`. Keys, tokens, private keys and `password=` style values are redacted first, and `sendHistory` can turn the history off. Details in [OPTIONS.md](OPTIONS.md#privacy). -**Does it need a config file?** -No. Set a key, pick a mode, done. Everything else has a default. - -**Why not just always use the best model?** -You can. `/jev efficient` and a pool of `opus` and `fable`. Your invoice will have opinions. - -**Why not just always use the cheap one?** -`/jev cheap`. The hard task will have opinions. - -**He kept my old model even though Jev wanted a cheaper one. Is it broken?** -No. Your cache was warm and a switch would have rewritten it. He did the maths. He was right. - -**What if Jev is down?** -Then you get your session model and one polite toast. He never blocks a turn. - -**Why a raccoon?** -Because he is a small creature that gets very serious about picking the right track. - -## Tests - -``` -claude plugin validate . -claude plugin test . -``` +**Will it slow Claude Code down?** Each Jev call adds about 250 ms; after three failures in a row it pauses for a minute and keeps your current model. -Benchmarks: see [benchmarks/](benchmarks/README.md). +**Why a raccoon?** He is a small creature that gets very serious about picking the right track. ## License -[MIT](LICENSE). +[MIT](LICENSE). Not affiliated with Anthropic or TypeSafe. Security issues: see [SECURITY.md](SECURITY.md). Keywords: Claude Code plugin, Claude model router, Claude Code cost optimization, Opus Sonnet Haiku switching, Claude Code subagents, prompt cache, reasoning effort, LLM routing. diff --git a/assets/benefits.png b/assets/benefits.png new file mode 100644 index 0000000..fd27b1e Binary files /dev/null and b/assets/benefits.png differ diff --git a/assets/commands.png b/assets/commands.png new file mode 100644 index 0000000..127b49b Binary files /dev/null and b/assets/commands.png differ diff --git a/assets/features.png b/assets/features.png new file mode 100644 index 0000000..b553ed1 Binary files /dev/null and b/assets/features.png differ diff --git a/assets/fitfor.png b/assets/fitfor.png new file mode 100644 index 0000000..64b3b50 Binary files /dev/null and b/assets/fitfor.png differ diff --git a/assets/src/make.mjs b/assets/src/make.mjs index 2b9449d..0984748 100644 --- a/assets/src/make.mjs +++ b/assets/src/make.mjs @@ -144,6 +144,56 @@ ${[[`-${100 - pctOpus('trivial', 'cheap', true)}%`, 'cost, trivial'], [`-${100 -
Jev buys deeper reasoning there: +${pctOpus('hard', 'balanced', true) - 100}% to +${pctOpus('hard', 'efficient', true) - 100}% at the default cap of high.
Lift it for one prompt with !full.
30 tasks, 270 Jev calls, ${147 + ag.length + ag2.length} real runs. Opus 5.5, Sonnet 5.5, Haiku 5.5. Multi-step runs check the task got done. Quality is measured on the tool-using tasks only.
` + +// ---- compact README cards ------------------------------------------------- +const sav = (a, b) => Math.round(100 - (100 * a) / b) +const leanHard = sav(aCost(LEAN[0], undefined, ag2), aCost(AS2[0][0], undefined, ag2)) +const leanMulti = sav(aCost(LEAN[0]), aCost(AS[0][0])) +const tile = (big, label, sub, color) => `
${big}
${label}
${sub}
` +const benefitsCard = `
Measured with real Claude Code runs · against always using Opus
+

Pay for the model each task actually needs.

+
+${tile(`-${pctOpus('trivial', 'cheap', true) < 50 ? 100 - pctOpus('trivial', 'cheap', true) : 0}%`, 'Small tasks', 'Renames, typos, lookups, version bumps. Haiku at low effort does them.')} +${tile(`-${100 - pctOpus('standard', 'balanced', true)}%`, 'Everyday work', 'Features, tests, refactors. Sonnet does them.')} +${tile(`-${leanHard}%`, 'Hard tasks, still 39/40 done', 'Graded by hidden tests. Sonnet with a safety valve to Opus.')} +${tile(`-${leanMulti}%`, 'Agent runs with subagents, 18/18 done', 'Every step and every subagent gets its own model.')} +
+
${['Zero config: the lean preset is the default', 'Live band shows every pick', 'Override any prompt with !opus or !full', 'Free, MIT'].map(t => `${t}`).join('')}
+
` + +const feat = (icon, t, d) => `
${icon}
${t}
${d}
` +const featuresCard = `
What it does
+

One lever per task. Nine things you get.

+
+${feat('🎯', 'Right model per prompt', 'Haiku, Sonnet, Opus or Fable, picked by TypeSafe Jev before the task runs.')} +${feat('🔁', 'Re-routes every step', 'A task that turns out easier or harder moves to a fitting model.')} +${feat('🤖', 'Routes subagents', 'Each subagent gets its own model, tagged right in the subagent list.')} +${feat('🧠', 'Effort cap', 'Stops xhigh reasoning from quietly tripling a bill. !full lifts it once.')} +${feat('🛡️', 'Sonnet ceiling', 'Opus only when Jev is 90% sure. Floor and presets included.')} +${feat('🔒', 'Cache guard', 'Holds the model while your prompt cache is warm: switches 6 to 1.')} +${feat('📟', 'Live band', 'Model, effort, confidence and cache above the prompt. Shows what really runs.')} +${feat('⌨️', 'Per-prompt control', '!opus, !sonnet, !haiku, !cheap, !full. One prompt, your call.')} +${feat('🔐', 'Privacy controls', 'Secrets redacted before anything leaves; history can be switched off.')} +
` + +const fitCol = (title, color, items) => `
${title}
${items.map(t => `
${title.startsWith('Great') ? '✓' : '✗'}${t}
`).join('')}
` +const fitForCard = `
Is it for you?
+

Built for people who default to Opus.

+
+${fitCol('Great for', '#4ade80', ['Opus by default, mixed prompts', 'Lots of small tasks and quick questions', 'Agent runs with subagents', 'Seeing and steering which model runs', 'A cheap safety net before you reach for Opus'])} +${fitCol('Skip it if', '#fbbf24', ['Plain Sonnet already does all your work: it is cheaper still', 'You live in one huge warm session: routing averaged Opus cost there', 'You need every answer on the strongest model'])} +
+
Every claim on this page is a measured result, including the two on the right. Details and raw runs are in the benchmarks folder.
` + +const cmd = (c, d) => `
${c}${d}
` +const commandsCard = `
Cheat sheet
+

Everything is one command away.

+
+
${[['/jev status', 'Settings, last pick, savings'], ['/jev preset lean|balanced|max', 'Switch the whole setup'], ['/jev ceiling sonnet', 'Never above this model'], ['/jev cap medium', 'Highest reasoning effort'], ['/jev full', 'Lift the cap for one prompt'], ['/jev key ', 'Store your TypeSafe key']].map(([c, d]) => cmd(c, d)).join('')}
+
${[['!opus !sonnet !haiku', 'Pin a model for one prompt'], ['!full', 'Lift the effort cap once'], ['!cheap !efficient', 'Set the mode for one prompt'], ['/jev off / /jev on', 'Use the session model'], ['/jev stats clear', 'Reset the savings tally'], ['/jev sticky auto|strict|off', 'Cache guard strength']].map(([c, d]) => cmd(c, d)).join('')}
+
+
Full option and command reference: OPTIONS.md in the repository.
` + mkdirSync(join(here, '..'), { recursive: true }) const browser = await chromium.launch({ executablePath: process.env.CHROME }) const shoot = async (html, out, w, type = 'png') => { @@ -154,7 +204,7 @@ const shoot = async (html, out, w, type = 'png') => { } await shoot(hero, 'hero.jpg', 880, 'jpeg') await shoot(costCard, 'cost.png', 880); await shoot(fitCard, 'fit.png', 880); await shoot(cacheCard, 'cache.png', 880) -await shoot(howCard, 'how.png', 880); await shoot(agenticCard, 'agentic.png', 880); await shoot(hardCard, 'hard.png', 880); await shoot(bandCard, 'band.png', 880) +await shoot(howCard, 'how.png', 880); await shoot(benefitsCard, 'benefits.png', 880); await shoot(featuresCard, 'features.png', 880); await shoot(fitForCard, 'fitfor.png', 880); await shoot(commandsCard, 'commands.png', 880); await shoot(agenticCard, 'agentic.png', 880); await shoot(hardCard, 'hard.png', 880); await shoot(bandCard, 'band.png', 880) await browser.close() // the GitHub Pages site serves its own copies -for (const f of ['hero.jpg', 'band.gif', 'cost.png', 'fit.png', 'hard.png', 'agentic.png', 'cache.png', 'how.png', 'logo.jpg']) cpSync(join(here, '..', f), join(here, '../../docs/assets', f)) +for (const f of ['hero.jpg', 'band.gif', 'cost.png', 'fit.png', 'hard.png', 'agentic.png', 'cache.png', 'how.png', 'benefits.png', 'features.png', 'fitfor.png', 'commands.png', 'logo.jpg']) cpSync(join(here, '..', f), join(here, '../../docs/assets', f)) diff --git a/docs/assets/benefits.png b/docs/assets/benefits.png new file mode 100644 index 0000000..fd27b1e Binary files /dev/null and b/docs/assets/benefits.png differ diff --git a/docs/assets/commands.png b/docs/assets/commands.png new file mode 100644 index 0000000..127b49b Binary files /dev/null and b/docs/assets/commands.png differ diff --git a/docs/assets/features.png b/docs/assets/features.png new file mode 100644 index 0000000..b553ed1 Binary files /dev/null and b/docs/assets/features.png differ diff --git a/docs/assets/fitfor.png b/docs/assets/fitfor.png new file mode 100644 index 0000000..64b3b50 Binary files /dev/null and b/docs/assets/fitfor.png differ