Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@ jobs:
- name: Answer it and install
run: |
jq -n --arg dir "$LENS_DIR" --arg ref "$LENS_REF" '{backend: "docker", target: "lean", dir: $dir, ref: $ref,
settings: {llm: {base_url: "http://localhost:11434/v1", model: "llama3", api_key: "sk-ci"}}}' >answers.json
settings: {llm: {base_url: "http://localhost:11434/v1", model: "llama3", api_key: "sk-ci"},
budget: {hard_usd: 25, period: "month", price_in: 3, price_out: 15}}}' >answers.json
curl -fsS -b jar -H 'X-Lens-Installer: 1' -H 'Content-Type: application/json' -d @answers.json \
http://127.0.0.1:3990/api/install
since=0
Expand All @@ -120,7 +121,7 @@ jobs:
curl -fsS -o /dev/null "$url"
curl -fsS "$base/api/v1/auth/status" | grep -q '"setup_required":true'
cd "$LENS_DIR"
docker compose -f docker-compose.prod.yml logs backend | grep "Applied the installer's settings: llm"
docker compose -f docker-compose.prod.yml logs backend | grep "Applied the installer's settings: llm, budget"
# saved in Lens: the key doesn't stay in .env
! grep -q '^LENS_SETUP_SETTINGS=' .env

Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **A spending limit from the start.** The setup wizard has a Budget step, and the web installer asks for one with a
cloud provider: a scale for a hard limit on everything Lens spends on AI, with a soft limit at 80% that warns. At
the hard limit runs wait (or are skipped) and paid model calls stop until it goes up; models on your own computer
keep working. The model's price comes with it, so the spending is counted. It's the new `all` budget, changeable in
Settings › Budgets ([Budgets](docs/budgets.md#one-budget-for-everything)). The web installer now asks only the
essentials: the folder, port and network under **Advanced**, the optional extras under **More options** on the
review.

- **The graph as files.** Graph nodes are folders, nested along its links: collections inside collections, topics
inside their broader topics, entities by type. Each folder holds its assets' files, with a `.json` sidecar of
metadata and a `.txt` of the text; everyone sees what the web app shows them. Browse and read it through the API
Expand Down
2 changes: 0 additions & 2 deletions ci/clicks/journeys.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,6 @@ export const JOURNEYS = [
if (!(await found.evaluate((el) => el.classList.contains("on"))))
await meter.click(found, "pick the model server found on this computer");
await meter.click(page.locator("#next"), "Continue (AI model)");
await page.getByRole("heading", { name: "Extras" }).waitFor();
await meter.click(page.locator("#next"), "Continue (extras)");
await page.getByRole("heading", { name: "Ready to install" }).waitFor();
await meter.click(page.locator("#next"), "Install Lens");
}
Expand Down
20 changes: 20 additions & 0 deletions docs/budgets.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,26 @@ PUT /api/v1/budgets?resource=routine:3
(also for a resource without a budget, with its estimated next run), and `DELETE /api/v1/budgets?resource=…`
removes one.

## One budget for everything

Besides budgets for single resources, `all` caps everything Lens spends: every model call, run and job, whatever it is
for. The setup wizard and the web installer ask for it after a cloud model (one on your own machine costs nothing), as a
scale for the hard limit with the soft limit at 80% of it; Settings › Budgets changes it.

```http
PUT /api/v1/setup/budget
{"hard_usd": 25, "soft_usd": 20, "period": "month", "on_over": "ask", "price_in": 3, "price_out": 15}
```

- **Soft limit**: past it, the budget is `near` and warns, as `warn_at` does (it is saved as `warn_at` = soft ÷ hard).
- **Hard limit**: at it, routines and jobs are held (or skipped, with `on_over: "skip"`) as any budget's are, and model
calls that cost money stop with the reason until the limit goes up or the period turns. Models without a price
(one on your own computer) keep working, unless the budget also caps `tokens`.
- `price_in`, `price_out`: the chat model's price in USD per million tokens (`ai.price_in`, `ai.price_out`), so
what it spends is counted. Without a price a budget in USD counts nothing for that model.
- `hard_usd: null` removes the budget. `GET /api/v1/setup` shows it under `budget`. `PUT /api/v1/budgets?resource=all`
sets it like any other budget, with `tokens` too.

## Before a run

Before a routine runs and before a job's first step, Lens looks at the budgets of everything the run counts for: the
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ with AES-GCM and are write-only: the API reports whether one is set, never its v
| `LENS_ADMIN_PASSWORD` | | give that admin a password instead (passwords stay on) |
| `LENS_NAMESPACE` | | create the first namespace at startup, while there is none |
| `LENS_LLM_BASE_URL` / `LENS_LLM_MODEL` / `LENS_LLM_API_KEY` / `LENS_LLM_VISION_MODEL` | | the model provider; wins over Settings, which show these locked |
| `LENS_SETUP_SETTINGS` | | settings to start with, as JSON `{"section": {"key": value}}` (or that JSON base64url-encoded): saved once at startup as if made in the app, so unlike the variables above they stay editable there. The [web installer](get-started.md#the-web-installer) writes its answers here |
| `LENS_SETUP_SETTINGS` | | settings to start with, as JSON `{"section": {"key": value}}` (or that JSON base64url-encoded): saved once at startup as if made in the app, so unlike the variables above they stay editable there. The [web installer](get-started.md#the-web-installer) writes its answers here; a `budget` section (`hard_usd`, `soft_usd`, `period`, `on_over`, `price_in`, `price_out`) sets [the overall budget](budgets.md#one-budget-for-everything) |
| `LENS_SETUP_WIZARD` | | `off`: never show the setup wizard |
| `LENS_TELEMETRY` / `LENS_TELEMETRY_ENDPOINT` / `LENS_TELEMETRY_HEADERS` | off | opt-in OpenTelemetry traces and metrics, sent only to this OTLP/HTTP endpoint ([Telemetry](telemetry.md)); `LENS_TELEMETRY=off` keeps it off whatever Settings say |
| `FRONTEND_URL` | `http://localhost:3000` | the web app's address as people use it: links in emails, and where apps send people to sign in ([OAuth](authentication.md#oauth)) |
Expand Down
39 changes: 22 additions & 17 deletions docs/get-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,26 @@ On a first install run at a terminal (on Linux or a Mac with Python 3.8 or later
above opens an installer in your browser instead of asking anything in the terminal:

1. **This computer**: the system, memory, free disk, whether Docker is there and running, and whether microsandbox can
run. On a first install, **Install with the defaults** skips the next steps: it picks a model server it finds on
the computer, if there is one, and installs straight away. **Choose options** goes through them.
2. **How it runs**: Docker or [microsandbox](microsandbox.md), the full or lean edition (lean is picked on a computer
with less than 4 GB of memory, such as a Raspberry Pi), the folder, the port, and whether other devices on the
network may reach it. It lists what it will install; when that needs administrator rights, it asks for your
password here, keeps it in memory for the install and hands it to `sudo` (it is never written down).
run. On a first install, **Install with the defaults** installs right away (Docker, the edition that suits the
computer, the model server found on it, if any) and shows its progress; **Choose options** goes through the steps
below, as it does when installing needs your password.
2. **How it runs**: Docker or [microsandbox](microsandbox.md) and the full or lean edition (lean is picked on a computer
with less than 4 GB of memory, such as a Raspberry Pi). It lists what it will install; when that needs
administrator rights, it asks for your password here, keeps it in memory for the install and hands it to `sudo` (it
is never written down). The folder, the port, whether other devices on the network may reach it and the version
wait under **Advanced**.
3. **AI model**: model servers already running on the computer (Ollama, LM Studio, llama.cpp, vLLM, LocalAI) with their
models, or a provider (OpenAI, Anthropic, Google Gemini, OpenRouter, Groq, Mistral or any OpenAI-compatible server)
with its API key; **Test** lists the models. It can be skipped and set in Lens later.
4. **Extras**, all optional: the first namespace, search by meaning (an embeddings server), the typesafe.ai decision
model's key, cloud transcription and its key, email (SMTP), remote access through a Cloudflare tunnel, anytopdf,
and telemetry.
5. **Review**, then **Install**: it runs `install.sh` with those answers and shows its progress (and the full output
under **Details**). If something fails, it says what, and **Try again** runs it again (what's done stays done).
6. **Open Lens** goes to the setup page with the setup code filled in, where you create the admin account with a
with its API key; **Test** lists the models. For a provider, a scale sets a monthly spending limit (none unless you
pick one) with the model's price, so Lens can count what it spends; **More** sets a soft limit (80% unless changed)
and a day or week instead ([One budget for everything](budgets.md#one-budget-for-everything)). It can all be
skipped and set in Lens later.
4. **Review**, then **Install**: **More options** under the review holds what's optional, each with its default: the
first namespace, search by meaning (an embeddings server), the typesafe.ai decision model's key, cloud
transcription and its key, email (SMTP), remote access through a Cloudflare tunnel, anytopdf, and telemetry.
**Install** runs `install.sh` with those answers and shows its progress (and the full output under **Details**). If
something fails, it says what, and **Try again** runs it again (what's done stays done).
5. **Open Lens** goes to the setup page with the setup code filled in, where you create the admin account with a
passkey.

The answers that are settings go to Lens as `LENS_SETUP_SETTINGS`, which it saves once at its first start as if they
Expand Down Expand Up @@ -120,10 +125,10 @@ On first start the API log prints a setup code, and a link that fills it in (`ma
```
No accounts yet. Create the first admin in the web app with setup code: … (or open http://localhost:3000/setup?code=…, which fills it in)
```
Open the link (or the web app, and enter the code) and create the admin account. A short wizard then asks for the first
namespace, the model provider, storage, apps and AI, and telemetry. Once a model is set, **Finish with defaults** skips
the rest; change any of it later in Settings ([Configuration](configuration.md#first-run-setup) lists the `.env` values
that answer it instead).
Open the link (or the web app, and enter the code) and create the admin account. A short wizard then asks for the first
namespace, the model provider, a [budget](budgets.md#one-budget-for-everything) (for a cloud model), storage, apps and
AI, and telemetry. Once a model is set, **Finish with defaults** skips the rest; change any of it later in Settings
([Configuration](configuration.md#first-run-setup) lists the `.env` values that answer it instead).

After the first sign-in, a short welcome card points to the two places most people come back for: the assistant, which
you can ask straight from the card, and **Reports**, with the charts of what is in the archive. Asking, opening
Expand Down
4 changes: 2 additions & 2 deletions fastapi_backend/app/api/v1/routes/budgets.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from app.schemas.common import Ok

router = APIRouter(tags=["budgets"])
RESOURCE = Query(description="routine:<id>, pipeline:<id>, workflow:<id> or space:<id>")
RESOURCE = Query(description="all (everything Lens spends), routine:<id>, pipeline:<id>, workflow:<id> or space:<id>")


@router.get("/budgets")
Expand Down Expand Up @@ -46,7 +46,7 @@ def budget_meter(
resource = f"space:{acl.namespace(ns)}"
ref = resource
elif user.admin:
resource, ref = None, budgets.GLOBAL
resource, ref = None, budgets.ALL
else:
raise HTTPException(403, "only admins see what all of Lens spent; pick a namespace")
out = budgets.meter(db, resource, ref, period, detail)
Expand Down
15 changes: 15 additions & 0 deletions fastapi_backend/app/api/v1/routes/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from app.domain import auth, setup
from app.schemas.setup import (
LocalModelServer,
SetupBudget,
SetupFinish,
SetupLlm,
SetupNamespace,
Expand Down Expand Up @@ -96,6 +97,20 @@ def save_telemetry(body: SetupTelemetry, user: AdminWriter, request: Request, db
return SetupSaved(saved=saved)


@router.put("/budget")
def save_budget(body: SetupBudget, user: AdminWriter, request: Request, db: Db) -> SetupSaved:
"""The overall AI budget: a soft limit that warns and a hard limit where runs wait and paid model calls stop
(docs/budgets.md#one-budget-for-everything), with the chat model's price so its spending is counted. No hard
limit: no budget."""
try:
saved = setup.save_budget(db, request.app.state.archive.base, user=user.email, **body.model_dump())
except ValueError as e:
raise HTTPException(400, str(e)) from None
if saved:
auth.audit(db, user.as_audit(), "budget.set", "all", body.model_dump(exclude_none=True))
return SetupSaved(saved=saved)


@router.put("/oauth")
def save_oauth(body: SetupOAuth, user: AdminWriter, request: Request, db: Db) -> SetupSaved:
"""Let apps and AI assistants (MCP clients such as Claude or Cursor) sign people in with their Lens account, or
Expand Down
14 changes: 14 additions & 0 deletions fastapi_backend/app/domain/activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,10 @@ def record(kind, action, refs=(), cfg=None, db=None, **fields):
return None


class OverBudget(RuntimeError):
"""A call refused because the overall budget's hard limit is reached (budgets.hard_stop)."""


class Call:
"""A call out, timed: `with activity.call("embeddings", model=m) as c: ...; c.usage(tin, tout)`. An exception that
leaves the block marks the row failed with its type."""
Expand All @@ -193,6 +197,16 @@ def __init__(self, action, cfg=None, model=None, refs=(), detail=None, price=Non
self.unpriced = False # it costs something, but the figure is missing (see model_cost)
self.t0 = time.monotonic()
self.done = False
if model: # a call to a model: the overall budget's hard limit (budgets.py) stops one that would cost money
sc = _scope.get() or {}
db = sc.get("db") or _default["db"]
if db is not None:
from . import budgets

why = budgets.hard_stop(db, cfg if cfg is not None else sc.get("cfg"), model)
if why:
self.done = True # nothing was called: no ledger row
raise OverBudget(why)

def usage(self, tokens_in=None, tokens_out=None, cost_usd=None):
if isinstance(tokens_in, int):
Expand Down
Loading
Loading