From 11180387a34208a9f625512c59e8ea8eecfc2982 Mon Sep 17 00:00:00 2001
From: Adeel Ahmad
Date: Sat, 10 Oct 2026 20:02:29 +0000
Subject: [PATCH 1/5] feat(analytics): usage metrics with churn and retention,
and opt-in Google Analytics events
Lens works out daily, weekly and monthly active people, stickiness, churn,
retention by sign-up week and feature use from the activity ledger and
sign-ins (GET /api/v1/admin/analytics). Google Analytics 4 is off by
default: with a measurement id and API secret (Settings, the setup
wizard's telemetry step, or .env) one process sends a documented set of
counted events once a day for the install, never personal content.
Co-Authored-By: Claude Opus 5.5 (1M context)
Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH
---
docker-compose.prod.yml | 4 +
docs/analytics.md | 90 +++
docs/configuration.md | 3 +-
fastapi_backend/.env.example | 5 +
fastapi_backend/app/api/v1/routes/admin.py | 24 +
fastapi_backend/app/api/v1/routes/setup.py | 15 +-
fastapi_backend/app/domain/analytics.py | 348 +++++++++++
fastapi_backend/app/domain/ops_tools.py | 5 +-
fastapi_backend/app/domain/routines.py | 6 +
fastapi_backend/app/domain/settings.py | 33 ++
fastapi_backend/app/domain/setup.py | 27 +
fastapi_backend/app/domain/store.py | 18 +
fastapi_backend/app/schemas/admin.py | 85 +++
fastapi_backend/app/schemas/setup.py | 16 +
fastapi_backend/tests/api/test_analytics.py | 204 +++++++
fastapi_backend/tests/api/test_telemetry.py | 21 +-
mkdocs.yml | 1 +
.../__tests__/setupWizard.test.tsx | 7 +-
nextjs-frontend/app/openapi-client/index.ts | 20 +
nextjs-frontend/app/openapi-client/sdk.gen.ts | 53 +-
.../app/openapi-client/types.gen.ts | 332 +++++++++++
nextjs-frontend/openapi.json | 546 +++++++++++++++++-
22 files changed, 1851 insertions(+), 12 deletions(-)
create mode 100644 docs/analytics.md
create mode 100644 fastapi_backend/app/domain/analytics.py
create mode 100644 fastapi_backend/tests/api/test_analytics.py
diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml
index c5b41f574..379b3422b 100644
--- a/docker-compose.prod.yml
+++ b/docker-compose.prod.yml
@@ -58,6 +58,10 @@ x-backend: &backend
LENS_TELEMETRY: ${LENS_TELEMETRY:-}
LENS_TELEMETRY_ENDPOINT: ${LENS_TELEMETRY_ENDPOINT:-}
LENS_TELEMETRY_HEADERS: ${LENS_TELEMETRY_HEADERS:-}
+ # opt-in usage events to your own Google Analytics 4 property (docs/analytics.md): unset, off unless turned on in Settings
+ LENS_ANALYTICS: ${LENS_ANALYTICS:-}
+ LENS_GA_MEASUREMENT_ID: ${LENS_GA_MEASUREMENT_ID:-}
+ LENS_GA_API_SECRET: ${LENS_GA_API_SECRET:-}
# the volume shared with the matterbridge service (its config, written from Settings → Chat rooms)
LENS_MATTERBRIDGE_DIR: /matterbridge
# upgrading from the web app (docs/updates.md): the folder install.sh's upgrade agent watches, and the install folder
diff --git a/docs/analytics.md b/docs/analytics.md
new file mode 100644
index 000000000..18b5714a1
--- /dev/null
+++ b/docs/analytics.md
@@ -0,0 +1,90 @@
+# Usage analytics
+
+Lens works out how it is being used (active people, churn, retention, which features get used) on your own machine,
+and can also send a few counted events to a **Google Analytics 4** property you own, to see the same across every
+install you run.
+
+Status: **built** (backend and API). The Settings page and the wizard toggle are **in progress**.
+
+## The numbers
+
+`GET /api/v1/admin/analytics?days=30` (admins) works these out from what Lens already keeps: the
+[activity ledger](activity.md)'s rows for API calls a person made, and sign-ins in the audit log. Nothing new is
+collected for them and nothing leaves the machine.
+
+| Number | What it is |
+| --- | --- |
+| `daily` | each day of the period: how many people were active, and how many actions they took |
+| `dau`, `wau`, `mau` | people active today, in the last 7 days, in the last 30 days |
+| `stickiness` | average daily active people ÷ monthly active people (how much of the month a regular uses Lens) |
+| `active_users`, `new_users`, `accounts` | people active in the period, accounts made in it, accounts not disabled |
+| `churn` | of the people active in the period before this one (`previous_active`), how many came back (`retained`) and how many didn't (`churned`); `rate` = churned ÷ previous_active; `returning` were active now but not before |
+| `retention` | for each of the last 8 weeks, the accounts made that week and the share of them active that week and each week after |
+| `features` | each feature (from the API routes: recordings, chats, graph, routines…) with its actions and people |
+| `last_active_day`, `days_since_active` | when anyone last used this install |
+
+A person is active on a day when they changed something through the API or signed in. With `activity.reads` on,
+reads count too. The ledger keeps `activity.keep_days` (365), which is as far back as the numbers can go.
+
+## Google Analytics
+
+**Off unless you turn it on.** It needs a GA4 property of your own: its **measurement id** (`G-…`) and a
+**Measurement Protocol API secret** (in GA: Admin → Data streams → your stream → Measurement Protocol API secrets).
+
+- **Settings → Usage analytics**: turn it on and set the id and secret. **Send a test event** sends one `lens_test`
+ event to GA's validation server, which checks it without recording it.
+- **The setup wizard**, on its telemetry step.
+- **.env**: `LENS_ANALYTICS=on`, `LENS_GA_MEASUREMENT_ID=G-…`, `LENS_GA_API_SECRET=…`. What .env sets wins over
+ Settings, which show those fields locked.
+
+Turned on, one process (a worker, or `lens watch`) sends once a day, for the install as a whole:
+
+| Event | When | Parameters |
+| --- | --- | --- |
+| `lens_install` | analytics first runs on this install | `version`, `platform` |
+| `lens_setup_done` | the setup wizard was finished or skipped | `skipped` (0 or 1) |
+| `lens_daily_active` | at least one person used Lens the day before | `active_users`, `actions` |
+| `lens_feature_used` | each feature used the day before (up to 20) | `feature`, `actions`, `users` |
+| `lens_upgrade` | Lens runs a new version | `from_version`, `to_version` |
+| `lens_inactive` | nobody has used Lens for `analytics.inactive_days` (30) | `days` |
+| `lens_return` | someone used Lens again after `lens_inactive` | `days_away` |
+
+Every event also carries `session_id` (the day) and `engagement_time_msec`, so GA counts the install as an active
+user that day, and the user properties `lens_version` and `lens_platform` (like `linux-x86_64`).
+
+**What is sent:** the install's id (a random UUID made on first use, not derived from the machine), the version, the
+platform and the counts above. **Never** a name, email, account id, title, file name, namespace, prompt or any text.
+Like any web request, it reaches Google from your server's address (GA4 doesn't keep IP addresses). `analytics.events` chooses which events are sent; leave one out and it never is.
+
+### Churn and retention in GA
+
+Each install is one GA user. In GA:
+
+- **Reports → Retention** shows installs coming back day after day and week after week, from `lens_daily_active`.
+- **Churn rate** for a period is installs active in the period before that sent no `lens_daily_active` in it. An
+ Exploration (Cohort exploration, with `lens_install` as the inclusion event and `lens_daily_active` as the return
+ event) gives it by install week. `lens_inactive` counts installs that went quiet: a self-hosted install that was
+ removed can't say so, so `inactive_days` without use is the uninstall-like signal.
+- **Feature use**: register `feature` as an event-scoped custom dimension (Admin → Custom definitions), and `actions`
+ and `active_users` as custom metrics, then break `lens_feature_used` down by `feature`.
+
+## Settings
+
+| Setting | Default | What it does |
+| --- | --- | --- |
+| `analytics.enabled` | `false` | send to Google Analytics |
+| `analytics.measurement_id` | none | the GA4 measurement id, like `G-ABC123XYZ` |
+| `analytics.api_secret` | none | the Measurement Protocol API secret (encrypted at rest, never shown again) |
+| `analytics.region` | `global` | `eu` sends to `region1.google-analytics.com`, which keeps collection in the EU |
+| `analytics.events` | all | which of the events above are sent |
+| `analytics.inactive_days` | `30` | days without anyone using Lens before `lens_inactive` |
+
+`GET /api/v1/settings/analytics/status` shows whether it is on, this install's id, which events are on, and how the
+last send went. Changing these settings through the setup assistant always waits for an admin's approval, as
+telemetry does.
+
+## Refine later
+
+- A Settings page with charts of the numbers, and the wizard toggle (in progress).
+- Per-namespace numbers.
+- Optional per-person pseudonymous ids in GA, for multi-person installs.
diff --git a/docs/configuration.md b/docs/configuration.md
index 550663fb8..d94fcbba4 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -34,6 +34,7 @@ with AES-GCM and are write-only: the API reports whether one is set, never its v
| `LENS_SETUP_SETTINGS` | | settings to start with, as JSON `{"section": {"key": value}}` (or that JSON base64url-encoded): saved once at startup as if made in the app, so unlike the variables above they stay editable there. The [web installer](get-started.md#the-web-installer) writes its answers here |
| `LENS_SETUP_WIZARD` | | `off`: never show the setup wizard |
| `LENS_TELEMETRY` / `LENS_TELEMETRY_ENDPOINT` / `LENS_TELEMETRY_HEADERS` | off | opt-in OpenTelemetry traces and metrics, sent only to this OTLP/HTTP endpoint ([Telemetry](telemetry.md)); `LENS_TELEMETRY=off` keeps it off whatever Settings say |
+| `LENS_ANALYTICS` / `LENS_GA_MEASUREMENT_ID` / `LENS_GA_API_SECRET` | off | opt-in usage events sent to your own Google Analytics 4 property ([Usage analytics](analytics.md)) |
| `FRONTEND_URL` | `http://localhost:3000` | the web app's address as people use it: links in emails, and where apps send people to sign in ([OAuth](authentication.md#oauth)) |
| `CORS_ORIGINS` | `["http://localhost:3000"]` | origins allowed to call the API from a browser |
| `OPENAPI_URL` | `/openapi.json` | `""` disables `/docs` and the schema |
@@ -57,7 +58,7 @@ A fresh install (no accounts when the API first starts) walks its first admin th
5. **Apps and AI**: whether apps and AI assistants (Claude, ChatGPT, Cursor and other MCP clients) may sign people in
with their Lens account ([OAuth](authentication.md#oauth)), on unless turned off, how long their tokens last, and
the MCP server's address to add to an assistant.
-6. **Telemetry**: off unless chosen ([Telemetry](telemetry.md)).
+6. **Telemetry**: off unless chosen ([Telemetry](telemetry.md)), and Google Analytics usage events, off unless chosen ([Usage analytics](analytics.md)).
Every step can be skipped, and the whole wizard too; all of it stays in Settings. Values from the environment win and
show locked. Installs that already had accounts never see the wizard. The API side is `GET /api/v1/setup`,
diff --git a/fastapi_backend/.env.example b/fastapi_backend/.env.example
index fb9b937fd..dcd4d5a0c 100644
--- a/fastapi_backend/.env.example
+++ b/fastapi_backend/.env.example
@@ -57,5 +57,10 @@ CORS_ORIGINS=["http://localhost:3000"]
# LENS_TELEMETRY_ENDPOINT=http://localhost:4318 # your OpenTelemetry collector's OTLP/HTTP address
# LENS_TELEMETRY_HEADERS= # optional: key=value pairs, comma-separated, URL-encoded
+# --- Usage analytics (docs/analytics.md): counted events sent to your own Google Analytics 4 property, off unless on ---
+# LENS_ANALYTICS=on
+# LENS_GA_MEASUREMENT_ID=G-ABC123XYZ
+# LENS_GA_API_SECRET= # a Measurement Protocol API secret from the GA data stream
+
# Uncomment to disable /docs and /openapi.json
# OPENAPI_URL=""
diff --git a/fastapi_backend/app/api/v1/routes/admin.py b/fastapi_backend/app/api/v1/routes/admin.py
index 5249bfb4a..2f203aa4c 100644
--- a/fastapi_backend/app/api/v1/routes/admin.py
+++ b/fastapi_backend/app/api/v1/routes/admin.py
@@ -14,6 +14,7 @@
from app.api.deps import AdminReader, AdminWriter, Cfg, Db, domain_errors
from app.core.middleware import host_name
from app.domain import (
+ analytics,
anytopdf,
auth,
blobs,
@@ -33,6 +34,9 @@
tunnel,
)
from app.schemas.admin import (
+ AnalyticsMetrics,
+ AnalyticsStatus,
+ AnalyticsTestResult,
AuditEntry,
BridgeStatus,
BridgeTestResult,
@@ -279,6 +283,26 @@ def test_telemetry(user: AdminWriter, cfg: Cfg) -> TelemetryTestResult:
return TelemetryTestResult(ok=ok, error=error, ms=ms)
+@router.get("/settings/analytics/status")
+def analytics_status(user: AdminReader, cfg: Cfg, db: Db) -> AnalyticsStatus:
+ """Whether usage statistics go to Google Analytics, which events, and how the last send went. Off by default."""
+ return AnalyticsStatus.model_validate(analytics.status(db, cfg))
+
+
+@router.post("/settings/analytics/test")
+def test_analytics(user: AdminWriter, cfg: Cfg, db: Db) -> AnalyticsTestResult:
+ """Send one lens_test event to Google Analytics' validation server (checked there, not recorded), on or off."""
+ ok, error, messages = analytics.test(db, cfg)
+ return AnalyticsTestResult(ok=ok, error=error, messages=messages)
+
+
+@router.get("/admin/analytics")
+def usage_metrics(user: AdminReader, db: Db, days: int = Query(30, ge=1, le=180)) -> AnalyticsMetrics:
+ """Daily, weekly and monthly active people, churn, retention by sign-up week and feature use over the last `days`
+ days, worked out here from the activity ledger and sign-ins; nothing is sent anywhere."""
+ return AnalyticsMetrics.model_validate(analytics.metrics(db, days))
+
+
@router.get("/audit")
def list_audit(user: AdminReader, db: Db, limit: int = Query(200, ge=1, le=1000)) -> list[AuditEntry]:
"""Who changed what, newest first."""
diff --git a/fastapi_backend/app/api/v1/routes/setup.py b/fastapi_backend/app/api/v1/routes/setup.py
index cf8f085ba..62d4682f8 100644
--- a/fastapi_backend/app/api/v1/routes/setup.py
+++ b/fastapi_backend/app/api/v1/routes/setup.py
@@ -86,14 +86,23 @@ def save_storage(body: SetupStorage, user: AdminWriter, request: Request, db: Db
@router.put("/telemetry")
def save_telemetry(body: SetupTelemetry, user: AdminWriter, request: Request, db: Db) -> SetupSaved:
- """Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector). Off unless chosen."""
+ """Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector), and usage statistics
+ to your own Google Analytics property when `analytics` is true. Each is off unless chosen."""
try:
- saved = setup.save_telemetry(db, request.app.state.archive.base, body.enabled, body.endpoint, user.email)
+ base = request.app.state.archive.base
+ saved = setup.save_telemetry(db, base, body.enabled, body.endpoint, user.email)
+ shared = (
+ setup.save_analytics(db, base, body.analytics, body.measurement_id, body.api_secret, user.email)
+ if body.analytics is not None
+ else []
+ )
except ValueError as e:
raise HTTPException(400, str(e)) from None
if saved:
auth.audit(db, user.as_audit(), "settings.save", "telemetry", saved)
- return SetupSaved(saved=saved)
+ if shared:
+ auth.audit(db, user.as_audit(), "settings.save", "analytics", shared)
+ return SetupSaved(saved=saved + [f"analytics.{k}" for k in shared])
@router.put("/oauth")
diff --git a/fastapi_backend/app/domain/analytics.py b/fastapi_backend/app/domain/analytics.py
new file mode 100644
index 000000000..66afd15d1
--- /dev/null
+++ b/fastapi_backend/app/domain/analytics.py
@@ -0,0 +1,348 @@
+"""Usage analytics (docs/analytics.md): who uses Lens and how often, worked out on this machine, and optionally a
+handful of counted events sent to a Google Analytics 4 property the admin owns.
+
+**The numbers** (`metrics`) come from what Lens already keeps: the activity ledger's rows for API calls a person made
+(activity.py) and sign-ins in the audit log. Nothing new is collected for them and nothing leaves the machine: daily,
+weekly and monthly active people, stickiness, churn and retention by sign-up week, and which features get used.
+
+**Google Analytics** is off by default (analytics.enabled), and needs a GA4 measurement id and a Measurement Protocol
+API secret, set in Settings → Usage analytics, the setup wizard, or .env (LENS_ANALYTICS, LENS_GA_MEASUREMENT_ID,
+LENS_GA_API_SECRET). Turned on, one process sends once a day the events in EVENTS that analytics.events names, for this
+install as a whole: the install's random id (made here, never a person, host or address), the Lens version and
+platform, and counts. Never a name, email, title, file name, namespace, prompt or text. Churn across installs is then
+GA's retention report over lens_daily_active; lens_inactive marks an install nobody has used for
+analytics.inactive_days, which is how a self-hosted install that quietly stopped shows up.
+"""
+
+from __future__ import annotations
+
+import datetime as dt
+import logging
+import platform
+import re
+import uuid
+
+import httpx
+
+from . import store
+
+log = logging.getLogger("lens")
+R = store.R
+ROW = R("app_service", "analytics")
+HOSTS = {"global": "www.google-analytics.com", "eu": "region1.google-analytics.com"} # tests point it at a fake
+BATCH = 25 # GA4 takes up to 25 events a request
+MEASUREMENT_RX = re.compile(r"^G-[A-Z0-9]{4,20}$")
+
+# The events Lens can send, and what each carries. Names are prefixed so none clashes with GA's reserved ones.
+EVENTS = {
+ "lens_install": "Analytics first ran on this install. version, platform.",
+ "lens_setup_done": "The setup wizard was finished (or skipped). skipped (0 or 1).",
+ "lens_daily_active": "At least one person used Lens on the day before. active_users, actions.",
+ "lens_feature_used": "A feature was used on the day before, one event per feature. feature, actions, users.",
+ "lens_upgrade": "Lens runs a new version. from_version, to_version.",
+ "lens_inactive": "Nobody has used Lens for analytics.inactive_days (the uninstall-like signal). days.",
+ "lens_return": "Someone used Lens again after lens_inactive. days_away.",
+}
+FEATURE_RX = re.compile(r"^[a-z][a-z0-9_-]{0,39}$")
+SCOPES = ("namespaces", "spaces") # /namespaces/{ns}/recordings counts as recordings
+
+
+# ---------- what people did, from the activity ledger and sign-ins ----------
+def feature(action):
+ """The feature an activity row's action names: `POST /api/v1/namespaces/{ns}/recordings` is recordings. Actions are
+ route templates (activity.py), so this is code's wording, never what a person typed."""
+ _, _, path = (action or "").partition(" ")
+ parts = [p for p in path.split("/") if p][2:] # past api/v1
+ i = 0
+ while i < len(parts) and parts[i] in SCOPES and i + 1 < len(parts) and parts[i + 1].startswith("{"):
+ i += 2
+ name = parts[i] if i < len(parts) else (parts[0] if parts else "")
+ return name if FEATURE_RX.match(name) else "other"
+
+
+def _day(at):
+ return str(at or "")[:10]
+
+
+def _days(db, since):
+ """{(account, day): actions} for every person since `since` (a date): API calls that changed something, and
+ sign-ins (one each)."""
+ out = {}
+ for r in db.rows(
+ "SELECT actor, string::slice(at, 0, 10) AS day, count() AS n FROM activity "
+ "WHERE kind = 'in' AND actor != NONE AND at >= $s GROUP BY actor, day",
+ s=since.isoformat(),
+ ):
+ out[(str(r["actor"]), r["day"])] = out.get((str(r["actor"]), r["day"]), 0) + int(r["n"])
+ for r in db.rows(
+ "SELECT account, string::slice(at, 0, 10) AS day, count() AS n FROM audit_log "
+ "WHERE action = 'login' AND account != NONE AND at >= $s GROUP BY account, day",
+ s=since.isoformat(),
+ ):
+ out[(str(r["account"]), r["day"])] = out.get((str(r["account"]), r["day"]), 0) + int(r["n"])
+ return out
+
+
+def _features(db, since, until=None):
+ """[{feature, actions, users}] since `since` (and before `until`), most used first."""
+ rows = db.rows(
+ "SELECT action, actor, count() AS n FROM activity WHERE kind = 'in' AND actor != NONE AND at >= $s"
+ + (" AND at < $u" if until else "")
+ + " GROUP BY action, actor",
+ s=since.isoformat(),
+ u=until.isoformat() if until else None,
+ )
+ acc = {}
+ for r in rows:
+ f = acc.setdefault(feature(r["action"]), {"actions": 0, "users": set()})
+ f["actions"] += int(r["n"])
+ f["users"].add(str(r["actor"]))
+ return sorted(
+ ({"feature": k, "actions": v["actions"], "users": len(v["users"])} for k, v in acc.items()),
+ key=lambda f: (-f["actions"], f["feature"]),
+ )
+
+
+def _rate(part, whole):
+ return round(part / whole, 4) if whole else None
+
+
+def metrics(db, days=30, now=None, weeks=8):
+ """Active people, churn, retention and feature use over the last `days` days, worked out from the activity ledger
+ and sign-ins (docs/analytics.md#the-numbers). Churn: of the people active in the `days` before this period, the
+ share not active in it."""
+ now = now or dt.datetime.now(dt.timezone.utc)
+ today = now.date()
+ start = today - dt.timedelta(days=days - 1) # the period is the last `days` days, today included
+ prev = start - dt.timedelta(days=days)
+ cohort_start = today - dt.timedelta(days=today.weekday()) - dt.timedelta(weeks=weeks - 1)
+ since = min(prev, cohort_start)
+ seen = _days(db, since)
+ by_day, people = {}, {}
+ for (who, day), n in seen.items():
+ d = by_day.setdefault(day, {"users": set(), "actions": 0})
+ d["users"].add(who)
+ d["actions"] += n
+ people.setdefault(who, set()).add(day)
+
+ def active(a, b):
+ """The people active from day a to day b, both included."""
+ lo, hi = a.isoformat(), b.isoformat()
+ return {w for w, ds in people.items() if any(lo <= d <= hi for d in ds)}
+
+ daily = []
+ for i in range(days):
+ d = (start + dt.timedelta(days=i)).isoformat()
+ got = by_day.get(d) or {"users": (), "actions": 0}
+ daily.append({"day": d, "active_users": len(got["users"]), "actions": got["actions"]})
+ current, before = active(start, today), active(prev, start - dt.timedelta(days=1))
+ mau = len(active(today - dt.timedelta(days=29), today))
+ avg_dau = sum(x["active_users"] for x in daily) / days
+ accounts = db.rows("SELECT record::id(id) AS id, created_at, disabled FROM account")
+ churned = before - current
+
+ cohorts = []
+ for w in range(weeks):
+ wk = cohort_start + dt.timedelta(weeks=w)
+ members = {str(a["id"]) for a in accounts if wk.isoformat() <= _day(a.get("created_at")) < (wk + dt.timedelta(weeks=1)).isoformat()}
+ if not members:
+ continue
+ shares = []
+ for k in range(weeks - w):
+ a = wk + dt.timedelta(weeks=k)
+ back = active(a, min(a + dt.timedelta(days=6), today)) & members
+ shares.append(_rate(len(back), len(members)))
+ cohorts.append({"week": wk.isoformat(), "users": len(members), "retention": shares})
+
+ last = max((d for ds in people.values() for d in ds), default=None)
+ return {
+ "days": days,
+ "since": start.isoformat(),
+ "until": today.isoformat(),
+ "daily": daily,
+ "dau": len(by_day.get(today.isoformat(), {}).get("users", ())),
+ "wau": len(active(today - dt.timedelta(days=6), today)),
+ "mau": mau,
+ "stickiness": _rate(avg_dau, mau),
+ "accounts": sum(1 for a in accounts if not a.get("disabled")),
+ "active_users": len(current),
+ "new_users": sum(1 for a in accounts if _day(a.get("created_at")) >= start.isoformat()),
+ "churn": {
+ "previous_active": len(before),
+ "retained": len(before & current),
+ "churned": len(churned),
+ "rate": _rate(len(churned), len(before)),
+ "returning": len(current - before),
+ },
+ "retention": cohorts,
+ "features": _features(db, dt.datetime.combine(start, dt.time(), dt.timezone.utc)),
+ "last_active_day": last,
+ "days_since_active": (today - dt.date.fromisoformat(last)).days if last else None,
+ }
+
+
+# ---------- Google Analytics ----------
+def _section(cfg):
+ return (cfg or {}).get("analytics") or {}
+
+
+def configured(cfg):
+ """On, with a measurement id and an API secret to send with."""
+ a = _section(cfg)
+ return bool(a.get("enabled") and a.get("measurement_id") and isinstance(a.get("api_secret"), str) and a["api_secret"])
+
+
+def _state(db):
+ return db.one("SELECT * OMIT id FROM $r", r=ROW) or {}
+
+
+def _put(db, **fields):
+ db.q("UPSERT $r MERGE $d", r=ROW, d=fields)
+
+
+def client_id(db):
+ """This install's random id for Google Analytics: made once, here; not derived from anything about the machine."""
+ st = _state(db)
+ if st.get("client_id"):
+ return st["client_id"]
+ cid = str(uuid.uuid4())
+ _put(db, client_id=cid)
+ return _state(db).get("client_id") or cid
+
+
+def _url(cfg, debug=False):
+ a = _section(cfg)
+ base = HOSTS.get(a.get("region") or "global", HOSTS["global"])
+ base = base if "://" in base else f"https://{base}"
+ return f"{base}/{'debug/' if debug else ''}mp/collect"
+
+
+def _platform():
+ return f"{platform.system().lower() or 'unknown'}-{platform.machine().lower() or 'unknown'}"[:40]
+
+
+def payload(cfg, cid, events, day=None):
+ """The Measurement Protocol body: the install's id, its version and platform as user properties, and the events,
+ each counted as engaged so GA's active-user and retention reports include it."""
+ from . import __version__
+
+ session = int(dt.datetime.combine(day or dt.date.today(), dt.time(), dt.timezone.utc).timestamp())
+ return {
+ "client_id": cid,
+ "non_personalized_ads": True,
+ "user_properties": {"lens_version": {"value": __version__}, "lens_platform": {"value": _platform()}},
+ "events": [{"name": name, "params": {**params, "session_id": session, "engagement_time_msec": 1}} for name, params in events],
+ }
+
+
+def send(db, cfg, events, day=None, debug=False):
+ """Send events ([(name, params)]) now; (ok, error, validation messages). Only events analytics.events names."""
+ a = _section(cfg)
+ wanted = set(a.get("events") or EVENTS)
+ events = [(n, p) for n, p in events if n in wanted or (debug and n == "lens_test")]
+ if not events:
+ return True, None, []
+ cid, msgs = client_id(db), []
+ params = {"measurement_id": a.get("measurement_id"), "api_secret": a.get("api_secret")}
+ try:
+ for i in range(0, len(events), BATCH):
+ r = httpx.post(_url(cfg, debug), params=params, json=payload(cfg, cid, events[i : i + BATCH], day), timeout=10)
+ if r.status_code >= 300:
+ raise ValueError(f"Google Analytics answered {r.status_code}")
+ if debug:
+ msgs += [str(m.get("description") or m)[:300] for m in (r.json().get("validationMessages") or [])]
+ except (httpx.HTTPError, ValueError) as e:
+ error = f"couldn't send to Google Analytics: {type(e).__name__}: {e}"[:300].replace(a.get("api_secret") or "\0", "…")
+ if not debug:
+ _put(db, last_error=error, last_error_at=store.now())
+ return False, error, msgs
+ if not debug:
+ _put(db, last_sent_at=store.now(), last_sent=[n for n, _ in events], last_error=None)
+ return not msgs, None, msgs
+
+
+def _claim(db, day):
+ """True for the one process that gets to send `day`'s events."""
+ if _state(db).get("day") == day:
+ return False
+ client_id(db)
+ got = db.rows("UPDATE $r SET day = $d WHERE day != $d RETURN AFTER", r=ROW, d=day)
+ return bool(got)
+
+
+def tick(db, cfg, now=None):
+ """Once a day, while analytics is on and set up: send what happened since the last time (docs/analytics.md)."""
+ if not configured(cfg):
+ return []
+ from . import __version__
+
+ now = now or dt.datetime.now(dt.timezone.utc)
+ today = now.date()
+ if not _claim(db, today.isoformat()):
+ return []
+ st, events = _state(db), []
+ if not st.get("installed"):
+ events.append(("lens_install", {"version": __version__, "platform": _platform()}))
+ wiz = db.one("SELECT done_at, skipped FROM $r", r=R("setup", "wizard")) or {}
+ if wiz.get("done_at") and not st.get("setup_done"):
+ events.append(("lens_setup_done", {"skipped": 1 if wiz.get("skipped") else 0}))
+ if st.get("version") and st["version"] != __version__:
+ events.append(("lens_upgrade", {"from_version": st["version"], "to_version": __version__}))
+ yesterday = today - dt.timedelta(days=1)
+ seen = _days(db, yesterday)
+ users = {w for (w, d) in seen if d == yesterday.isoformat()}
+ inactive_days = int(_section(cfg).get("inactive_days") or 30)
+ marks = {"installed": True, "setup_done": bool(wiz.get("done_at")), "version": __version__}
+ if users:
+ events.append(
+ (
+ "lens_daily_active",
+ {"active_users": len(users), "actions": sum(n for (_, d), n in seen.items() if d == yesterday.isoformat())},
+ )
+ )
+ start = dt.datetime.combine(yesterday, dt.time(), dt.timezone.utc)
+ for f in _features(db, start, start + dt.timedelta(days=1))[:20]:
+ events.append(("lens_feature_used", f))
+ if st.get("inactive_since"):
+ away = (yesterday - dt.date.fromisoformat(st["inactive_since"])).days
+ events.append(("lens_return", {"days_away": max(away, inactive_days)}))
+ marks["inactive_since"] = None
+ elif not st.get("inactive_since"):
+ last = (db.one("SELECT at FROM activity WHERE kind = 'in' AND actor != NONE ORDER BY at DESC LIMIT 1") or {}).get("at")
+ last = last or (db.one("SELECT at FROM audit_log WHERE action = 'login' ORDER BY at DESC LIMIT 1") or {}).get("at")
+ since = dt.date.fromisoformat(_day(last)) if last else None
+ if since and (today - since).days >= inactive_days:
+ events.append(("lens_inactive", {"days": (today - since).days}))
+ marks["inactive_since"] = since.isoformat()
+ ok, error, _ = send(db, cfg, events, day=yesterday)
+ if ok:
+ _put(db, **marks)
+ else:
+ log.info("analytics: %s", error)
+ db.q("UPDATE $r SET day = NONE", r=ROW) # try again on the next tick
+ return [n for n, _ in events]
+
+
+def test(db, cfg):
+ """Send one lens_test event to GA's validation server (it's checked, not recorded); (ok, error, messages)."""
+ a = _section(cfg)
+ if not a.get("measurement_id") or not isinstance(a.get("api_secret"), str) or not a.get("api_secret"):
+ return False, "set the measurement id and the API secret first", []
+ from . import __version__
+
+ return send(db, cfg, [("lens_test", {"version": __version__})], debug=True)
+
+
+def status(db, cfg):
+ """Whether analytics is on and how the last send went, for Settings."""
+ a, st = _section(cfg), _state(db)
+ return {
+ "enabled": bool(a.get("enabled")),
+ "configured": configured(cfg),
+ "measurement_id": a.get("measurement_id"),
+ "client_id": st.get("client_id"),
+ "last_sent_at": st.get("last_sent_at"),
+ "last_sent": st.get("last_sent") or [],
+ "last_error": st.get("last_error"),
+ "events": [{"name": k, "description": v, "on": k in set(a.get("events") or EVENTS)} for k, v in EVENTS.items()],
+ }
diff --git a/fastapi_backend/app/domain/ops_tools.py b/fastapi_backend/app/domain/ops_tools.py
index 8b494c627..991e7510d 100644
--- a/fastapi_backend/app/domain/ops_tools.py
+++ b/fastapi_backend/app/domain/ops_tools.py
@@ -38,9 +38,11 @@
"bridge",
"matterbridge",
"telemetry",
+ "analytics",
"sensors",
)
-ALWAYS_ASK = {"telemetry", "sensors", "files"} # sensors open ports on the network; files moves where new files go
+# telemetry and analytics send data elsewhere; sensors open ports on the network; files moves where new files go
+ALWAYS_ASK = {"telemetry", "analytics", "sensors", "files"}
ADMIN_TOOLS = [
(
@@ -138,6 +140,7 @@ def status(db, cfg, base):
"max_upload_mb": view["storage"]["max_upload_mb"],
"jobs": q,
"telemetry_on": view["telemetry"]["enabled"],
+ "google_analytics_on": view["telemetry"]["analytics"]["enabled"],
# engines and models the workers are still fetching for themselves, or couldn't (Settings → Components)
"components_not_ready": fetching,
}
diff --git a/fastapi_backend/app/domain/routines.py b/fastapi_backend/app/domain/routines.py
index 1cc0b7b9d..ac6b1ae6c 100644
--- a/fastapi_backend/app/domain/routines.py
+++ b/fastapi_backend/app/domain/routines.py
@@ -24,6 +24,7 @@
from . import (
activity,
+ analytics,
budgets,
fedora,
jobs,
@@ -601,6 +602,11 @@ def run_due(db, cfg, log=print, now=None):
stamp = _iso(now)
sweep(db, now)
activity.tidy(db, cfg) # the activity ledger past activity.keep_days, once an hour
+ try:
+ analytics.tick(db, cfg, now) # once a day, while Google Analytics is on (docs/analytics.md)
+ except Exception as e: # noqa: BLE001 - the schedule carries on
+ if log:
+ log(f"analytics: {type(e).__name__}: {e}")
try:
budgets.sweep(db, cfg, every=budgets.SWEEP_SECONDS) # budgets that crossed warn_at or their cap: warned once
except Exception as e: # noqa: BLE001 - the schedule carries on
diff --git a/fastapi_backend/app/domain/settings.py b/fastapi_backend/app/domain/settings.py
index 4941151ea..9c2c49478 100644
--- a/fastapi_backend/app/domain/settings.py
+++ b/fastapi_backend/app/domain/settings.py
@@ -47,6 +47,7 @@
"matterbridge": None,
"notifications": None,
"telemetry": None,
+ "analytics": None,
# repo is a startup setting only (what gets installed isn't chosen in the web app)
"updates": ("check", "check_hours", "auto_install"),
"activity": None,
@@ -117,6 +118,7 @@
"bridge": ("token",),
"matterbridge": ("slack_token", "discord_token", "telegram_token", "matrix_password"),
"telemetry": ("headers",),
+ "analytics": ("api_secret",),
"fedora": ("password",),
"tunnel": ("token", "api_token"),
"documents": ("anytopdf_token",),
@@ -144,6 +146,7 @@
("tunnel", "mode"): {"off", "quick", "token", "managed"},
("files", "store"): {"local", "connection"},
("documents", "converter"): {"auto", "anytopdf", "lens"},
+ ("analytics", "region"): {"global", "eu"},
}
# Settings the environment (.env) sets, which win over archive.yaml and the app and show as locked there: the
# break-glass allowed hosts, the model provider so an install can be configured without the setup wizard, and
@@ -167,6 +170,9 @@
("telemetry", "enabled"): "LENS_TELEMETRY",
("telemetry", "endpoint"): "LENS_TELEMETRY_ENDPOINT",
("telemetry", "headers"): "LENS_TELEMETRY_HEADERS",
+ ("analytics", "enabled"): "LENS_ANALYTICS",
+ ("analytics", "measurement_id"): "LENS_GA_MEASUREMENT_ID",
+ ("analytics", "api_secret"): "LENS_GA_API_SECRET",
("fedora", "enabled"): "LENS_FEDORA",
("fedora", "url"): "LENS_FEDORA_URL",
("fedora", "user"): "LENS_FEDORA_USER",
@@ -396,6 +402,8 @@ def _check(section, key, value, default):
return _notify_setting(key, value)
if section == "telemetry":
return _telemetry_setting(key, value)
+ if section == "analytics" and key != "region":
+ return _analytics_setting(key, value)
if section == "updates":
return _updates_setting(key, value)
if section == "activity":
@@ -898,6 +906,31 @@ def _telemetry_setting(key, value):
return value
+def _analytics_setting(key, value):
+ from . import analytics
+
+ if key == "enabled":
+ if not isinstance(value, bool):
+ raise ValueError("analytics.enabled is true or false")
+ return value
+ if key == "measurement_id":
+ if value in (None, ""):
+ return None
+ v = value.strip().upper() if isinstance(value, str) else ""
+ if not analytics.MEASUREMENT_RX.match(v):
+ raise ValueError("analytics.measurement_id is a GA4 measurement id, like G-ABC123XYZ")
+ return v
+ if key == "events":
+ if not (isinstance(value, list) and all(isinstance(x, str) for x in value)) or set(value) - set(analytics.EVENTS):
+ raise ValueError(f"analytics.events is a list of: {', '.join(analytics.EVENTS)}")
+ return [e for e in analytics.EVENTS if e in value]
+ if key == "inactive_days":
+ if not (isinstance(value, int) and not isinstance(value, bool) and 1 <= value <= 365):
+ raise ValueError("analytics.inactive_days is a whole number of days from 1 to 365")
+ return value
+ raise ValueError(f"unknown setting analytics.{key}")
+
+
def _origin(url):
try:
u = urllib.parse.urlsplit(url or "")
diff --git a/fastapi_backend/app/domain/setup.py b/fastapi_backend/app/domain/setup.py
index a4ab3255a..fe79ebda0 100644
--- a/fastapi_backend/app/domain/setup.py
+++ b/fastapi_backend/app/domain/setup.py
@@ -178,6 +178,12 @@ def view(db, cfg):
"enabled": bool(shown["telemetry"].get("enabled")),
"endpoint": shown["telemetry"].get("endpoint"),
"locked": [k for k in settings.locked("telemetry") if k in ("enabled", "endpoint")],
+ "analytics": {
+ "enabled": bool(shown["analytics"].get("enabled")),
+ "measurement_id": shown["analytics"].get("measurement_id"),
+ "secret_set": isinstance(shown["analytics"].get("api_secret"), dict),
+ "locked": settings.locked("analytics"),
+ },
},
"oauth": {
"enabled": bool(shown["tokens"].get("oauth_enabled", True)),
@@ -219,6 +225,27 @@ def save_telemetry(db, cfg, enabled, endpoint, user=None):
return sorted(keep)
+def save_analytics(db, cfg, enabled, measurement_id=None, api_secret=None, user=None):
+ """Opt in to Google Analytics (or stay out), leaving out what the environment sets. On needs a measurement id and
+ an API secret, given here or saved before."""
+ locked = settings.locked("analytics")
+ values = {"enabled": bool(enabled)}
+ if (measurement_id or "").strip():
+ values["measurement_id"] = measurement_id.strip()
+ if (api_secret or "").strip():
+ values["api_secret"] = api_secret.strip()
+ keep = {k: v for k, v in values.items() if k not in locked}
+ if enabled:
+ shown = settings.effective(db, cfg, reveal=False)["analytics"]
+ if not (values.get("measurement_id") or shown.get("measurement_id")):
+ raise ValueError("Google Analytics needs a GA4 measurement id, like G-ABC123XYZ")
+ if not (values.get("api_secret") or shown.get("api_secret")):
+ raise ValueError("Google Analytics needs a Measurement Protocol API secret (Admin → Data streams in GA)")
+ if keep:
+ settings.save(db, cfg, "analytics", keep, user)
+ return sorted(keep)
+
+
def save_oauth(db, cfg, enabled, access_minutes=None, refresh_days=None, user=None):
"""Whether apps and MCP clients may sign people in through OAuth, and how long their tokens last."""
values = {"oauth_enabled": bool(enabled)}
diff --git a/fastapi_backend/app/domain/store.py b/fastapi_backend/app/domain/store.py
index cbae2c2df..5dfaf4dcc 100644
--- a/fastapi_backend/app/domain/store.py
+++ b/fastapi_backend/app/domain/store.py
@@ -482,6 +482,24 @@ def labels():
"service_name": "lens",
"prices": {},
},
+ # usage analytics (docs/analytics.md): the numbers are worked out locally from the activity ledger; Google Analytics
+ # is off unless an admin turns it on with a GA4 measurement id and a Measurement Protocol API secret (sealed)
+ "analytics": {
+ "enabled": False,
+ "measurement_id": None,
+ "api_secret": None,
+ "region": "global",
+ "events": [
+ "lens_install",
+ "lens_setup_done",
+ "lens_daily_active",
+ "lens_feature_used",
+ "lens_upgrade",
+ "lens_inactive",
+ "lens_return",
+ ],
+ "inactive_days": 30,
+ },
# new releases (docs/updates.md): check asks GitHub for repo's latest release every check_hours, for the web app's
# banner; auto_install installs one as soon as it's seen, through the machine's upgrade agent (install.sh sets it up)
"updates": {"check": True, "check_hours": 24, "auto_install": False, "repo": "adeelahmad/lense"},
diff --git a/fastapi_backend/app/schemas/admin.py b/fastapi_backend/app/schemas/admin.py
index 6a843ad4b..bba0f0bc6 100644
--- a/fastapi_backend/app/schemas/admin.py
+++ b/fastapi_backend/app/schemas/admin.py
@@ -223,6 +223,91 @@ class TelemetryTestResult(ResponseModel):
ms: int | None = None
+class AnalyticsEvent(ResponseModel):
+ name: str
+ description: str
+ # analytics.events names it
+ on: bool
+
+
+class AnalyticsStatus(ResponseModel):
+ """Whether usage statistics go to Google Analytics, and how the last send went (docs/analytics.md)."""
+
+ enabled: bool
+ # on, with a measurement id and an API secret
+ configured: bool
+ measurement_id: str | None = None
+ # this install's random id in Google Analytics
+ client_id: str | None = None
+ last_sent_at: str | None = None
+ last_sent: list[str] = []
+ last_error: str | None = None
+ events: list[AnalyticsEvent]
+
+
+class AnalyticsTestResult(ResponseModel):
+ ok: bool
+ error: str | None = None
+ # what Google Analytics' validation server said about the test event
+ messages: list[str] = []
+
+
+class AnalyticsDay(ResponseModel):
+ day: str
+ active_users: int
+ actions: int
+
+
+class AnalyticsChurn(ResponseModel):
+ # people active in the period before this one
+ previous_active: int
+ # of those, active again in this one
+ retained: int
+ # of those, not active in this one
+ churned: int
+ # churned / previous_active; null with no one before
+ rate: float | None = None
+ # active now, not in the period before
+ returning: int
+
+
+class AnalyticsCohort(ResponseModel):
+ # the Monday of the week these accounts were made
+ week: str
+ users: int
+ # the share of them active in that week and each week after
+ retention: list[float | None]
+
+
+class AnalyticsFeature(ResponseModel):
+ feature: str
+ actions: int
+ users: int
+
+
+class AnalyticsMetrics(ResponseModel):
+ """Active people, churn, retention and feature use, worked out on this machine from the activity ledger and
+ sign-ins (docs/analytics.md#the-numbers)."""
+
+ days: int
+ since: str
+ until: str
+ daily: list[AnalyticsDay]
+ dau: int
+ wau: int
+ mau: int
+ # average daily active / monthly active
+ stickiness: float | None = None
+ accounts: int
+ active_users: int
+ new_users: int
+ churn: AnalyticsChurn
+ retention: list[AnalyticsCohort]
+ features: list[AnalyticsFeature]
+ last_active_day: str | None = None
+ days_since_active: int | None = None
+
+
class FileStoreTry(RequestModel):
"""Where to try keeping files, before saving it (Settings → Storage); nothing given tries what is saved."""
diff --git a/fastapi_backend/app/schemas/setup.py b/fastapi_backend/app/schemas/setup.py
index 4f9498f74..fba566d81 100644
--- a/fastapi_backend/app/schemas/setup.py
+++ b/fastapi_backend/app/schemas/setup.py
@@ -30,11 +30,22 @@ class SetupStorageView(ResponseModel):
watches: int
+class SetupAnalyticsView(ResponseModel):
+ enabled: bool
+ measurement_id: str | None = None
+ # an API secret is saved (it is never shown)
+ secret_set: bool
+ # what .env sets (LENS_ANALYTICS, LENS_GA_MEASUREMENT_ID, LENS_GA_API_SECRET): shown locked
+ locked: list[str]
+
+
class SetupTelemetryView(ResponseModel):
enabled: bool
endpoint: str | None = None
# what .env sets (LENS_TELEMETRY, LENS_TELEMETRY_ENDPOINT): shown locked
locked: list[str]
+ # usage statistics sent to the admin's own Google Analytics property (docs/analytics.md); off by default
+ analytics: SetupAnalyticsView
class SetupOAuthView(ResponseModel):
@@ -85,6 +96,11 @@ class SetupTelemetry(RequestModel):
enabled: bool = False
# the OTLP/HTTP address of a collector, like http://localhost:4318; needed to turn telemetry on
endpoint: str | None = Field(default=None, max_length=500)
+ # Google Analytics (docs/analytics.md): left out, unchanged
+ analytics: bool | None = None
+ measurement_id: str | None = Field(default=None, max_length=40)
+ # empty: leave the saved secret as it is
+ api_secret: str | None = Field(default=None, max_length=200)
class SetupOAuth(RequestModel):
diff --git a/fastapi_backend/tests/api/test_analytics.py b/fastapi_backend/tests/api/test_analytics.py
new file mode 100644
index 000000000..b3e50d3c3
--- /dev/null
+++ b/fastapi_backend/tests/api/test_analytics.py
@@ -0,0 +1,204 @@
+"""Usage analytics: the numbers worked out locally, and Google Analytics off by default, opt-in, counts only."""
+
+from __future__ import annotations
+
+import datetime as dt
+
+import httpx
+import pytest
+
+from app.domain import __version__, analytics, routines, store
+from tests.helpers import login, make_user
+
+NOW = dt.datetime(2026, 10, 10, 12, tzinfo=dt.UTC)
+
+
+def admin(client, db):
+ make_user(db, "root@x.io", "root password 1", admin=True)
+ return login(client, "root@x.io", "root password 1")
+
+
+def act(db, who, day, action="POST /api/v1/namespaces/{ns}/recordings", n=1):
+ for _ in range(n):
+ db.q("CREATE activity CONTENT $d", d={"at": f"{day}T10:00:00.000+00:00", "kind": "in", "actor": who, "action": action, "ok": True})
+
+
+def account(db, uid, created):
+ db.q("UPSERT $r CONTENT $d", r=store.R("account", uid), d={"email": f"u{uid}@x.io", "created_at": f"{created}T09:00:00+00:00"})
+
+
+class Sent(list):
+ state: dict
+
+
+@pytest.fixture
+def ga(monkeypatch):
+ """A fake Google Analytics: records each request; `status` sets its answer."""
+ sent = Sent()
+ state = {"status": 204, "messages": []}
+
+ def post(url, params=None, json=None, timeout=None):
+ sent.append({"url": url, "params": params, "body": json})
+ return httpx.Response(state["status"], json={"validationMessages": state["messages"]}, request=httpx.Request("POST", url))
+
+ monkeypatch.setattr(analytics.httpx, "post", post)
+ sent.state = state
+ return sent
+
+
+def on(cfg, **more):
+ cfg["analytics"].update(enabled=True, measurement_id="G-TEST1234", api_secret="s3cret", **more)
+ return cfg
+
+
+def names(sent):
+ return [e["name"] for s in sent for e in s["body"]["events"]]
+
+
+def test_feature_names_come_from_route_templates():
+ assert analytics.feature("POST /api/v1/namespaces/{ns}/recordings") == "recordings"
+ assert analytics.feature("PUT /api/v1/chats/{cid}") == "chats"
+ assert analytics.feature("POST /api/v1/namespaces") == "namespaces"
+ assert analytics.feature("POST /api/v1/{weird}") == "other"
+ assert analytics.feature("") == "other"
+
+
+def test_metrics_active_churn_retention(db):
+ today = NOW.date()
+
+ def d(n):
+ return (today - dt.timedelta(days=n)).isoformat()
+
+ # 1 and 2 were active in the previous 7 days; only 1 came back; 3 is new this week
+ account(db, 1, d(40))
+ account(db, 2, d(40))
+ account(db, 3, d(2))
+ act(db, 1, d(10))
+ act(db, 2, d(9))
+ act(db, 1, d(1), "PUT /api/v1/chats/{cid}", n=2)
+ act(db, 3, d(0))
+ db.q("CREATE audit_log CONTENT $d", d={"at": f"{d(0)}T08:00:00+00:00", "account": 1, "action": "login"})
+ m = analytics.metrics(db, days=7, now=NOW)
+ assert (m["since"], m["until"], len(m["daily"])) == (d(6), d(0), 7)
+ assert m["daily"][-1] == {"day": d(0), "active_users": 2, "actions": 2}
+ assert (m["dau"], m["wau"], m["mau"], m["active_users"], m["accounts"], m["new_users"]) == (2, 2, 3, 2, 3, 1)
+ assert m["churn"] == {"previous_active": 2, "retained": 1, "churned": 1, "rate": 0.5, "returning": 1}
+ assert m["stickiness"] == round((1 + 2) / 7 / 3, 4)
+ assert [f["feature"] for f in m["features"]] == ["chats", "recordings"]
+ assert m["features"][0] == {"feature": "chats", "actions": 2, "users": 1}
+ assert (m["last_active_day"], m["days_since_active"]) == (d(0), 0)
+ week = [c for c in m["retention"] if c["users"] == 1]
+ assert week and week[-1]["retention"][0] == 1.0 # 3 was active in its sign-up week
+
+
+def test_off_by_default_sends_nothing(db, cfg, ga):
+ act(db, 1, (NOW.date() - dt.timedelta(days=1)).isoformat())
+ assert cfg["analytics"]["enabled"] is False
+ assert analytics.tick(db, cfg, NOW) == []
+ routines.run_due(db, cfg, log=None, now=NOW)
+ assert ga == []
+ # on without a secret isn't set up either
+ cfg["analytics"].update(enabled=True, measurement_id="G-TEST1234")
+ assert analytics.tick(db, cfg, NOW) == []
+ assert ga == []
+
+
+def test_daily_events_once_a_day_counts_only(db, cfg, ga):
+ on(cfg)
+ yesterday = (NOW.date() - dt.timedelta(days=1)).isoformat()
+ act(db, 7, yesterday, n=3)
+ act(db, 8, yesterday, "PUT /api/v1/chats/{cid}")
+ db.q("UPSERT setup:wizard CONTENT {done_at: '2026-10-01T00:00:00+00:00', skipped: false}")
+ got = analytics.tick(db, cfg, NOW)
+ assert got == ["lens_install", "lens_setup_done", "lens_daily_active", "lens_feature_used", "lens_feature_used"]
+ assert analytics.tick(db, cfg, NOW + dt.timedelta(hours=3)) == [] # once a day
+ (req,) = ga
+ assert req["url"] == "https://www.google-analytics.com/mp/collect"
+ assert req["params"] == {"measurement_id": "G-TEST1234", "api_secret": "s3cret"}
+ body = req["body"]
+ assert body["client_id"] == analytics.client_id(db)
+ assert body["user_properties"]["lens_version"] == {"value": __version__}
+ ev = {e["name"]: e["params"] for e in body["events"]}
+ assert ev["lens_daily_active"]["active_users"] == 2 and ev["lens_daily_active"]["actions"] == 4
+ assert ev["lens_daily_active"]["engagement_time_msec"] == 1
+ feats = [e["params"] for e in body["events"] if e["name"] == "lens_feature_used"]
+ assert [(f["feature"], f["actions"], f["users"]) for f in feats] == [("recordings", 3, 1), ("chats", 1, 1)]
+ # nothing personal: no emails, account ids or paths
+ text = str(body)
+ assert "x.io" not in text and "{cid}" not in text and "'actor'" not in text
+ # the next day: no install or setup again; a new version is an upgrade
+ db.q("UPDATE $r SET version = '0.0.1'", r=analytics.ROW)
+ assert analytics.tick(db, cfg, NOW + dt.timedelta(days=1)) == ["lens_upgrade"]
+ assert ga[-1]["body"]["events"][0]["params"]["from_version"] == "0.0.1"
+
+
+def test_inactive_then_return(db, cfg, ga):
+ on(cfg, inactive_days=5)
+ act(db, 1, (NOW.date() - dt.timedelta(days=9)).isoformat())
+ assert analytics.tick(db, cfg, NOW) == ["lens_install", "lens_inactive"]
+ assert ga[-1]["body"]["events"][1]["params"]["days"] == 9
+ assert "lens_inactive" not in analytics.tick(db, cfg, NOW + dt.timedelta(days=1)) # once per stretch
+ act(db, 1, (NOW.date() + dt.timedelta(days=1)).isoformat())
+ got = analytics.tick(db, cfg, NOW + dt.timedelta(days=2))
+ assert "lens_return" in got and "lens_daily_active" in got
+
+
+def test_events_can_be_left_out_and_failures_retry(db, cfg, ga):
+ on(cfg, events=["lens_install"])
+ act(db, 1, (NOW.date() - dt.timedelta(days=1)).isoformat())
+ ga.state["status"] = 500
+ analytics.tick(db, cfg, NOW)
+ assert names(ga) == ["lens_install"]
+ assert "500" in analytics.status(db, cfg)["last_error"]
+ ga.state["status"] = 204
+ analytics.tick(db, cfg, NOW + dt.timedelta(minutes=5)) # the failed day is tried again
+ assert names(ga) == ["lens_install", "lens_install"]
+ st = analytics.status(db, cfg)
+ assert st["last_error"] is None and st["last_sent"] == ["lens_install"]
+
+
+def test_settings_api_status_and_test(client, db, ga):
+ h = admin(client, db)
+ a = client.get("/api/v1/settings", headers=h).json()["analytics"]
+ assert a["values"]["enabled"] is False and a["values"]["api_secret"] == {"secret": True, "set": False}
+
+ def put(body):
+ return client.put("/api/v1/settings/analytics", json=body, headers=h)
+
+ for bad in ({"measurement_id": "UA-1234-1"}, {"events": ["page_view"]}, {"inactive_days": 0}, {"region": "mars"}, {"enabled": "yes"}):
+ assert put(bad).status_code == 400, bad
+ r = client.post("/api/v1/settings/analytics/test", headers=h).json()
+ assert not r["ok"] and "measurement id" in r["error"]
+ assert put({"measurement_id": "g-test1234", "api_secret": "s3cret", "region": "eu"}).status_code == 200
+ a = client.get("/api/v1/settings", headers=h).json()["analytics"]["values"]
+ assert a["measurement_id"] == "G-TEST1234" and a["api_secret"] == {"secret": True, "set": True}
+ ga.state["messages"] = [{"description": "event name is bad"}]
+ r = client.post("/api/v1/settings/analytics/test", headers=h).json()
+ assert r == {"ok": False, "error": None, "messages": ["event name is bad"]}
+ assert ga[-1]["url"] == "https://region1.google-analytics.com/debug/mp/collect"
+ assert ga[-1]["body"]["events"][0]["name"] == "lens_test"
+ s = client.get("/api/v1/settings/analytics/status", headers=h).json()
+ assert s["enabled"] is False and s["configured"] is False and len(s["events"]) == len(analytics.EVENTS)
+ assert all(e["on"] for e in s["events"])
+ m = client.get("/api/v1/admin/analytics?days=7", headers=h).json()
+ assert m["days"] == 7 and m["accounts"] == 1 and m["dau"] == 1 # the sign-in above
+ make_user(db, "ed@x.io", "ed password 12")
+ h2 = login(client, "ed@x.io", "ed password 12")
+ assert client.get("/api/v1/admin/analytics", headers=h2).status_code == 403
+
+
+def test_setup_wizard_opt_in(client, db):
+ h = admin(client, db)
+ r = client.put("/api/v1/setup/telemetry", json={"enabled": False, "analytics": True}, headers=h)
+ assert r.status_code == 400 and "measurement id" in r.json()["detail"]
+ r = client.put(
+ "/api/v1/setup/telemetry",
+ json={"enabled": False, "analytics": True, "measurement_id": "G-TEST1234", "api_secret": "s3cret"},
+ headers=h,
+ )
+ assert r.status_code == 200 and "analytics.enabled" in r.json()["saved"]
+ v = client.get("/api/v1/setup", headers=h).json()["telemetry"]["analytics"]
+ assert v == {"enabled": True, "measurement_id": "G-TEST1234", "secret_set": True, "locked": []}
+ # off again keeps the id and secret, and no secret is needed to turn it back on
+ client.put("/api/v1/setup/telemetry", json={"enabled": False, "analytics": False}, headers=h)
+ assert client.put("/api/v1/setup/telemetry", json={"enabled": False, "analytics": True}, headers=h).status_code == 200
diff --git a/fastapi_backend/tests/api/test_telemetry.py b/fastapi_backend/tests/api/test_telemetry.py
index dc90423b6..852142373 100644
--- a/fastapi_backend/tests/api/test_telemetry.py
+++ b/fastapi_backend/tests/api/test_telemetry.py
@@ -212,16 +212,31 @@ def test_environment_turns_it_on_or_keeps_it_off(client, db, collector, monkeypa
def test_setup_wizard_opt_in(client, db, collector):
h = admin(client, db)
v = client.get("/api/v1/setup", headers=h).json()["telemetry"]
- assert v == {"enabled": False, "endpoint": None, "locked": []}
+ assert v == {
+ "enabled": False,
+ "endpoint": None,
+ "locked": [],
+ "analytics": {"enabled": False, "measurement_id": None, "secret_set": False, "locked": []},
+ }
r = client.put("/api/v1/setup/telemetry", json={"enabled": True}, headers=h)
assert r.status_code == 400 and "endpoint" in r.json()["detail"]
assert client.put("/api/v1/setup/telemetry", json={"enabled": True, "endpoint": "nope"}, headers=h).status_code == 400
r = client.put("/api/v1/setup/telemetry", json={"enabled": True, "endpoint": collector}, headers=h)
assert r.json()["saved"] == ["enabled", "endpoint"]
- assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == {"enabled": True, "endpoint": collector, "locked": []}
+ assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == {
+ "enabled": True,
+ "endpoint": collector,
+ "locked": [],
+ "analytics": {"enabled": False, "measurement_id": None, "secret_set": False, "locked": []},
+ }
# off keeps the address for later
client.put("/api/v1/setup/telemetry", json={"enabled": False}, headers=h)
- assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == {"enabled": False, "endpoint": collector, "locked": []}
+ assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == {
+ "enabled": False,
+ "endpoint": collector,
+ "locked": [],
+ "analytics": {"enabled": False, "measurement_id": None, "secret_set": False, "locked": []},
+ }
audit = client.get("/api/v1/audit", headers=h).json()
assert any(a["action"] == "settings.save" and a["target"] == "telemetry" for a in audit)
diff --git a/mkdocs.yml b/mkdocs.yml
index d1f90077a..289851ddd 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -79,6 +79,7 @@ nav:
- Chat rooms: chat-rooms.md
- Sensors: sensors.md
- Telemetry: telemetry.md
+ - Usage analytics: analytics.md
- Activity and costs: activity.md
- Budgets: budgets.md
- IIIF: iiif.md
diff --git a/nextjs-frontend/__tests__/setupWizard.test.tsx b/nextjs-frontend/__tests__/setupWizard.test.tsx
index 0fcb1de53..36b3bb9ff 100644
--- a/nextjs-frontend/__tests__/setupWizard.test.tsx
+++ b/nextjs-frontend/__tests__/setupWizard.test.tsx
@@ -41,7 +41,12 @@ const VIEW = (over: Partial = {}): SetupView => ({
max_upload_mb: 4096,
watches: 0,
},
- telemetry: { enabled: false, endpoint: null, locked: [] },
+ telemetry: {
+ enabled: false,
+ endpoint: null,
+ locked: [],
+ analytics: { enabled: false, measurement_id: null, secret_set: false, locked: [] },
+ },
oauth: { enabled: true, access_minutes: 60, refresh_days: 30 },
...over,
});
diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts
index 32ab2f424..a50ac936a 100644
--- a/nextjs-frontend/app/openapi-client/index.ts
+++ b/nextjs-frontend/app/openapi-client/index.ts
@@ -96,6 +96,17 @@ export type {
AddVaultPasskeyErrors,
AddVaultPasskeyResponse,
AddVaultPasskeyResponses,
+ AnalyticsChurn,
+ AnalyticsCohort,
+ AnalyticsDay,
+ AnalyticsEvent,
+ AnalyticsFeature,
+ AnalyticsMetrics,
+ AnalyticsStatus,
+ AnalyticsStatusData,
+ AnalyticsStatusResponse,
+ AnalyticsStatusResponses,
+ AnalyticsTestResult,
AnswerCheck,
AnswerData,
AnswerError,
@@ -2378,6 +2389,7 @@ export type {
SetRecordingContentTypeResponse,
SetRecordingContentTypeResponses,
SetupAdmin,
+ SetupAnalyticsView,
SetupData,
SetupError,
SetupErrors,
@@ -2535,6 +2547,9 @@ export type {
TemplateVersionCreate,
TemplateVersionInfo,
TermSuggestion,
+ TestAnalyticsData,
+ TestAnalyticsResponse,
+ TestAnalyticsResponses,
TestBridgeData,
TestBridgeResponse,
TestBridgeResponses,
@@ -2861,6 +2876,11 @@ export type {
UploadLimitsResponse,
UploadLimitsResponses,
UploadStart,
+ UsageMetricsData,
+ UsageMetricsError,
+ UsageMetricsErrors,
+ UsageMetricsResponse,
+ UsageMetricsResponses,
UserCreate,
UserPublic,
UserUpdate,
diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts
index 6e5801f34..bacf4113b 100644
--- a/nextjs-frontend/app/openapi-client/sdk.gen.ts
+++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts
@@ -34,6 +34,8 @@ import type {
AddVaultPasskeyData,
AddVaultPasskeyErrors,
AddVaultPasskeyResponses,
+ AnalyticsStatusData,
+ AnalyticsStatusResponses,
AnswerData,
AnswerErrors,
AnswerResponses,
@@ -1221,6 +1223,8 @@ import type {
TakeBackupResponses,
TelemetryStatusData,
TelemetryStatusResponses,
+ TestAnalyticsData,
+ TestAnalyticsResponses,
TestBridgeData,
TestBridgeResponses,
TestConversionNodeData,
@@ -1405,6 +1409,9 @@ import type {
UpdateWorkflowResponses,
UploadLimitsData,
UploadLimitsResponses,
+ UsageMetricsData,
+ UsageMetricsErrors,
+ UsageMetricsResponses,
VaultOptionsData,
VaultOptionsErrors,
VaultOptionsResponses,
@@ -2621,6 +2628,49 @@ export class Admin {
});
}
+ /**
+ * Analytics Status
+ *
+ * Whether usage statistics go to Google Analytics, which events, and how the last send went. Off by default.
+ */
+ public static analyticsStatus(
+ options?: Options,
+ ): RequestResult {
+ return (options?.client ?? client).get({
+ url: "/api/v1/settings/analytics/status",
+ ...options,
+ });
+ }
+
+ /**
+ * Test Analytics
+ *
+ * Send one lens_test event to Google Analytics' validation server (checked there, not recorded), on or off.
+ */
+ public static testAnalytics(
+ options?: Options,
+ ): RequestResult {
+ return (options?.client ?? client).post({
+ url: "/api/v1/settings/analytics/test",
+ ...options,
+ });
+ }
+
+ /**
+ * Usage Metrics
+ *
+ * Daily, weekly and monthly active people, churn, retention by sign-up week and feature use over the last `days`
+ * days, worked out here from the activity ledger and sign-ins; nothing is sent anywhere.
+ */
+ public static usageMetrics(
+ options?: Options,
+ ): RequestResult {
+ return (options?.client ?? client).get({
+ url: "/api/v1/admin/analytics",
+ ...options,
+ });
+ }
+
/**
* List Audit
*
@@ -3060,7 +3110,8 @@ export class Setup {
/**
* Save Telemetry
*
- * Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector). Off unless chosen.
+ * Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector), and usage statistics
+ * to your own Google Analytics property when `analytics` is true. Each is off unless chosen.
*/
public static saveTelemetry(
options: Options,
diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts
index fd83bbb05..34e1d4dc1 100644
--- a/nextjs-frontend/app/openapi-client/types.gen.ts
+++ b/nextjs-frontend/app/openapi-client/types.gen.ts
@@ -288,6 +288,240 @@ export type ActivityTotals = {
[key: string]: unknown;
};
+/**
+ * AnalyticsChurn
+ */
+export type AnalyticsChurn = {
+ /**
+ * Previous Active
+ */
+ previous_active: number;
+ /**
+ * Retained
+ */
+ retained: number;
+ /**
+ * Churned
+ */
+ churned: number;
+ /**
+ * Rate
+ */
+ rate?: number | null;
+ /**
+ * Returning
+ */
+ returning: number;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsCohort
+ */
+export type AnalyticsCohort = {
+ /**
+ * Week
+ */
+ week: string;
+ /**
+ * Users
+ */
+ users: number;
+ /**
+ * Retention
+ */
+ retention: Array;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsDay
+ */
+export type AnalyticsDay = {
+ /**
+ * Day
+ */
+ day: string;
+ /**
+ * Active Users
+ */
+ active_users: number;
+ /**
+ * Actions
+ */
+ actions: number;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsEvent
+ */
+export type AnalyticsEvent = {
+ /**
+ * Name
+ */
+ name: string;
+ /**
+ * Description
+ */
+ description: string;
+ /**
+ * On
+ */
+ on: boolean;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsFeature
+ */
+export type AnalyticsFeature = {
+ /**
+ * Feature
+ */
+ feature: string;
+ /**
+ * Actions
+ */
+ actions: number;
+ /**
+ * Users
+ */
+ users: number;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsMetrics
+ *
+ * Active people, churn, retention and feature use, worked out on this machine from the activity ledger and
+ * sign-ins (docs/analytics.md#the-numbers).
+ */
+export type AnalyticsMetrics = {
+ /**
+ * Days
+ */
+ days: number;
+ /**
+ * Since
+ */
+ since: string;
+ /**
+ * Until
+ */
+ until: string;
+ /**
+ * Daily
+ */
+ daily: Array;
+ /**
+ * Dau
+ */
+ dau: number;
+ /**
+ * Wau
+ */
+ wau: number;
+ /**
+ * Mau
+ */
+ mau: number;
+ /**
+ * Stickiness
+ */
+ stickiness?: number | null;
+ /**
+ * Accounts
+ */
+ accounts: number;
+ /**
+ * Active Users
+ */
+ active_users: number;
+ /**
+ * New Users
+ */
+ new_users: number;
+ churn: AnalyticsChurn;
+ /**
+ * Retention
+ */
+ retention: Array;
+ /**
+ * Features
+ */
+ features: Array;
+ /**
+ * Last Active Day
+ */
+ last_active_day?: string | null;
+ /**
+ * Days Since Active
+ */
+ days_since_active?: number | null;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsStatus
+ *
+ * Whether usage statistics go to Google Analytics, and how the last send went (docs/analytics.md).
+ */
+export type AnalyticsStatus = {
+ /**
+ * Enabled
+ */
+ enabled: boolean;
+ /**
+ * Configured
+ */
+ configured: boolean;
+ /**
+ * Measurement Id
+ */
+ measurement_id?: string | null;
+ /**
+ * Client Id
+ */
+ client_id?: string | null;
+ /**
+ * Last Sent At
+ */
+ last_sent_at?: string | null;
+ /**
+ * Last Sent
+ */
+ last_sent?: Array;
+ /**
+ * Last Error
+ */
+ last_error?: string | null;
+ /**
+ * Events
+ */
+ events: Array;
+ [key: string]: unknown;
+};
+
+/**
+ * AnalyticsTestResult
+ */
+export type AnalyticsTestResult = {
+ /**
+ * Ok
+ */
+ ok: boolean;
+ /**
+ * Error
+ */
+ error?: string | null;
+ /**
+ * Messages
+ */
+ messages?: Array;
+ [key: string]: unknown;
+};
+
/**
* AnswerCheck
*/
@@ -13742,6 +13976,29 @@ export type SetupAdmin = {
[key: string]: unknown;
};
+/**
+ * SetupAnalyticsView
+ */
+export type SetupAnalyticsView = {
+ /**
+ * Enabled
+ */
+ enabled: boolean;
+ /**
+ * Measurement Id
+ */
+ measurement_id?: string | null;
+ /**
+ * Secret Set
+ */
+ secret_set: boolean;
+ /**
+ * Locked
+ */
+ locked: Array;
+ [key: string]: unknown;
+};
+
/**
* SetupFinish
*/
@@ -13955,6 +14212,18 @@ export type SetupTelemetry = {
* Endpoint
*/
endpoint?: string | null;
+ /**
+ * Analytics
+ */
+ analytics?: boolean | null;
+ /**
+ * Measurement Id
+ */
+ measurement_id?: string | null;
+ /**
+ * Api Secret
+ */
+ api_secret?: string | null;
};
/**
@@ -13973,6 +14242,7 @@ export type SetupTelemetryView = {
* Locked
*/
locked: Array;
+ analytics: SetupAnalyticsView;
[key: string]: unknown;
};
@@ -19193,6 +19463,68 @@ export type TestTelemetryResponses = {
export type TestTelemetryResponse = TestTelemetryResponses[keyof TestTelemetryResponses];
+export type AnalyticsStatusData = {
+ body?: never;
+ path?: never;
+ query?: never;
+ url: "/api/v1/settings/analytics/status";
+};
+
+export type AnalyticsStatusResponses = {
+ /**
+ * Successful Response
+ */
+ 200: AnalyticsStatus;
+};
+
+export type AnalyticsStatusResponse = AnalyticsStatusResponses[keyof AnalyticsStatusResponses];
+
+export type TestAnalyticsData = {
+ body?: never;
+ path?: never;
+ query?: never;
+ url: "/api/v1/settings/analytics/test";
+};
+
+export type TestAnalyticsResponses = {
+ /**
+ * Successful Response
+ */
+ 200: AnalyticsTestResult;
+};
+
+export type TestAnalyticsResponse = TestAnalyticsResponses[keyof TestAnalyticsResponses];
+
+export type UsageMetricsData = {
+ body?: never;
+ path?: never;
+ query?: {
+ /**
+ * Days
+ */
+ days?: number;
+ };
+ url: "/api/v1/admin/analytics";
+};
+
+export type UsageMetricsErrors = {
+ /**
+ * Validation Error
+ */
+ 422: HttpValidationError;
+};
+
+export type UsageMetricsError = UsageMetricsErrors[keyof UsageMetricsErrors];
+
+export type UsageMetricsResponses = {
+ /**
+ * Successful Response
+ */
+ 200: AnalyticsMetrics;
+};
+
+export type UsageMetricsResponse = UsageMetricsResponses[keyof UsageMetricsResponses];
+
export type ListAuditData = {
body?: never;
path?: never;
diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json
index cfe128052..fcb2cf488 100644
--- a/nextjs-frontend/openapi.json
+++ b/nextjs-frontend/openapi.json
@@ -2732,6 +2732,96 @@
}
}
},
+ "/api/v1/settings/analytics/status": {
+ "get": {
+ "tags": [
+ "admin"
+ ],
+ "summary": "Analytics Status",
+ "description": "Whether usage statistics go to Google Analytics, which events, and how the last send went. Off by default.",
+ "operationId": "analytics_status",
+ "responses": {
+ "200": {
+ "description": "Successful Response",
+ "content": {
+ "application/json": {
+ "schema": {
+ "$ref": "#/components/schemas/AnalyticsStatus"
+ }
+ }
+ }
+ }
+ }
+ }
+ },
+ "/api/v1/settings/analytics/test": {
+ "post": {
+ "tags": [
+ "admin"
+ ],
+ "summary": "Test Analytics",
+ "description": "Send one lens_test event to Google Analytics' validation server (checked there, not recorded), on or off.",
+ "operationId": "test_analytics",
+ "responses": {
+ "200": {
+ "description": "Successful Response",
+ "content": {
+ "application/json": {
+ "schema": {
+ "$ref": "#/components/schemas/AnalyticsTestResult"
+ }
+ }
+ }
+ }
+ }
+ }
+ },
+ "/api/v1/admin/analytics": {
+ "get": {
+ "tags": [
+ "admin"
+ ],
+ "summary": "Usage Metrics",
+ "description": "Daily, weekly and monthly active people, churn, retention by sign-up week and feature use over the last `days`\ndays, worked out here from the activity ledger and sign-ins; nothing is sent anywhere.",
+ "operationId": "usage_metrics",
+ "parameters": [
+ {
+ "name": "days",
+ "in": "query",
+ "required": false,
+ "schema": {
+ "type": "integer",
+ "maximum": 180,
+ "minimum": 1,
+ "default": 30,
+ "title": "Days"
+ }
+ }
+ ],
+ "responses": {
+ "200": {
+ "description": "Successful Response",
+ "content": {
+ "application/json": {
+ "schema": {
+ "$ref": "#/components/schemas/AnalyticsMetrics"
+ }
+ }
+ }
+ },
+ "422": {
+ "description": "Validation Error",
+ "content": {
+ "application/json": {
+ "schema": {
+ "$ref": "#/components/schemas/HTTPValidationError"
+ }
+ }
+ }
+ }
+ }
+ }
+ },
"/api/v1/audit": {
"get": {
"tags": [
@@ -3812,7 +3902,7 @@
"setup"
],
"summary": "Save Telemetry",
- "description": "Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector). Off unless chosen.",
+ "description": "Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector), and usage statistics\nto your own Google Analytics property when `analytics` is true. Each is off unless chosen.",
"operationId": "save_telemetry",
"requestBody": {
"content": {
@@ -24755,6 +24845,381 @@
],
"title": "ActivityTotals"
},
+ "AnalyticsChurn": {
+ "properties": {
+ "previous_active": {
+ "type": "integer",
+ "title": "Previous Active"
+ },
+ "retained": {
+ "type": "integer",
+ "title": "Retained"
+ },
+ "churned": {
+ "type": "integer",
+ "title": "Churned"
+ },
+ "rate": {
+ "anyOf": [
+ {
+ "type": "number"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Rate"
+ },
+ "returning": {
+ "type": "integer",
+ "title": "Returning"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "previous_active",
+ "retained",
+ "churned",
+ "returning"
+ ],
+ "title": "AnalyticsChurn"
+ },
+ "AnalyticsCohort": {
+ "properties": {
+ "week": {
+ "type": "string",
+ "title": "Week"
+ },
+ "users": {
+ "type": "integer",
+ "title": "Users"
+ },
+ "retention": {
+ "items": {
+ "anyOf": [
+ {
+ "type": "number"
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "type": "array",
+ "title": "Retention"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "week",
+ "users",
+ "retention"
+ ],
+ "title": "AnalyticsCohort"
+ },
+ "AnalyticsDay": {
+ "properties": {
+ "day": {
+ "type": "string",
+ "title": "Day"
+ },
+ "active_users": {
+ "type": "integer",
+ "title": "Active Users"
+ },
+ "actions": {
+ "type": "integer",
+ "title": "Actions"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "day",
+ "active_users",
+ "actions"
+ ],
+ "title": "AnalyticsDay"
+ },
+ "AnalyticsEvent": {
+ "properties": {
+ "name": {
+ "type": "string",
+ "title": "Name"
+ },
+ "description": {
+ "type": "string",
+ "title": "Description"
+ },
+ "on": {
+ "type": "boolean",
+ "title": "On"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "name",
+ "description",
+ "on"
+ ],
+ "title": "AnalyticsEvent"
+ },
+ "AnalyticsFeature": {
+ "properties": {
+ "feature": {
+ "type": "string",
+ "title": "Feature"
+ },
+ "actions": {
+ "type": "integer",
+ "title": "Actions"
+ },
+ "users": {
+ "type": "integer",
+ "title": "Users"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "feature",
+ "actions",
+ "users"
+ ],
+ "title": "AnalyticsFeature"
+ },
+ "AnalyticsMetrics": {
+ "properties": {
+ "days": {
+ "type": "integer",
+ "title": "Days"
+ },
+ "since": {
+ "type": "string",
+ "title": "Since"
+ },
+ "until": {
+ "type": "string",
+ "title": "Until"
+ },
+ "daily": {
+ "items": {
+ "$ref": "#/components/schemas/AnalyticsDay"
+ },
+ "type": "array",
+ "title": "Daily"
+ },
+ "dau": {
+ "type": "integer",
+ "title": "Dau"
+ },
+ "wau": {
+ "type": "integer",
+ "title": "Wau"
+ },
+ "mau": {
+ "type": "integer",
+ "title": "Mau"
+ },
+ "stickiness": {
+ "anyOf": [
+ {
+ "type": "number"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Stickiness"
+ },
+ "accounts": {
+ "type": "integer",
+ "title": "Accounts"
+ },
+ "active_users": {
+ "type": "integer",
+ "title": "Active Users"
+ },
+ "new_users": {
+ "type": "integer",
+ "title": "New Users"
+ },
+ "churn": {
+ "$ref": "#/components/schemas/AnalyticsChurn"
+ },
+ "retention": {
+ "items": {
+ "$ref": "#/components/schemas/AnalyticsCohort"
+ },
+ "type": "array",
+ "title": "Retention"
+ },
+ "features": {
+ "items": {
+ "$ref": "#/components/schemas/AnalyticsFeature"
+ },
+ "type": "array",
+ "title": "Features"
+ },
+ "last_active_day": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Last Active Day"
+ },
+ "days_since_active": {
+ "anyOf": [
+ {
+ "type": "integer"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Days Since Active"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "days",
+ "since",
+ "until",
+ "daily",
+ "dau",
+ "wau",
+ "mau",
+ "accounts",
+ "active_users",
+ "new_users",
+ "churn",
+ "retention",
+ "features"
+ ],
+ "title": "AnalyticsMetrics",
+ "description": "Active people, churn, retention and feature use, worked out on this machine from the activity ledger and\nsign-ins (docs/analytics.md#the-numbers)."
+ },
+ "AnalyticsStatus": {
+ "properties": {
+ "enabled": {
+ "type": "boolean",
+ "title": "Enabled"
+ },
+ "configured": {
+ "type": "boolean",
+ "title": "Configured"
+ },
+ "measurement_id": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Measurement Id"
+ },
+ "client_id": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Client Id"
+ },
+ "last_sent_at": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Last Sent At"
+ },
+ "last_sent": {
+ "items": {
+ "type": "string"
+ },
+ "type": "array",
+ "title": "Last Sent",
+ "default": []
+ },
+ "last_error": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Last Error"
+ },
+ "events": {
+ "items": {
+ "$ref": "#/components/schemas/AnalyticsEvent"
+ },
+ "type": "array",
+ "title": "Events"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "enabled",
+ "configured",
+ "events"
+ ],
+ "title": "AnalyticsStatus",
+ "description": "Whether usage statistics go to Google Analytics, and how the last send went (docs/analytics.md)."
+ },
+ "AnalyticsTestResult": {
+ "properties": {
+ "ok": {
+ "type": "boolean",
+ "title": "Ok"
+ },
+ "error": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Error"
+ },
+ "messages": {
+ "items": {
+ "type": "string"
+ },
+ "type": "array",
+ "title": "Messages",
+ "default": []
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "ok"
+ ],
+ "title": "AnalyticsTestResult"
+ },
"AnswerCheck": {
"properties": {
"claims": {
@@ -48150,6 +48615,44 @@
],
"title": "SetupAdmin"
},
+ "SetupAnalyticsView": {
+ "properties": {
+ "enabled": {
+ "type": "boolean",
+ "title": "Enabled"
+ },
+ "measurement_id": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Measurement Id"
+ },
+ "secret_set": {
+ "type": "boolean",
+ "title": "Secret Set"
+ },
+ "locked": {
+ "items": {
+ "type": "string"
+ },
+ "type": "array",
+ "title": "Locked"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "enabled",
+ "secret_set",
+ "locked"
+ ],
+ "title": "SetupAnalyticsView"
+ },
"SetupFinish": {
"properties": {
"skipped": {
@@ -48500,6 +49003,41 @@
}
],
"title": "Endpoint"
+ },
+ "analytics": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Analytics"
+ },
+ "measurement_id": {
+ "anyOf": [
+ {
+ "type": "string",
+ "maxLength": 40
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Measurement Id"
+ },
+ "api_secret": {
+ "anyOf": [
+ {
+ "type": "string",
+ "maxLength": 200
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Api Secret"
}
},
"additionalProperties": false,
@@ -48529,13 +49067,17 @@
},
"type": "array",
"title": "Locked"
+ },
+ "analytics": {
+ "$ref": "#/components/schemas/SetupAnalyticsView"
}
},
"additionalProperties": true,
"type": "object",
"required": [
"enabled",
- "locked"
+ "locked",
+ "analytics"
],
"title": "SetupTelemetryView"
},
From e0e76ee433321d951b64c92b8b64078232f718e8 Mon Sep 17 00:00:00 2001
From: Adeel Ahmad
Date: Sat, 10 Oct 2026 20:09:05 +0000
Subject: [PATCH 2/5] feat(analytics): Settings page with usage numbers and the
wizard's Google Analytics opt-in
Co-Authored-By: Claude Opus 5.5 (1M context)
Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH
---
.../__tests__/setupWizard.test.tsx | 47 +++++-
.../components/settings/bodies.tsx | 90 ++++++++++++
nextjs-frontend/components/settings/model.ts | 70 +++++++++
.../components/settings/settings-shell.tsx | 2 +
.../components/settings/usage-analytics.tsx | 139 ++++++++++++++++++
.../components/setup/setup-wizard.tsx | 110 +++++++++++++-
6 files changed, 449 insertions(+), 9 deletions(-)
create mode 100644 nextjs-frontend/components/settings/usage-analytics.tsx
diff --git a/nextjs-frontend/__tests__/setupWizard.test.tsx b/nextjs-frontend/__tests__/setupWizard.test.tsx
index 36b3bb9ff..4cc1d8b54 100644
--- a/nextjs-frontend/__tests__/setupWizard.test.tsx
+++ b/nextjs-frontend/__tests__/setupWizard.test.tsx
@@ -1,5 +1,5 @@
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
-import { fireEvent, render, screen, waitFor } from "@testing-library/react";
+import { fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import "@testing-library/jest-dom";
import type { ReactNode } from "react";
@@ -148,7 +148,13 @@ describe("SetupWizard", () => {
await waitFor(() =>
expect(Setup.finish).toHaveBeenCalledWith(expect.objectContaining({ body: { skipped: false } })),
);
- expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({ enabled: false, endpoint: null });
+ expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({
+ enabled: false,
+ endpoint: null,
+ analytics: false,
+ measurement_id: null,
+ api_secret: null,
+ });
await waitFor(() => expect(replace).toHaveBeenCalledWith("/"));
});
@@ -159,7 +165,9 @@ describe("SetupWizard", () => {
m(Setup.finish).mockImplementation(() => ok({ ok: true }));
wrap();
fireEvent.click(await screen.findByRole("button", { name: /Telemetry/ }));
- fireEvent.click(await screen.findByRole("radio", { name: "On" }));
+ fireEvent.click(
+ within(await screen.findByRole("radiogroup", { name: "Send telemetry" })).getByRole("radio", { name: "On" }),
+ );
const save = screen.getByRole("button", { name: "Save and finish" });
expect(save).toBeDisabled(); // on needs somewhere to send to
fireEvent.change(screen.getByLabelText("OTLP endpoint"), { target: { value: "localhost:4318" } });
@@ -170,7 +178,38 @@ describe("SetupWizard", () => {
expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({ enabled: false, endpoint: "http://localhost:4318" });
fireEvent.click(save);
await waitFor(() => expect(Setup.finish).toHaveBeenCalled());
- expect(m(Setup.saveTelemetry).mock.calls[1][0].body).toEqual({ enabled: true, endpoint: "http://localhost:4318" });
+ expect(m(Setup.saveTelemetry).mock.calls[1][0].body).toEqual({
+ enabled: true,
+ endpoint: "http://localhost:4318",
+ analytics: false,
+ measurement_id: null,
+ api_secret: null,
+ });
+ });
+
+ it("opts in to Google Analytics with a measurement id and secret", async () => {
+ m(Setup.getSetup).mockImplementation(() => ok(VIEW({ namespace: { existing: ["media"], locked: false } })));
+ m(Setup.saveTelemetry).mockImplementation(() => ok({ ok: true }));
+ m(Setup.finish).mockImplementation(() => ok({ ok: true }));
+ wrap();
+ fireEvent.click(await screen.findByRole("button", { name: /Telemetry/ }));
+ const group = await screen.findByRole("radiogroup", { name: "Share usage statistics with Google Analytics" });
+ fireEvent.click(within(group).getByRole("radio", { name: "On" }));
+ const save = screen.getByRole("button", { name: "Save and finish" });
+ expect(save).toBeDisabled(); // on needs an id and a secret
+ fireEvent.change(screen.getByLabelText("Measurement id"), { target: { value: "UA-1" } });
+ fireEvent.change(screen.getByLabelText("Measurement Protocol API secret"), { target: { value: "s3cret" } });
+ expect(save).toBeDisabled();
+ fireEvent.change(screen.getByLabelText("Measurement id"), { target: { value: "G-TEST1234" } });
+ fireEvent.click(save);
+ await waitFor(() => expect(Setup.finish).toHaveBeenCalled());
+ expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({
+ enabled: false,
+ endpoint: null,
+ analytics: true,
+ measurement_id: "G-TEST1234",
+ api_secret: "s3cret",
+ });
});
it("lets apps sign in for a set time, or turns it off", async () => {
diff --git a/nextjs-frontend/components/settings/bodies.tsx b/nextjs-frontend/components/settings/bodies.tsx
index d7d16bf49..6e21e96ce 100644
--- a/nextjs-frontend/components/settings/bodies.tsx
+++ b/nextjs-frontend/components/settings/bodies.tsx
@@ -14,6 +14,7 @@ import { RIGHTS } from "@/components/iiif/rights";
import { hubText } from "@/components/sensors/sensor-model";
import { SecretSetting, SettingField, ZoneBar, type FieldState } from "@/components/settings/fields";
import { SignInProviders } from "@/components/settings/sign-in-providers";
+import { UsageNumbers } from "@/components/settings/usage-analytics";
import { AddConnection } from "@/components/storage/add-connection";
import { NOT_STORAGE, fileStoreTry } from "@/components/storage/file-store";
import {
@@ -379,6 +380,8 @@ export function SectionBody({ ctx }: { ctx: BodyCtx }) {
return ;
case "telemetry":
return ;
+ case "analytics":
+ return ;
case "fedora":
return ;
case "sensors":
@@ -610,6 +613,93 @@ function TelemetryBody({ ctx }: { ctx: BodyCtx }) {
);
}
+function AnalyticsBody({ ctx }: { ctx: BodyCtx }) {
+ const client = useApiClient();
+ const a = ctx.view.analytics;
+ const saved = a?.values ?? {};
+ const secret = (saved.api_secret ?? {}) as { set?: boolean };
+ const apiSecret = ctx.state("analytics.api_secret");
+ const locked = a?.locked ?? [];
+ const status = useQuery({
+ queryKey: ["analytics-status", a?.updated_at ?? null],
+ queryFn: () => data(Admin.analyticsStatus({ client })),
+ refetchInterval: 60_000,
+ });
+ const test = useMutation({ mutationFn: () => data(Admin.testAnalytics({ client })) });
+ const s = status.data;
+ const on = Boolean(saved.enabled);
+ const id = (saved.measurement_id as string | null | undefined) ?? null;
+ return (
+ <>
+
+ {on && s?.configured ? (
+
+ {s.last_error
+ ? s.last_error
+ : s.last_sent_at
+ ? `Last sent ${new Date(s.last_sent_at).toLocaleString()}: ${(s.last_sent ?? []).join(", ")}`
+ : "Nothing sent yet; the first events go with the next daily check."}
+
+ ) : on ? (
+
+ Set the measurement id and the API secret below; until then nothing is sent.
+
+ ) : (
+
+ Turn it on here, in the setup wizard or with LENS_ANALYTICS=on in .env.
+ Events go only to the GA4 property whose id and secret you set.
+
+ )}
+
+
+
+
+
+ apiSecret.onChange(x)}
+ />
+
+
+
+ What is sent, once a day: a random id for this install{s?.client_id ? ` (${s.client_id})` : ""}, the Lens
+ version and platform, and counts. Never names, emails, account ids, titles, file names, namespaces, prompts or
+ any text. In GA, the Retention report and a cohort exploration on lens_daily_active give churn across your
+ installs.
+
+
+ } onClick={() => test.mutate()} disabled={test.isPending || !id}>
+ {test.isPending ? "Sending…" : "Send a test event"}
+
+
+ {ctx.dirty
+ ? "Uses the saved settings, not your unsaved changes"
+ : "GA’s validation server checks one lens_test event without recording it"}
+
+
+ {test.data &&
+ (test.data.ok ? (
+
+ ) : (
+
+ {test.data.error ?? (test.data.messages ?? []).join(" · ")}
+
+ ))}
+ {test.isError && {test.error.message}}
+ >
+ );
+}
+
function EncryptionBody({ ctx }: { ctx: BodyCtx }) {
const client = useApiClient();
const progress = useQuery({
diff --git a/nextjs-frontend/components/settings/model.ts b/nextjs-frontend/components/settings/model.ts
index 6c59496a1..7c17f56b0 100644
--- a/nextjs-frontend/components/settings/model.ts
+++ b/nextjs-frontend/components/settings/model.ts
@@ -74,6 +74,7 @@ export type SectionId =
| "mail"
| "bridge"
| "telemetry"
+ | "analytics"
| "fedora"
| "sensors"
| "uploads"
@@ -239,6 +240,13 @@ export const SECTIONS: SectionSpec[] = [
description:
"Off unless you turn it on. Traces and metrics from the server and its workers, sent only to an OpenTelemetry collector you run or choose; nothing goes anywhere else.",
},
+ {
+ id: "analytics",
+ label: "Usage analytics",
+ backend: ["analytics"],
+ description:
+ "Active people, churn, retention and feature use, worked out on this server. Optionally, counted events for this install sent to your own Google Analytics property; off unless you turn it on.",
+ },
{
id: "fedora",
label: "Fedora repository",
@@ -1866,6 +1874,59 @@ export const FIELDS: FieldSpec[] = [
max: 20,
hint: "A failed send waits 30 seconds, then four times longer each time, up to 6 hours",
},
+ // Usage analytics (Google Analytics opt-in)
+ {
+ section: "analytics",
+ key: "enabled",
+ label: "Send usage events to Google Analytics",
+ kind: "switch",
+ hint: "Off by default; needs the measurement id and API secret below",
+ },
+ {
+ section: "analytics",
+ key: "measurement_id",
+ label: "Measurement id",
+ kind: "text",
+ nullable: true,
+ mono: true,
+ placeholder: "G-ABC123XYZ",
+ hint: "Your GA4 property’s web data stream id",
+ },
+ { section: "analytics", key: "api_secret", label: "Measurement Protocol API secret", kind: "secret" },
+ {
+ section: "analytics",
+ key: "region",
+ label: "Collection",
+ kind: "select",
+ options: [
+ { value: "global", label: "Global" },
+ { value: "eu", label: "EU only (region1.google-analytics.com)" },
+ ],
+ },
+ {
+ section: "analytics",
+ key: "events",
+ label: "Events sent",
+ kind: "checks",
+ options: [
+ { value: "lens_install", label: "Install", hint: "Analytics first runs here" },
+ { value: "lens_setup_done", label: "Setup done", hint: "The setup wizard was finished" },
+ { value: "lens_daily_active", label: "Daily active", hint: "People and actions the day before" },
+ { value: "lens_feature_used", label: "Feature used", hint: "Each feature used the day before, with counts" },
+ { value: "lens_upgrade", label: "Upgrade", hint: "From and to which version" },
+ { value: "lens_inactive", label: "Inactive", hint: "Nobody used Lens for the days below" },
+ { value: "lens_return", label: "Return", hint: "Used again after going inactive" },
+ ],
+ },
+ {
+ section: "analytics",
+ key: "inactive_days",
+ label: "Inactive after (days)",
+ kind: "int",
+ min: 1,
+ max: 365,
+ hint: "Days without anyone using Lens before it counts as churned",
+ },
// Telemetry (opt-in)
{
section: "telemetry",
@@ -2397,6 +2458,9 @@ export function crossErrors(values: Record): Record = {
mail: Mail,
bridge: MessagesSquare,
telemetry: Activity,
+ analytics: BarChart3,
fedora: Archive,
sensors: RadioTower,
uploads: Upload,
diff --git a/nextjs-frontend/components/settings/usage-analytics.tsx b/nextjs-frontend/components/settings/usage-analytics.tsx
new file mode 100644
index 000000000..deda8866c
--- /dev/null
+++ b/nextjs-frontend/components/settings/usage-analytics.tsx
@@ -0,0 +1,139 @@
+"use client";
+
+import { useQuery } from "@tanstack/react-query";
+
+import { Admin } from "@/app/openapi-client";
+import type { AnalyticsMetrics } from "@/app/openapi-client/types.gen";
+import { Banner } from "@/components/ui/banner";
+import { Skeleton } from "@/components/ui/states";
+import { data, useApiClient } from "@/lib/api/browser";
+
+function pct(v: number | null | undefined) {
+ return v == null ? "—" : `${Math.round(v * 1000) / 10}%`;
+}
+
+function Tile({ label, value, hint }: { label: string; value: string | number; hint?: string }) {
+ return (
+
+ {label}
+ {value}
+ {hint && {hint}}
+
+ );
+}
+
+/** Daily active people as a row of bars, one per day. */
+function DailyBars({ days }: { days: AnalyticsMetrics["daily"] }) {
+ const top = Math.max(1, ...days.map((d) => d.active_users));
+ return (
+
+ Active people each day
+
+ {days.map((d) => (
+
+ ))}
+
+
+ {days[0]?.day}
+ {days[days.length - 1]?.day}
+
+
+ );
+}
+
+/** Active people, churn, retention and feature use, worked out on this server (docs/analytics.md). */
+export function UsageNumbers({ days = 30 }: { days?: number }) {
+ const client = useApiClient();
+ const q = useQuery({
+ queryKey: ["usage-analytics", days],
+ queryFn: () => data(Admin.usageMetrics({ client, query: { days } })),
+ });
+ if (q.isError) return {q.error.message};
+ const m = q.data;
+ if (!m) return ;
+ const c = m.churn;
+ const weeks = Math.max(0, ...m.retention.map((r) => r.retention.length));
+ return (
+
+
+ The last {m.days} days ({m.since} to {m.until}), from the activity ledger and sign-ins on this server. Nothing
+ here is sent anywhere.
+
+ Sent once a day: a random id for this install, the version and counts. Never names, emails, titles, files or
+ text.
+
+ >
+ )}
{tested &&
(tested.ok ? (
The collector took a test span in {tested.ms} ms.
@@ -985,10 +1087,8 @@ function TelemetryStep({
From f2f3ce1cfab85f451631dd9231c2d4adbedb6ca1 Mon Sep 17 00:00:00 2001
From: Adeel Ahmad
Date: Sat, 10 Oct 2026 20:09:51 +0000
Subject: [PATCH 3/5] docs(analytics): the Settings page is built
Co-Authored-By: Claude Opus 5.5 (1M context)
Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH
---
docs/analytics.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/analytics.md b/docs/analytics.md
index 18b5714a1..715ac79a9 100644
--- a/docs/analytics.md
+++ b/docs/analytics.md
@@ -4,7 +4,7 @@ Lens works out how it is being used (active people, churn, retention, which feat
and can also send a few counted events to a **Google Analytics 4** property you own, to see the same across every
install you run.
-Status: **built** (backend and API). The Settings page and the wizard toggle are **in progress**.
+Status: **built**: the numbers, Settings → Usage analytics, the wizard toggle and the Google Analytics events.
## The numbers
@@ -85,6 +85,6 @@ telemetry does.
## Refine later
-- A Settings page with charts of the numbers, and the wizard toggle (in progress).
+- Choose the period in Settings (it shows the last 30 days).
- Per-namespace numbers.
- Optional per-person pseudonymous ids in GA, for multi-person installs.
From bdf598f96873d2355c5abe63870622969db8d1a3 Mon Sep 17 00:00:00 2001
From: Adeel Ahmad
Date: Sat, 10 Oct 2026 20:15:48 +0000
Subject: [PATCH 4/5] feat(analytics): built-in Analytics page, with each day's
events kept locally
The daily events are counted and kept on the server whether or not Google
Analytics is on; GA is an optional export of them. A new Analytics page
(admins) shows active people, churn, retention, feature use and the event
log for 7 to 180 days.
Co-Authored-By: Claude Opus 5.5 (1M context)
Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH
---
docs/analytics.md | 20 ++-
fastapi_backend/app/api/v1/routes/admin.py | 8 ++
fastapi_backend/app/domain/analytics.py | 99 ++++++++++----
fastapi_backend/app/schemas/admin.py | 12 ++
fastapi_backend/tests/api/test_analytics.py | 26 +++-
.../__tests__/analyticsPage.test.tsx | 92 +++++++++++++
nextjs-frontend/__tests__/navigation.test.ts | 2 +
nextjs-frontend/app/(app)/analytics/page.tsx | 10 ++
nextjs-frontend/app/openapi-client/index.ts | 6 +
nextjs-frontend/app/openapi-client/sdk.gen.ts | 18 +++
.../app/openapi-client/types.gen.ts | 63 +++++++++
.../components/analytics/analytics-page.tsx | 126 ++++++++++++++++++
.../usage-numbers.tsx} | 0
.../components/app-shell/nav-config.ts | 2 +
.../components/settings/bodies.tsx | 10 +-
nextjs-frontend/openapi.json | 92 +++++++++++++
16 files changed, 549 insertions(+), 37 deletions(-)
create mode 100644 nextjs-frontend/__tests__/analyticsPage.test.tsx
create mode 100644 nextjs-frontend/app/(app)/analytics/page.tsx
create mode 100644 nextjs-frontend/components/analytics/analytics-page.tsx
rename nextjs-frontend/components/{settings/usage-analytics.tsx => analytics/usage-numbers.tsx} (100%)
diff --git a/docs/analytics.md b/docs/analytics.md
index 715ac79a9..0fd2f0a59 100644
--- a/docs/analytics.md
+++ b/docs/analytics.md
@@ -1,10 +1,12 @@
# Usage analytics
-Lens works out how it is being used (active people, churn, retention, which features get used) on your own machine,
-and can also send a few counted events to a **Google Analytics 4** property you own, to see the same across every
-install you run.
+Lens has built-in analytics: the **Analytics** page (admins) shows how Lens is being used, with active people, churn,
+retention, which features get used and the events it counts each day. All of it is worked out and kept on your own
+machine. Optionally, the same daily events also go to a **Google Analytics 4** property you own, to see them across
+every install you run.
-Status: **built**: the numbers, Settings → Usage analytics, the wizard toggle and the Google Analytics events.
+Status: **built**: the Analytics page, the daily events, the Google Analytics export, its settings and the wizard
+toggle.
## The numbers
@@ -26,6 +28,13 @@ collected for them and nothing leaves the machine.
A person is active on a day when they changed something through the API or signed in. With `activity.reads` on,
reads count too. The ledger keeps `activity.keep_days` (365), which is as far back as the numbers can go.
+## Events
+
+Once a day, a worker (or `lens watch`) counts the day before and keeps the result in the `usage_event` table, with
+Google Analytics on or off. The Analytics page lists them, and so does `GET /api/v1/admin/analytics/events`. Each one
+shows whether it was sent to Google Analytics, is waiting to be sent, or is kept here only. Events are kept for
+`activity.keep_days`.
+
## Google Analytics
**Off unless you turn it on.** It needs a GA4 property of your own: its **measurement id** (`G-…`) and a
@@ -37,7 +46,8 @@ reads count too. The ledger keeps `activity.keep_days` (365), which is as far ba
- **.env**: `LENS_ANALYTICS=on`, `LENS_GA_MEASUREMENT_ID=G-…`, `LENS_GA_API_SECRET=…`. What .env sets wins over
Settings, which show those fields locked.
-Turned on, one process (a worker, or `lens watch`) sends once a day, for the install as a whole:
+Turned on, the daily events are also sent, for the install as a whole. A send that fails is tried again 10 minutes
+later. Events kept while Google Analytics was off are never sent later. These are the events:
| Event | When | Parameters |
| --- | --- | --- |
diff --git a/fastapi_backend/app/api/v1/routes/admin.py b/fastapi_backend/app/api/v1/routes/admin.py
index 2f203aa4c..aecdc9d6c 100644
--- a/fastapi_backend/app/api/v1/routes/admin.py
+++ b/fastapi_backend/app/api/v1/routes/admin.py
@@ -59,6 +59,7 @@
TelemetryStatus,
TelemetryTestResult,
TunnelStatus,
+ UsageEvent,
)
from app.schemas.auth import AccountToken
from app.schemas.common import Ok
@@ -303,6 +304,13 @@ def usage_metrics(user: AdminReader, db: Db, days: int = Query(30, ge=1, le=180)
return AnalyticsMetrics.model_validate(analytics.metrics(db, days))
+@router.get("/admin/analytics/events")
+def usage_events(user: AdminReader, db: Db, limit: int = Query(200, ge=1, le=1000)) -> list[UsageEvent]:
+ """The events Lens counted each day (install, daily active, feature used, upgrade, inactive...), newest first, and
+ whether each went to Google Analytics. Kept here whether or not Google Analytics is on."""
+ return [UsageEvent.model_validate(r) for r in analytics.recent_events(db, limit)]
+
+
@router.get("/audit")
def list_audit(user: AdminReader, db: Db, limit: int = Query(200, ge=1, le=1000)) -> list[AuditEntry]:
"""Who changed what, newest first."""
diff --git a/fastapi_backend/app/domain/analytics.py b/fastapi_backend/app/domain/analytics.py
index 66afd15d1..8f0bff801 100644
--- a/fastapi_backend/app/domain/analytics.py
+++ b/fastapi_backend/app/domain/analytics.py
@@ -5,10 +5,12 @@
(activity.py) and sign-ins in the audit log. Nothing new is collected for them and nothing leaves the machine: daily,
weekly and monthly active people, stickiness, churn and retention by sign-up week, and which features get used.
+**The events** in EVENTS are worked out once a day and kept here (`usage_event`), so the Analytics page shows them
+with Google Analytics off.
+
**Google Analytics** is off by default (analytics.enabled), and needs a GA4 measurement id and a Measurement Protocol
API secret, set in Settings → Usage analytics, the setup wizard, or .env (LENS_ANALYTICS, LENS_GA_MEASUREMENT_ID,
-LENS_GA_API_SECRET). Turned on, one process sends once a day the events in EVENTS that analytics.events names, for this
-install as a whole: the install's random id (made here, never a person, host or address), the Lens version and
+LENS_GA_API_SECRET). Turned on, the kept events that analytics.events names also go to GA, for this install as a whole: the install's random id (made here, never a person, host or address), the Lens version and
platform, and counts. Never a name, email, title, file name, namespace, prompt or text. Churn across installs is then
GA's retention report over lens_daily_active; lens_inactive marks an install nobody has used for
analytics.inactive_days, which is how a self-hosted install that quietly stopped shows up.
@@ -31,6 +33,7 @@
ROW = R("app_service", "analytics")
HOSTS = {"global": "www.google-analytics.com", "eu": "region1.google-analytics.com"} # tests point it at a fake
BATCH = 25 # GA4 takes up to 25 events a request
+RETRY_SECONDS = 600
MEASUREMENT_RX = re.compile(r"^G-[A-Z0-9]{4,20}$")
# The events Lens can send, and what each carries. Names are prefixed so none clashes with GA's reserved ones.
@@ -271,35 +274,59 @@ def _claim(db, day):
def tick(db, cfg, now=None):
- """Once a day, while analytics is on and set up: send what happened since the last time (docs/analytics.md)."""
- if not configured(cfg):
- return []
+ """Once a day: work out the day before's events and keep them here (`usage_event`, docs/analytics.md#events),
+ whether or not Google Analytics is on; with it on and set up, send them too. Every tick sends what is still unsent
+ (a send that failed is tried again a few minutes later). Returns the names of the events made this time."""
from . import __version__
now = now or dt.datetime.now(dt.timezone.utc)
today = now.date()
- if not _claim(db, today.isoformat()):
- return []
+ made = []
+ if _claim(db, today.isoformat()):
+ events, marks = _today(db, cfg, today, __version__)
+ a = _section(cfg)
+ wanted = configured(cfg) and set(a.get("events") or EVENTS)
+ yesterday = (today - dt.timedelta(days=1)).isoformat()
+ at = store.now()
+ for n, (name, params) in enumerate(events):
+ db.q(
+ "CREATE usage_event CONTENT $d",
+ d={
+ "at": at,
+ "n": n,
+ "day": yesterday,
+ "name": name,
+ "params": params,
+ "sent": False if wanted and name in wanted else None,
+ },
+ )
+ _put(db, **marks)
+ keep = int(((cfg or {}).get("activity") or {}).get("keep_days") or 365)
+ db.q("DELETE usage_event WHERE day < $d", d=(today - dt.timedelta(days=keep)).isoformat())
+ made = [n for n, _ in events]
+ if configured(cfg):
+ flush_unsent(db, cfg, now)
+ return made
+
+
+def _today(db, cfg, today, version):
+ """The events for the day before `today`, and what to remember so none is made twice."""
st, events = _state(db), []
if not st.get("installed"):
- events.append(("lens_install", {"version": __version__, "platform": _platform()}))
+ events.append(("lens_install", {"version": version, "platform": _platform()}))
wiz = db.one("SELECT done_at, skipped FROM $r", r=R("setup", "wizard")) or {}
if wiz.get("done_at") and not st.get("setup_done"):
events.append(("lens_setup_done", {"skipped": 1 if wiz.get("skipped") else 0}))
- if st.get("version") and st["version"] != __version__:
- events.append(("lens_upgrade", {"from_version": st["version"], "to_version": __version__}))
+ if st.get("version") and st["version"] != version:
+ events.append(("lens_upgrade", {"from_version": st["version"], "to_version": version}))
yesterday = today - dt.timedelta(days=1)
seen = _days(db, yesterday)
users = {w for (w, d) in seen if d == yesterday.isoformat()}
inactive_days = int(_section(cfg).get("inactive_days") or 30)
- marks = {"installed": True, "setup_done": bool(wiz.get("done_at")), "version": __version__}
+ marks = {"installed": True, "setup_done": bool(wiz.get("done_at")), "version": version}
if users:
- events.append(
- (
- "lens_daily_active",
- {"active_users": len(users), "actions": sum(n for (_, d), n in seen.items() if d == yesterday.isoformat())},
- )
- )
+ actions = sum(n for (_, d), n in seen.items() if d == yesterday.isoformat())
+ events.append(("lens_daily_active", {"active_users": len(users), "actions": actions}))
start = dt.datetime.combine(yesterday, dt.time(), dt.timezone.utc)
for f in _features(db, start, start + dt.timedelta(days=1))[:20]:
events.append(("lens_feature_used", f))
@@ -314,13 +341,37 @@ def tick(db, cfg, now=None):
if since and (today - since).days >= inactive_days:
events.append(("lens_inactive", {"days": (today - since).days}))
marks["inactive_since"] = since.isoformat()
- ok, error, _ = send(db, cfg, events, day=yesterday)
- if ok:
- _put(db, **marks)
- else:
- log.info("analytics: %s", error)
- db.q("UPDATE $r SET day = NONE", r=ROW) # try again on the next tick
- return [n for n, _ in events]
+ return events, marks
+
+
+def flush_unsent(db, cfg, now=None):
+ """Send the kept events still waiting for Google Analytics, a day at a time; after a failure, wait RETRY_SECONDS."""
+ now = now or dt.datetime.now(dt.timezone.utc)
+ st = _state(db)
+ if st.get("last_error_at") and st.get("last_error"):
+ try:
+ if (now - dt.datetime.fromisoformat(st["last_error_at"])).total_seconds() < RETRY_SECONDS:
+ return False
+ except ValueError:
+ pass
+ rows = db.rows("SELECT record::id(id) AS id, at, n, day, name, params FROM usage_event WHERE sent = false ORDER BY at, n LIMIT 500")
+ for day in sorted({r["day"] for r in rows}):
+ batch = [r for r in rows if r["day"] == day]
+ ok, error, _ = send(db, cfg, [(r["name"], r["params"] or {}) for r in batch], day=dt.date.fromisoformat(day))
+ if not ok:
+ log.info("analytics: %s", error)
+ _put(db, last_error_at=now.isoformat())
+ return False
+ db.q("UPDATE usage_event SET sent = true, sent_at = $t WHERE record::id(id) IN $ids", t=store.now(), ids=[r["id"] for r in batch])
+ return True
+
+
+def recent_events(db, limit=200):
+ """The kept events, newest first: what Lens counted each day, and whether each went to Google Analytics."""
+ return db.rows(
+ "SELECT at, n, day, name, params, sent FROM usage_event ORDER BY at DESC, n LIMIT $n",
+ n=max(1, min(int(limit), 1000)),
+ )
def test(db, cfg):
diff --git a/fastapi_backend/app/schemas/admin.py b/fastapi_backend/app/schemas/admin.py
index bba0f0bc6..82f4f10e5 100644
--- a/fastapi_backend/app/schemas/admin.py
+++ b/fastapi_backend/app/schemas/admin.py
@@ -308,6 +308,18 @@ class AnalyticsMetrics(ResponseModel):
days_since_active: int | None = None
+class UsageEvent(ResponseModel):
+ """An event Lens counted for a day (docs/analytics.md#events)."""
+
+ at: str
+ # the day it counts
+ day: str
+ name: str
+ params: dict[str, Any] = {}
+ # true: sent to Google Analytics; false: waiting to be sent; null: kept here only (GA off, or the event left out)
+ sent: bool | None = None
+
+
class FileStoreTry(RequestModel):
"""Where to try keeping files, before saving it (Settings → Storage); nothing given tries what is saved."""
diff --git a/fastapi_backend/tests/api/test_analytics.py b/fastapi_backend/tests/api/test_analytics.py
index b3e50d3c3..e09264bcb 100644
--- a/fastapi_backend/tests/api/test_analytics.py
+++ b/fastapi_backend/tests/api/test_analytics.py
@@ -91,15 +91,20 @@ def d(n):
assert week and week[-1]["retention"][0] == 1.0 # 3 was active in its sign-up week
-def test_off_by_default_sends_nothing(db, cfg, ga):
+def test_events_kept_locally_with_ga_off(db, cfg, ga):
act(db, 1, (NOW.date() - dt.timedelta(days=1)).isoformat())
assert cfg["analytics"]["enabled"] is False
- assert analytics.tick(db, cfg, NOW) == []
- routines.run_due(db, cfg, log=None, now=NOW)
+ assert analytics.tick(db, cfg, NOW) == ["lens_install", "lens_daily_active", "lens_feature_used"]
+ routines.run_due(db, cfg, log=None, now=NOW) # once a day
assert ga == []
- # on without a secret isn't set up either
+ kept = analytics.recent_events(db)
+ assert sorted(e["name"] for e in kept) == ["lens_daily_active", "lens_feature_used", "lens_install"]
+ assert all(e["sent"] is None and e["day"] == "2026-10-09" for e in kept)
+ # on without a secret isn't set up either, and turning it on later doesn't send what was kept while off
cfg["analytics"].update(enabled=True, measurement_id="G-TEST1234")
- assert analytics.tick(db, cfg, NOW) == []
+ analytics.tick(db, cfg, NOW + dt.timedelta(days=1))
+ on(cfg)
+ analytics.tick(db, cfg, NOW + dt.timedelta(days=1, hours=1))
assert ga == []
@@ -151,8 +156,14 @@ def test_events_can_be_left_out_and_failures_retry(db, cfg, ga):
assert names(ga) == ["lens_install"]
assert "500" in analytics.status(db, cfg)["last_error"]
ga.state["status"] = 204
- analytics.tick(db, cfg, NOW + dt.timedelta(minutes=5)) # the failed day is tried again
+ analytics.tick(db, cfg, NOW + dt.timedelta(minutes=5)) # waits a while after a failure
+ assert names(ga) == ["lens_install"]
+ analytics.tick(db, cfg, NOW + dt.timedelta(minutes=11)) # then tries again
assert names(ga) == ["lens_install", "lens_install"]
+ analytics.tick(db, cfg, NOW + dt.timedelta(minutes=20)) # and sends each once
+ assert len(ga) == 2
+ kept = {e["name"]: e["sent"] for e in analytics.recent_events(db)}
+ assert kept == {"lens_install": True, "lens_daily_active": None, "lens_feature_used": None}
st = analytics.status(db, cfg)
assert st["last_error"] is None and st["last_sent"] == ["lens_install"]
@@ -180,6 +191,9 @@ def put(body):
s = client.get("/api/v1/settings/analytics/status", headers=h).json()
assert s["enabled"] is False and s["configured"] is False and len(s["events"]) == len(analytics.EVENTS)
assert all(e["on"] for e in s["events"])
+ analytics.tick(db, client.app.state.settings.current())
+ ev = client.get("/api/v1/admin/analytics/events", headers=h).json()
+ assert [e["name"] for e in ev] == ["lens_install"] and ev[0]["sent"] is None
m = client.get("/api/v1/admin/analytics?days=7", headers=h).json()
assert m["days"] == 7 and m["accounts"] == 1 and m["dau"] == 1 # the sign-in above
make_user(db, "ed@x.io", "ed password 12")
diff --git a/nextjs-frontend/__tests__/analyticsPage.test.tsx b/nextjs-frontend/__tests__/analyticsPage.test.tsx
new file mode 100644
index 000000000..57aa3fb93
--- /dev/null
+++ b/nextjs-frontend/__tests__/analyticsPage.test.tsx
@@ -0,0 +1,92 @@
+import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
+import { fireEvent, render, screen, waitFor } from "@testing-library/react";
+import "@testing-library/jest-dom";
+import type { ReactNode } from "react";
+
+import { Admin } from "@/app/openapi-client";
+import { AnalyticsPage } from "@/components/analytics/analytics-page";
+import { TooltipProvider } from "@/components/ui/tooltip";
+
+jest.mock("@/app/openapi-client", () => ({
+ Admin: { usageMetrics: jest.fn(), usageEvents: jest.fn(), analyticsStatus: jest.fn() },
+}));
+jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) }));
+const archive = { admin: true };
+jest.mock("@/lib/hooks/session", () => ({ useArchive: () => archive }));
+
+const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } });
+const m = (f: unknown) => f as jest.Mock;
+
+const METRICS = (days: number) => ({
+ days,
+ since: "2026-10-04",
+ until: "2026-10-10",
+ daily: [
+ { day: "2026-10-09", active_users: 1, actions: 3 },
+ { day: "2026-10-10", active_users: 2, actions: 2 },
+ ],
+ dau: 2,
+ wau: 2,
+ mau: 3,
+ stickiness: 0.1429,
+ accounts: 3,
+ active_users: 2,
+ new_users: 1,
+ churn: { previous_active: 2, retained: 1, churned: 1, rate: 0.5, returning: 1 },
+ retention: [{ week: "2026-10-05", users: 1, retention: [1] }],
+ features: [{ feature: "chats", actions: 2, users: 1 }],
+ last_active_day: "2026-10-10",
+ days_since_active: 0,
+});
+
+function wrap(ui: ReactNode) {
+ const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
+ return render(
+
+ {ui}
+ ,
+ );
+}
+
+describe("AnalyticsPage", () => {
+ beforeEach(() => {
+ archive.admin = true;
+ m(Admin.usageMetrics).mockImplementation(({ query }: { query: { days: number } }) => ok(METRICS(query.days)));
+ m(Admin.usageEvents).mockImplementation(() =>
+ ok([
+ {
+ at: "2026-10-10T00:05:00+00:00",
+ day: "2026-10-09",
+ name: "lens_daily_active",
+ params: { active_users: 1 },
+ sent: null,
+ },
+ ]),
+ );
+ m(Admin.analyticsStatus).mockImplementation(() => ok({ enabled: false, configured: false, events: [] }));
+ });
+
+ it("shows churn, retention and the kept events with Google Analytics off", async () => {
+ wrap();
+ expect(await screen.findByText("50%")).toBeInTheDocument(); // churn rate
+ expect(screen.getByText("1 of 2 didn’t come back")).toBeInTheDocument();
+ expect(screen.getByText("Retention by sign-up week")).toBeInTheDocument();
+ expect(await screen.findByText("Daily active")).toBeInTheDocument();
+ expect(screen.getByText("kept here only")).toBeInTheDocument();
+ expect(screen.getByRole("link", { name: "Settings → Usage analytics" })).toHaveAttribute(
+ "href",
+ "/settings/analytics",
+ );
+ fireEvent.click(screen.getByRole("radio", { name: "90 days" }));
+ await waitFor(() =>
+ expect(Admin.usageMetrics).toHaveBeenCalledWith(expect.objectContaining({ query: { days: 90 } })),
+ );
+ });
+
+ it("is for admins", () => {
+ archive.admin = false;
+ wrap();
+ expect(screen.getByText("Analytics are for admins")).toBeInTheDocument();
+ expect(Admin.usageMetrics).not.toHaveBeenCalled();
+ });
+});
diff --git a/nextjs-frontend/__tests__/navigation.test.ts b/nextjs-frontend/__tests__/navigation.test.ts
index 7f00825e1..02e8f0a66 100644
--- a/nextjs-frontend/__tests__/navigation.test.ts
+++ b/nextjs-frontend/__tests__/navigation.test.ts
@@ -8,6 +8,8 @@ describe("navigation", () => {
expect(labels(true)).toEqual(expect.arrayContaining(["Settings", "Admin"]));
expect(labels(false)).toContain("Settings");
expect(labels(false)).not.toContain("Admin");
+ expect(labels(true)).toContain("Analytics");
+ expect(labels(false)).not.toContain("Analytics");
expect(labels(true)).not.toContain("Speakers");
// Admins see every source as a sensor; members keep Sources, the watched folders feeding their namespaces.
expect(labels(true)).toContain("Sensors");
diff --git a/nextjs-frontend/app/(app)/analytics/page.tsx b/nextjs-frontend/app/(app)/analytics/page.tsx
new file mode 100644
index 000000000..38b65eb2f
--- /dev/null
+++ b/nextjs-frontend/app/(app)/analytics/page.tsx
@@ -0,0 +1,10 @@
+import type { Metadata } from "next";
+
+import { AnalyticsPage } from "@/components/analytics/analytics-page";
+
+export const metadata: Metadata = { title: "Analytics" };
+
+/** Built-in usage analytics: active people, churn, retention, features and the daily events (docs/analytics.md). */
+export default function Page() {
+ return ;
+}
diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts
index a50ac936a..7165c1257 100644
--- a/nextjs-frontend/app/openapi-client/index.ts
+++ b/nextjs-frontend/app/openapi-client/index.ts
@@ -2876,6 +2876,12 @@ export type {
UploadLimitsResponse,
UploadLimitsResponses,
UploadStart,
+ UsageEvent,
+ UsageEventsData,
+ UsageEventsError,
+ UsageEventsErrors,
+ UsageEventsResponse,
+ UsageEventsResponses,
UsageMetricsData,
UsageMetricsError,
UsageMetricsErrors,
diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts
index bacf4113b..65fb5b0ed 100644
--- a/nextjs-frontend/app/openapi-client/sdk.gen.ts
+++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts
@@ -1409,6 +1409,9 @@ import type {
UpdateWorkflowResponses,
UploadLimitsData,
UploadLimitsResponses,
+ UsageEventsData,
+ UsageEventsErrors,
+ UsageEventsResponses,
UsageMetricsData,
UsageMetricsErrors,
UsageMetricsResponses,
@@ -2671,6 +2674,21 @@ export class Admin {
});
}
+ /**
+ * Usage Events
+ *
+ * The events Lens counted each day (install, daily active, feature used, upgrade, inactive...), newest first, and
+ * whether each went to Google Analytics. Kept here whether or not Google Analytics is on.
+ */
+ public static usageEvents(
+ options?: Options,
+ ): RequestResult {
+ return (options?.client ?? client).get({
+ url: "/api/v1/admin/analytics/events",
+ ...options,
+ });
+ }
+
/**
* List Audit
*
diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts
index 34e1d4dc1..cbcde94d4 100644
--- a/nextjs-frontend/app/openapi-client/types.gen.ts
+++ b/nextjs-frontend/app/openapi-client/types.gen.ts
@@ -16495,6 +16495,37 @@ export type UploadStart = {
hold?: boolean;
};
+/**
+ * UsageEvent
+ *
+ * An event Lens counted for a day (docs/analytics.md#events).
+ */
+export type UsageEvent = {
+ /**
+ * At
+ */
+ at: string;
+ /**
+ * Day
+ */
+ day: string;
+ /**
+ * Name
+ */
+ name: string;
+ /**
+ * Params
+ */
+ params?: {
+ [key: string]: unknown;
+ };
+ /**
+ * Sent
+ */
+ sent?: boolean | null;
+ [key: string]: unknown;
+};
+
/**
* UserCreate
*/
@@ -19525,6 +19556,38 @@ export type UsageMetricsResponses = {
export type UsageMetricsResponse = UsageMetricsResponses[keyof UsageMetricsResponses];
+export type UsageEventsData = {
+ body?: never;
+ path?: never;
+ query?: {
+ /**
+ * Limit
+ */
+ limit?: number;
+ };
+ url: "/api/v1/admin/analytics/events";
+};
+
+export type UsageEventsErrors = {
+ /**
+ * Validation Error
+ */
+ 422: HttpValidationError;
+};
+
+export type UsageEventsError = UsageEventsErrors[keyof UsageEventsErrors];
+
+export type UsageEventsResponses = {
+ /**
+ * Response Admin-Usage Events
+ *
+ * Successful Response
+ */
+ 200: Array;
+};
+
+export type UsageEventsResponse = UsageEventsResponses[keyof UsageEventsResponses];
+
export type ListAuditData = {
body?: never;
path?: never;
diff --git a/nextjs-frontend/components/analytics/analytics-page.tsx b/nextjs-frontend/components/analytics/analytics-page.tsx
new file mode 100644
index 000000000..7883f27ab
--- /dev/null
+++ b/nextjs-frontend/components/analytics/analytics-page.tsx
@@ -0,0 +1,126 @@
+"use client";
+
+import { useQuery } from "@tanstack/react-query";
+import Link from "next/link";
+import { useState } from "react";
+
+import { Admin } from "@/app/openapi-client";
+import { UsageNumbers } from "@/components/analytics/usage-numbers";
+import { SegmentedChoice } from "@/components/settings/controls";
+import { Banner } from "@/components/ui/banner";
+import { EmptyState, PageHeader, Skeleton } from "@/components/ui/states";
+import { data, useApiClient } from "@/lib/api/browser";
+import { useArchive } from "@/lib/hooks/session";
+
+const PERIODS = [
+ { value: "7", label: "7 days" },
+ { value: "30", label: "30 days" },
+ { value: "90", label: "90 days" },
+ { value: "180", label: "180 days" },
+];
+
+const EVENT_LABEL: Record = {
+ lens_install: "Install",
+ lens_setup_done: "Setup done",
+ lens_daily_active: "Daily active",
+ lens_feature_used: "Feature used",
+ lens_upgrade: "Upgrade",
+ lens_inactive: "Inactive",
+ lens_return: "Return",
+};
+
+function params(p: Record | undefined) {
+ return Object.entries(p ?? {})
+ .map(([k, v]) => `${k.replaceAll("_", " ")} ${String(v)}`)
+ .join(" · ");
+}
+
+/** The events Lens counted each day, and whether each went to Google Analytics. */
+function EventLog() {
+ const client = useApiClient();
+ const q = useQuery({
+ queryKey: ["usage-events"],
+ queryFn: () => data(Admin.usageEvents({ client, query: { limit: 200 } })),
+ });
+ if (q.isError) return {q.error.message};
+ if (!q.data) return ;
+ if (!q.data.length)
+ return (
+
+ No events yet. Lens counts the day before once a day, so the first ones show up tomorrow.
+
+ );
+}
+
+/** Built-in analytics (docs/analytics.md): works with Google Analytics off; GA is an optional export. */
+export function AnalyticsPage() {
+ const { admin } = useArchive();
+ const client = useApiClient();
+ const [days, setDays] = useState("30");
+ const ga = useQuery({
+ queryKey: ["analytics-status"],
+ queryFn: () => data(Admin.analyticsStatus({ client })),
+ enabled: admin,
+ });
+ if (!admin)
+ return (
+
+ Ask an admin of this archive.
+
+ );
+ const s = ga.data;
+ return (
+
+ }
+ className="mb-0"
+ />
+
+
+
Events
+
+ {s?.configured ? (
+ <>Each day’s events are also sent to Google Analytics ({s.measurement_id}).>
+ ) : (
+ <>
+ Kept on this server only. To also send them to your own Google Analytics property, turn it on in{" "}
+
+ Settings → Usage analytics
+
+ .
+ >
+ )}
+
+
+
+
+ );
+}
diff --git a/nextjs-frontend/components/settings/usage-analytics.tsx b/nextjs-frontend/components/analytics/usage-numbers.tsx
similarity index 100%
rename from nextjs-frontend/components/settings/usage-analytics.tsx
rename to nextjs-frontend/components/analytics/usage-numbers.tsx
diff --git a/nextjs-frontend/components/app-shell/nav-config.ts b/nextjs-frontend/components/app-shell/nav-config.ts
index cae4a1cbb..782b54350 100644
--- a/nextjs-frontend/components/app-shell/nav-config.ts
+++ b/nextjs-frontend/components/app-shell/nav-config.ts
@@ -1,6 +1,7 @@
import {
Blocks,
CalendarClock,
+ ChartLine,
ChartNoAxesColumn,
HardDriveDownload,
House,
@@ -61,6 +62,7 @@ export const NAV: NavItem[] = [
// Admins see every source as a sensor; members see the watched folders feeding their namespaces.
{ label: "Sensors", href: "/sensors", icon: RadioTower, adminOnly: true },
{ label: "Sources", href: "/sources", icon: HardDriveDownload, membersOnly: true },
+ { label: "Analytics", href: "/analytics", icon: ChartLine, adminOnly: true },
// Everyone has Settings: members find Speakers there, admins the archive's settings too.
{ label: "Settings", href: "/settings", icon: Settings, countKey: "reviews" },
{ label: "Admin", href: "/admin", icon: Shield, adminOnly: true },
diff --git a/nextjs-frontend/components/settings/bodies.tsx b/nextjs-frontend/components/settings/bodies.tsx
index 6e21e96ce..ab3ca2842 100644
--- a/nextjs-frontend/components/settings/bodies.tsx
+++ b/nextjs-frontend/components/settings/bodies.tsx
@@ -14,7 +14,6 @@ import { RIGHTS } from "@/components/iiif/rights";
import { hubText } from "@/components/sensors/sensor-model";
import { SecretSetting, SettingField, ZoneBar, type FieldState } from "@/components/settings/fields";
import { SignInProviders } from "@/components/settings/sign-in-providers";
-import { UsageNumbers } from "@/components/settings/usage-analytics";
import { AddConnection } from "@/components/storage/add-connection";
import { NOT_STORAGE, fileStoreTry } from "@/components/storage/file-store";
import {
@@ -631,7 +630,14 @@ function AnalyticsBody({ ctx }: { ctx: BodyCtx }) {
const id = (saved.measurement_id as string | null | undefined) ?? null;
return (
<>
-
+
+ Active people, churn, retention and feature use are on the{" "}
+
+ Analytics
+ {" "}
+ page, worked out on this server whether or not Google Analytics is on. This section only chooses whether the
+ daily events also go to your Google Analytics property.
+
{on && s?.configured ? (
{s.last_error
diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json
index fcb2cf488..ecd8860c7 100644
--- a/nextjs-frontend/openapi.json
+++ b/nextjs-frontend/openapi.json
@@ -2822,6 +2822,56 @@
}
}
},
+ "/api/v1/admin/analytics/events": {
+ "get": {
+ "tags": [
+ "admin"
+ ],
+ "summary": "Usage Events",
+ "description": "The events Lens counted each day (install, daily active, feature used, upgrade, inactive...), newest first, and\nwhether each went to Google Analytics. Kept here whether or not Google Analytics is on.",
+ "operationId": "usage_events",
+ "parameters": [
+ {
+ "name": "limit",
+ "in": "query",
+ "required": false,
+ "schema": {
+ "type": "integer",
+ "maximum": 1000,
+ "minimum": 1,
+ "default": 200,
+ "title": "Limit"
+ }
+ }
+ ],
+ "responses": {
+ "200": {
+ "description": "Successful Response",
+ "content": {
+ "application/json": {
+ "schema": {
+ "type": "array",
+ "items": {
+ "$ref": "#/components/schemas/UsageEvent"
+ },
+ "title": "Response Admin-Usage Events"
+ }
+ }
+ }
+ },
+ "422": {
+ "description": "Validation Error",
+ "content": {
+ "application/json": {
+ "schema": {
+ "$ref": "#/components/schemas/HTTPValidationError"
+ }
+ }
+ }
+ }
+ }
+ }
+ },
"/api/v1/audit": {
"get": {
"tags": [
@@ -52908,6 +52958,48 @@
],
"title": "UploadStart"
},
+ "UsageEvent": {
+ "properties": {
+ "at": {
+ "type": "string",
+ "title": "At"
+ },
+ "day": {
+ "type": "string",
+ "title": "Day"
+ },
+ "name": {
+ "type": "string",
+ "title": "Name"
+ },
+ "params": {
+ "additionalProperties": true,
+ "type": "object",
+ "title": "Params",
+ "default": {}
+ },
+ "sent": {
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "title": "Sent"
+ }
+ },
+ "additionalProperties": true,
+ "type": "object",
+ "required": [
+ "at",
+ "day",
+ "name"
+ ],
+ "title": "UsageEvent",
+ "description": "An event Lens counted for a day (docs/analytics.md#events)."
+ },
"UserCreate": {
"properties": {
"email": {
From 6f23496226f961867be15717549195ce32a8a167 Mon Sep 17 00:00:00 2001
From: Adeel Ahmad
Date: Sat, 10 Oct 2026 21:14:30 +0000
Subject: [PATCH 5/5] docs(analytics): threat model and changelog entries for
usage analytics
Co-Authored-By: Claude Opus 5.5 (1M context)
Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH
---
CHANGELOG.md | 6 ++++++
threat-model.md | 5 +++--
threat-model.yaml | 2 +-
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 10ea98810..a70255d03 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,12 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
## Unreleased
+- **Analytics.** A new Analytics page (admins) shows active people per day, week and month, stickiness, churn rate,
+ retention by sign-up week and feature use, worked out on the server from the activity ledger and sign-ins. Each day
+ Lens also keeps a few counted events (install, daily active, feature used, upgrade, inactive, return). They can
+ optionally go to your own Google Analytics 4 property too; that is off by default and is set in Settings → Usage
+ analytics or the setup wizard ([Usage analytics](docs/analytics.md)).
+
- **Prometheus metrics.** Lens can serve `/metrics` for Prometheus and Grafana: jobs by status and pipeline, imports,
model calls, tokens, cost and failures by model, budgets, workers and disk. Off by default; turned on in Settings →
Telemetry (or `LENS_PROMETHEUS=on`), it answers only a scraper with the scrape token, and follows `LENS_BIND`. A
diff --git a/threat-model.md b/threat-model.md
index 36f617348..58c8a55b9 100644
--- a/threat-model.md
+++ b/threat-model.md
@@ -199,7 +199,8 @@ flowchart LR
and workers may run at once; throttle counters and the setup code live in each process's memory *(inferred, Q5)*.
**Host side effects.** Lens makes no outbound connection except to configured sources, model servers, mail,
-notification and telemetry targets; telemetry is off by default and there is no update check *(maintainer, 2026-10)*.
+notification and telemetry targets; telemetry and the Google Analytics export (`analytics.py`: daily counts for the
+install, no personal content) are off by default and there is no update check *(maintainer, 2026-10)*.
| Behaviour | Observed | Tag |
| --- | --- | --- |
@@ -239,7 +240,7 @@ machine requires HTTPS, and a changed DB credential when SurrealDB runs as its o
| `documents.web_networks`, `LENS_WEB_NETWORKS`; `notifications.networks` | empty | widen the outbound guard to listed private networks | Yes (operator and admin input) | *(documented, docs/configuration.md "loopback and cloud metadata addresses stay out")* |
| Chromium sandbox | off as root or without user namespaces | the `netguard` proxy is the only barrier | Yes | *(documented, docs/configuration.md "runs with its sandbox where it can")* |
| `sensors.enabled`, `sensors.mqtt_anonymous` | off (`bind` `0.0.0.0`); off | opens MQTT and syslog listeners; anonymous MQTT publish | Yes; anonymous is the admin's choice | *(documented, docs/sensors.md "Sensors are off until an admin turns them on")* |
-| `tunnel.mode`, `local_llm.enabled`, `telemetry` | off | outbound tunnel (Cloudflare terminates TLS, Q30); local model server; OTLP export | Yes | *(documented, docs/remote-access.md "It's off until an admin turns it on")* |
+| `tunnel.mode`, `local_llm.enabled`, `telemetry`, `analytics.enabled` | off | outbound tunnel (Cloudflare terminates TLS, Q30); local model server; OTLP export; GA4 Measurement Protocol export of daily counts | Yes | *(documented, docs/remote-access.md "It's off until an admin turns it on")* |
| `components.auto` | on | unpinned-hash pip installs at runtime | Yes | *(maintainer, 2026-10)* |
| `encryption.files` | on for new archives | file encryption at rest | Yes | *(documented, docs/encryption.md "A new archive turns this on at its first start")* |
| `ai.extensions`, `ai.disabled_tools` | on, none | turn assistant extensions or named tools off | Yes | *(documented, docs/assistant.md "Extending the assistant")* |
diff --git a/threat-model.yaml b/threat-model.yaml
index 76ecb5aa5..71810ef05 100644
--- a/threat-model.yaml
+++ b/threat-model.yaml
@@ -2,7 +2,7 @@
# Authority order: prose > yaml > json. Regenerate whenever the prose changes.
schema: threat-model-sidecar/v2
project: lens
-prose_version: "threat-model.md@sha256:4525de4bf73248fe9511a6ad69c55ff7c021b11a9a3e46a285bf25fe5208ea99"
+prose_version: "threat-model.md@sha256:067cef03edea230dfa3c5fec7f288dffea4e0c063c899c02b22ea41f3d8d2389"
model_status: under-review
triage_policy: strict
confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8}