From 11180387a34208a9f625512c59e8ea8eecfc2982 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 10 Oct 2026 20:02:29 +0000 Subject: [PATCH 1/5] feat(analytics): usage metrics with churn and retention, and opt-in Google Analytics events Lens works out daily, weekly and monthly active people, stickiness, churn, retention by sign-up week and feature use from the activity ledger and sign-ins (GET /api/v1/admin/analytics). Google Analytics 4 is off by default: with a measurement id and API secret (Settings, the setup wizard's telemetry step, or .env) one process sends a documented set of counted events once a day for the install, never personal content. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH --- docker-compose.prod.yml | 4 + docs/analytics.md | 90 +++ docs/configuration.md | 3 +- fastapi_backend/.env.example | 5 + fastapi_backend/app/api/v1/routes/admin.py | 24 + fastapi_backend/app/api/v1/routes/setup.py | 15 +- fastapi_backend/app/domain/analytics.py | 348 +++++++++++ fastapi_backend/app/domain/ops_tools.py | 5 +- fastapi_backend/app/domain/routines.py | 6 + fastapi_backend/app/domain/settings.py | 33 ++ fastapi_backend/app/domain/setup.py | 27 + fastapi_backend/app/domain/store.py | 18 + fastapi_backend/app/schemas/admin.py | 85 +++ fastapi_backend/app/schemas/setup.py | 16 + fastapi_backend/tests/api/test_analytics.py | 204 +++++++ fastapi_backend/tests/api/test_telemetry.py | 21 +- mkdocs.yml | 1 + .../__tests__/setupWizard.test.tsx | 7 +- nextjs-frontend/app/openapi-client/index.ts | 20 + nextjs-frontend/app/openapi-client/sdk.gen.ts | 53 +- .../app/openapi-client/types.gen.ts | 332 +++++++++++ nextjs-frontend/openapi.json | 546 +++++++++++++++++- 22 files changed, 1851 insertions(+), 12 deletions(-) create mode 100644 docs/analytics.md create mode 100644 fastapi_backend/app/domain/analytics.py create mode 100644 fastapi_backend/tests/api/test_analytics.py diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml index c5b41f574..379b3422b 100644 --- a/docker-compose.prod.yml +++ b/docker-compose.prod.yml @@ -58,6 +58,10 @@ x-backend: &backend LENS_TELEMETRY: ${LENS_TELEMETRY:-} LENS_TELEMETRY_ENDPOINT: ${LENS_TELEMETRY_ENDPOINT:-} LENS_TELEMETRY_HEADERS: ${LENS_TELEMETRY_HEADERS:-} + # opt-in usage events to your own Google Analytics 4 property (docs/analytics.md): unset, off unless turned on in Settings + LENS_ANALYTICS: ${LENS_ANALYTICS:-} + LENS_GA_MEASUREMENT_ID: ${LENS_GA_MEASUREMENT_ID:-} + LENS_GA_API_SECRET: ${LENS_GA_API_SECRET:-} # the volume shared with the matterbridge service (its config, written from Settings → Chat rooms) LENS_MATTERBRIDGE_DIR: /matterbridge # upgrading from the web app (docs/updates.md): the folder install.sh's upgrade agent watches, and the install folder diff --git a/docs/analytics.md b/docs/analytics.md new file mode 100644 index 000000000..18b5714a1 --- /dev/null +++ b/docs/analytics.md @@ -0,0 +1,90 @@ +# Usage analytics + +Lens works out how it is being used (active people, churn, retention, which features get used) on your own machine, +and can also send a few counted events to a **Google Analytics 4** property you own, to see the same across every +install you run. + +Status: **built** (backend and API). The Settings page and the wizard toggle are **in progress**. + +## The numbers + +`GET /api/v1/admin/analytics?days=30` (admins) works these out from what Lens already keeps: the +[activity ledger](activity.md)'s rows for API calls a person made, and sign-ins in the audit log. Nothing new is +collected for them and nothing leaves the machine. + +| Number | What it is | +| --- | --- | +| `daily` | each day of the period: how many people were active, and how many actions they took | +| `dau`, `wau`, `mau` | people active today, in the last 7 days, in the last 30 days | +| `stickiness` | average daily active people ÷ monthly active people (how much of the month a regular uses Lens) | +| `active_users`, `new_users`, `accounts` | people active in the period, accounts made in it, accounts not disabled | +| `churn` | of the people active in the period before this one (`previous_active`), how many came back (`retained`) and how many didn't (`churned`); `rate` = churned ÷ previous_active; `returning` were active now but not before | +| `retention` | for each of the last 8 weeks, the accounts made that week and the share of them active that week and each week after | +| `features` | each feature (from the API routes: recordings, chats, graph, routines…) with its actions and people | +| `last_active_day`, `days_since_active` | when anyone last used this install | + +A person is active on a day when they changed something through the API or signed in. With `activity.reads` on, +reads count too. The ledger keeps `activity.keep_days` (365), which is as far back as the numbers can go. + +## Google Analytics + +**Off unless you turn it on.** It needs a GA4 property of your own: its **measurement id** (`G-…`) and a +**Measurement Protocol API secret** (in GA: Admin → Data streams → your stream → Measurement Protocol API secrets). + +- **Settings → Usage analytics**: turn it on and set the id and secret. **Send a test event** sends one `lens_test` + event to GA's validation server, which checks it without recording it. +- **The setup wizard**, on its telemetry step. +- **.env**: `LENS_ANALYTICS=on`, `LENS_GA_MEASUREMENT_ID=G-…`, `LENS_GA_API_SECRET=…`. What .env sets wins over + Settings, which show those fields locked. + +Turned on, one process (a worker, or `lens watch`) sends once a day, for the install as a whole: + +| Event | When | Parameters | +| --- | --- | --- | +| `lens_install` | analytics first runs on this install | `version`, `platform` | +| `lens_setup_done` | the setup wizard was finished or skipped | `skipped` (0 or 1) | +| `lens_daily_active` | at least one person used Lens the day before | `active_users`, `actions` | +| `lens_feature_used` | each feature used the day before (up to 20) | `feature`, `actions`, `users` | +| `lens_upgrade` | Lens runs a new version | `from_version`, `to_version` | +| `lens_inactive` | nobody has used Lens for `analytics.inactive_days` (30) | `days` | +| `lens_return` | someone used Lens again after `lens_inactive` | `days_away` | + +Every event also carries `session_id` (the day) and `engagement_time_msec`, so GA counts the install as an active +user that day, and the user properties `lens_version` and `lens_platform` (like `linux-x86_64`). + +**What is sent:** the install's id (a random UUID made on first use, not derived from the machine), the version, the +platform and the counts above. **Never** a name, email, account id, title, file name, namespace, prompt or any text. +Like any web request, it reaches Google from your server's address (GA4 doesn't keep IP addresses). `analytics.events` chooses which events are sent; leave one out and it never is. + +### Churn and retention in GA + +Each install is one GA user. In GA: + +- **Reports → Retention** shows installs coming back day after day and week after week, from `lens_daily_active`. +- **Churn rate** for a period is installs active in the period before that sent no `lens_daily_active` in it. An + Exploration (Cohort exploration, with `lens_install` as the inclusion event and `lens_daily_active` as the return + event) gives it by install week. `lens_inactive` counts installs that went quiet: a self-hosted install that was + removed can't say so, so `inactive_days` without use is the uninstall-like signal. +- **Feature use**: register `feature` as an event-scoped custom dimension (Admin → Custom definitions), and `actions` + and `active_users` as custom metrics, then break `lens_feature_used` down by `feature`. + +## Settings + +| Setting | Default | What it does | +| --- | --- | --- | +| `analytics.enabled` | `false` | send to Google Analytics | +| `analytics.measurement_id` | none | the GA4 measurement id, like `G-ABC123XYZ` | +| `analytics.api_secret` | none | the Measurement Protocol API secret (encrypted at rest, never shown again) | +| `analytics.region` | `global` | `eu` sends to `region1.google-analytics.com`, which keeps collection in the EU | +| `analytics.events` | all | which of the events above are sent | +| `analytics.inactive_days` | `30` | days without anyone using Lens before `lens_inactive` | + +`GET /api/v1/settings/analytics/status` shows whether it is on, this install's id, which events are on, and how the +last send went. Changing these settings through the setup assistant always waits for an admin's approval, as +telemetry does. + +## Refine later + +- A Settings page with charts of the numbers, and the wizard toggle (in progress). +- Per-namespace numbers. +- Optional per-person pseudonymous ids in GA, for multi-person installs. diff --git a/docs/configuration.md b/docs/configuration.md index 550663fb8..d94fcbba4 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -34,6 +34,7 @@ with AES-GCM and are write-only: the API reports whether one is set, never its v | `LENS_SETUP_SETTINGS` | | settings to start with, as JSON `{"section": {"key": value}}` (or that JSON base64url-encoded): saved once at startup as if made in the app, so unlike the variables above they stay editable there. The [web installer](get-started.md#the-web-installer) writes its answers here | | `LENS_SETUP_WIZARD` | | `off`: never show the setup wizard | | `LENS_TELEMETRY` / `LENS_TELEMETRY_ENDPOINT` / `LENS_TELEMETRY_HEADERS` | off | opt-in OpenTelemetry traces and metrics, sent only to this OTLP/HTTP endpoint ([Telemetry](telemetry.md)); `LENS_TELEMETRY=off` keeps it off whatever Settings say | +| `LENS_ANALYTICS` / `LENS_GA_MEASUREMENT_ID` / `LENS_GA_API_SECRET` | off | opt-in usage events sent to your own Google Analytics 4 property ([Usage analytics](analytics.md)) | | `FRONTEND_URL` | `http://localhost:3000` | the web app's address as people use it: links in emails, and where apps send people to sign in ([OAuth](authentication.md#oauth)) | | `CORS_ORIGINS` | `["http://localhost:3000"]` | origins allowed to call the API from a browser | | `OPENAPI_URL` | `/openapi.json` | `""` disables `/docs` and the schema | @@ -57,7 +58,7 @@ A fresh install (no accounts when the API first starts) walks its first admin th 5. **Apps and AI**: whether apps and AI assistants (Claude, ChatGPT, Cursor and other MCP clients) may sign people in with their Lens account ([OAuth](authentication.md#oauth)), on unless turned off, how long their tokens last, and the MCP server's address to add to an assistant. -6. **Telemetry**: off unless chosen ([Telemetry](telemetry.md)). +6. **Telemetry**: off unless chosen ([Telemetry](telemetry.md)), and Google Analytics usage events, off unless chosen ([Usage analytics](analytics.md)). Every step can be skipped, and the whole wizard too; all of it stays in Settings. Values from the environment win and show locked. Installs that already had accounts never see the wizard. The API side is `GET /api/v1/setup`, diff --git a/fastapi_backend/.env.example b/fastapi_backend/.env.example index fb9b937fd..dcd4d5a0c 100644 --- a/fastapi_backend/.env.example +++ b/fastapi_backend/.env.example @@ -57,5 +57,10 @@ CORS_ORIGINS=["http://localhost:3000"] # LENS_TELEMETRY_ENDPOINT=http://localhost:4318 # your OpenTelemetry collector's OTLP/HTTP address # LENS_TELEMETRY_HEADERS= # optional: key=value pairs, comma-separated, URL-encoded +# --- Usage analytics (docs/analytics.md): counted events sent to your own Google Analytics 4 property, off unless on --- +# LENS_ANALYTICS=on +# LENS_GA_MEASUREMENT_ID=G-ABC123XYZ +# LENS_GA_API_SECRET= # a Measurement Protocol API secret from the GA data stream + # Uncomment to disable /docs and /openapi.json # OPENAPI_URL="" diff --git a/fastapi_backend/app/api/v1/routes/admin.py b/fastapi_backend/app/api/v1/routes/admin.py index 5249bfb4a..2f203aa4c 100644 --- a/fastapi_backend/app/api/v1/routes/admin.py +++ b/fastapi_backend/app/api/v1/routes/admin.py @@ -14,6 +14,7 @@ from app.api.deps import AdminReader, AdminWriter, Cfg, Db, domain_errors from app.core.middleware import host_name from app.domain import ( + analytics, anytopdf, auth, blobs, @@ -33,6 +34,9 @@ tunnel, ) from app.schemas.admin import ( + AnalyticsMetrics, + AnalyticsStatus, + AnalyticsTestResult, AuditEntry, BridgeStatus, BridgeTestResult, @@ -279,6 +283,26 @@ def test_telemetry(user: AdminWriter, cfg: Cfg) -> TelemetryTestResult: return TelemetryTestResult(ok=ok, error=error, ms=ms) +@router.get("/settings/analytics/status") +def analytics_status(user: AdminReader, cfg: Cfg, db: Db) -> AnalyticsStatus: + """Whether usage statistics go to Google Analytics, which events, and how the last send went. Off by default.""" + return AnalyticsStatus.model_validate(analytics.status(db, cfg)) + + +@router.post("/settings/analytics/test") +def test_analytics(user: AdminWriter, cfg: Cfg, db: Db) -> AnalyticsTestResult: + """Send one lens_test event to Google Analytics' validation server (checked there, not recorded), on or off.""" + ok, error, messages = analytics.test(db, cfg) + return AnalyticsTestResult(ok=ok, error=error, messages=messages) + + +@router.get("/admin/analytics") +def usage_metrics(user: AdminReader, db: Db, days: int = Query(30, ge=1, le=180)) -> AnalyticsMetrics: + """Daily, weekly and monthly active people, churn, retention by sign-up week and feature use over the last `days` + days, worked out here from the activity ledger and sign-ins; nothing is sent anywhere.""" + return AnalyticsMetrics.model_validate(analytics.metrics(db, days)) + + @router.get("/audit") def list_audit(user: AdminReader, db: Db, limit: int = Query(200, ge=1, le=1000)) -> list[AuditEntry]: """Who changed what, newest first.""" diff --git a/fastapi_backend/app/api/v1/routes/setup.py b/fastapi_backend/app/api/v1/routes/setup.py index cf8f085ba..62d4682f8 100644 --- a/fastapi_backend/app/api/v1/routes/setup.py +++ b/fastapi_backend/app/api/v1/routes/setup.py @@ -86,14 +86,23 @@ def save_storage(body: SetupStorage, user: AdminWriter, request: Request, db: Db @router.put("/telemetry") def save_telemetry(body: SetupTelemetry, user: AdminWriter, request: Request, db: Db) -> SetupSaved: - """Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector). Off unless chosen.""" + """Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector), and usage statistics + to your own Google Analytics property when `analytics` is true. Each is off unless chosen.""" try: - saved = setup.save_telemetry(db, request.app.state.archive.base, body.enabled, body.endpoint, user.email) + base = request.app.state.archive.base + saved = setup.save_telemetry(db, base, body.enabled, body.endpoint, user.email) + shared = ( + setup.save_analytics(db, base, body.analytics, body.measurement_id, body.api_secret, user.email) + if body.analytics is not None + else [] + ) except ValueError as e: raise HTTPException(400, str(e)) from None if saved: auth.audit(db, user.as_audit(), "settings.save", "telemetry", saved) - return SetupSaved(saved=saved) + if shared: + auth.audit(db, user.as_audit(), "settings.save", "analytics", shared) + return SetupSaved(saved=saved + [f"analytics.{k}" for k in shared]) @router.put("/oauth") diff --git a/fastapi_backend/app/domain/analytics.py b/fastapi_backend/app/domain/analytics.py new file mode 100644 index 000000000..66afd15d1 --- /dev/null +++ b/fastapi_backend/app/domain/analytics.py @@ -0,0 +1,348 @@ +"""Usage analytics (docs/analytics.md): who uses Lens and how often, worked out on this machine, and optionally a +handful of counted events sent to a Google Analytics 4 property the admin owns. + +**The numbers** (`metrics`) come from what Lens already keeps: the activity ledger's rows for API calls a person made +(activity.py) and sign-ins in the audit log. Nothing new is collected for them and nothing leaves the machine: daily, +weekly and monthly active people, stickiness, churn and retention by sign-up week, and which features get used. + +**Google Analytics** is off by default (analytics.enabled), and needs a GA4 measurement id and a Measurement Protocol +API secret, set in Settings → Usage analytics, the setup wizard, or .env (LENS_ANALYTICS, LENS_GA_MEASUREMENT_ID, +LENS_GA_API_SECRET). Turned on, one process sends once a day the events in EVENTS that analytics.events names, for this +install as a whole: the install's random id (made here, never a person, host or address), the Lens version and +platform, and counts. Never a name, email, title, file name, namespace, prompt or text. Churn across installs is then +GA's retention report over lens_daily_active; lens_inactive marks an install nobody has used for +analytics.inactive_days, which is how a self-hosted install that quietly stopped shows up. +""" + +from __future__ import annotations + +import datetime as dt +import logging +import platform +import re +import uuid + +import httpx + +from . import store + +log = logging.getLogger("lens") +R = store.R +ROW = R("app_service", "analytics") +HOSTS = {"global": "www.google-analytics.com", "eu": "region1.google-analytics.com"} # tests point it at a fake +BATCH = 25 # GA4 takes up to 25 events a request +MEASUREMENT_RX = re.compile(r"^G-[A-Z0-9]{4,20}$") + +# The events Lens can send, and what each carries. Names are prefixed so none clashes with GA's reserved ones. +EVENTS = { + "lens_install": "Analytics first ran on this install. version, platform.", + "lens_setup_done": "The setup wizard was finished (or skipped). skipped (0 or 1).", + "lens_daily_active": "At least one person used Lens on the day before. active_users, actions.", + "lens_feature_used": "A feature was used on the day before, one event per feature. feature, actions, users.", + "lens_upgrade": "Lens runs a new version. from_version, to_version.", + "lens_inactive": "Nobody has used Lens for analytics.inactive_days (the uninstall-like signal). days.", + "lens_return": "Someone used Lens again after lens_inactive. days_away.", +} +FEATURE_RX = re.compile(r"^[a-z][a-z0-9_-]{0,39}$") +SCOPES = ("namespaces", "spaces") # /namespaces/{ns}/recordings counts as recordings + + +# ---------- what people did, from the activity ledger and sign-ins ---------- +def feature(action): + """The feature an activity row's action names: `POST /api/v1/namespaces/{ns}/recordings` is recordings. Actions are + route templates (activity.py), so this is code's wording, never what a person typed.""" + _, _, path = (action or "").partition(" ") + parts = [p for p in path.split("/") if p][2:] # past api/v1 + i = 0 + while i < len(parts) and parts[i] in SCOPES and i + 1 < len(parts) and parts[i + 1].startswith("{"): + i += 2 + name = parts[i] if i < len(parts) else (parts[0] if parts else "") + return name if FEATURE_RX.match(name) else "other" + + +def _day(at): + return str(at or "")[:10] + + +def _days(db, since): + """{(account, day): actions} for every person since `since` (a date): API calls that changed something, and + sign-ins (one each).""" + out = {} + for r in db.rows( + "SELECT actor, string::slice(at, 0, 10) AS day, count() AS n FROM activity " + "WHERE kind = 'in' AND actor != NONE AND at >= $s GROUP BY actor, day", + s=since.isoformat(), + ): + out[(str(r["actor"]), r["day"])] = out.get((str(r["actor"]), r["day"]), 0) + int(r["n"]) + for r in db.rows( + "SELECT account, string::slice(at, 0, 10) AS day, count() AS n FROM audit_log " + "WHERE action = 'login' AND account != NONE AND at >= $s GROUP BY account, day", + s=since.isoformat(), + ): + out[(str(r["account"]), r["day"])] = out.get((str(r["account"]), r["day"]), 0) + int(r["n"]) + return out + + +def _features(db, since, until=None): + """[{feature, actions, users}] since `since` (and before `until`), most used first.""" + rows = db.rows( + "SELECT action, actor, count() AS n FROM activity WHERE kind = 'in' AND actor != NONE AND at >= $s" + + (" AND at < $u" if until else "") + + " GROUP BY action, actor", + s=since.isoformat(), + u=until.isoformat() if until else None, + ) + acc = {} + for r in rows: + f = acc.setdefault(feature(r["action"]), {"actions": 0, "users": set()}) + f["actions"] += int(r["n"]) + f["users"].add(str(r["actor"])) + return sorted( + ({"feature": k, "actions": v["actions"], "users": len(v["users"])} for k, v in acc.items()), + key=lambda f: (-f["actions"], f["feature"]), + ) + + +def _rate(part, whole): + return round(part / whole, 4) if whole else None + + +def metrics(db, days=30, now=None, weeks=8): + """Active people, churn, retention and feature use over the last `days` days, worked out from the activity ledger + and sign-ins (docs/analytics.md#the-numbers). Churn: of the people active in the `days` before this period, the + share not active in it.""" + now = now or dt.datetime.now(dt.timezone.utc) + today = now.date() + start = today - dt.timedelta(days=days - 1) # the period is the last `days` days, today included + prev = start - dt.timedelta(days=days) + cohort_start = today - dt.timedelta(days=today.weekday()) - dt.timedelta(weeks=weeks - 1) + since = min(prev, cohort_start) + seen = _days(db, since) + by_day, people = {}, {} + for (who, day), n in seen.items(): + d = by_day.setdefault(day, {"users": set(), "actions": 0}) + d["users"].add(who) + d["actions"] += n + people.setdefault(who, set()).add(day) + + def active(a, b): + """The people active from day a to day b, both included.""" + lo, hi = a.isoformat(), b.isoformat() + return {w for w, ds in people.items() if any(lo <= d <= hi for d in ds)} + + daily = [] + for i in range(days): + d = (start + dt.timedelta(days=i)).isoformat() + got = by_day.get(d) or {"users": (), "actions": 0} + daily.append({"day": d, "active_users": len(got["users"]), "actions": got["actions"]}) + current, before = active(start, today), active(prev, start - dt.timedelta(days=1)) + mau = len(active(today - dt.timedelta(days=29), today)) + avg_dau = sum(x["active_users"] for x in daily) / days + accounts = db.rows("SELECT record::id(id) AS id, created_at, disabled FROM account") + churned = before - current + + cohorts = [] + for w in range(weeks): + wk = cohort_start + dt.timedelta(weeks=w) + members = {str(a["id"]) for a in accounts if wk.isoformat() <= _day(a.get("created_at")) < (wk + dt.timedelta(weeks=1)).isoformat()} + if not members: + continue + shares = [] + for k in range(weeks - w): + a = wk + dt.timedelta(weeks=k) + back = active(a, min(a + dt.timedelta(days=6), today)) & members + shares.append(_rate(len(back), len(members))) + cohorts.append({"week": wk.isoformat(), "users": len(members), "retention": shares}) + + last = max((d for ds in people.values() for d in ds), default=None) + return { + "days": days, + "since": start.isoformat(), + "until": today.isoformat(), + "daily": daily, + "dau": len(by_day.get(today.isoformat(), {}).get("users", ())), + "wau": len(active(today - dt.timedelta(days=6), today)), + "mau": mau, + "stickiness": _rate(avg_dau, mau), + "accounts": sum(1 for a in accounts if not a.get("disabled")), + "active_users": len(current), + "new_users": sum(1 for a in accounts if _day(a.get("created_at")) >= start.isoformat()), + "churn": { + "previous_active": len(before), + "retained": len(before & current), + "churned": len(churned), + "rate": _rate(len(churned), len(before)), + "returning": len(current - before), + }, + "retention": cohorts, + "features": _features(db, dt.datetime.combine(start, dt.time(), dt.timezone.utc)), + "last_active_day": last, + "days_since_active": (today - dt.date.fromisoformat(last)).days if last else None, + } + + +# ---------- Google Analytics ---------- +def _section(cfg): + return (cfg or {}).get("analytics") or {} + + +def configured(cfg): + """On, with a measurement id and an API secret to send with.""" + a = _section(cfg) + return bool(a.get("enabled") and a.get("measurement_id") and isinstance(a.get("api_secret"), str) and a["api_secret"]) + + +def _state(db): + return db.one("SELECT * OMIT id FROM $r", r=ROW) or {} + + +def _put(db, **fields): + db.q("UPSERT $r MERGE $d", r=ROW, d=fields) + + +def client_id(db): + """This install's random id for Google Analytics: made once, here; not derived from anything about the machine.""" + st = _state(db) + if st.get("client_id"): + return st["client_id"] + cid = str(uuid.uuid4()) + _put(db, client_id=cid) + return _state(db).get("client_id") or cid + + +def _url(cfg, debug=False): + a = _section(cfg) + base = HOSTS.get(a.get("region") or "global", HOSTS["global"]) + base = base if "://" in base else f"https://{base}" + return f"{base}/{'debug/' if debug else ''}mp/collect" + + +def _platform(): + return f"{platform.system().lower() or 'unknown'}-{platform.machine().lower() or 'unknown'}"[:40] + + +def payload(cfg, cid, events, day=None): + """The Measurement Protocol body: the install's id, its version and platform as user properties, and the events, + each counted as engaged so GA's active-user and retention reports include it.""" + from . import __version__ + + session = int(dt.datetime.combine(day or dt.date.today(), dt.time(), dt.timezone.utc).timestamp()) + return { + "client_id": cid, + "non_personalized_ads": True, + "user_properties": {"lens_version": {"value": __version__}, "lens_platform": {"value": _platform()}}, + "events": [{"name": name, "params": {**params, "session_id": session, "engagement_time_msec": 1}} for name, params in events], + } + + +def send(db, cfg, events, day=None, debug=False): + """Send events ([(name, params)]) now; (ok, error, validation messages). Only events analytics.events names.""" + a = _section(cfg) + wanted = set(a.get("events") or EVENTS) + events = [(n, p) for n, p in events if n in wanted or (debug and n == "lens_test")] + if not events: + return True, None, [] + cid, msgs = client_id(db), [] + params = {"measurement_id": a.get("measurement_id"), "api_secret": a.get("api_secret")} + try: + for i in range(0, len(events), BATCH): + r = httpx.post(_url(cfg, debug), params=params, json=payload(cfg, cid, events[i : i + BATCH], day), timeout=10) + if r.status_code >= 300: + raise ValueError(f"Google Analytics answered {r.status_code}") + if debug: + msgs += [str(m.get("description") or m)[:300] for m in (r.json().get("validationMessages") or [])] + except (httpx.HTTPError, ValueError) as e: + error = f"couldn't send to Google Analytics: {type(e).__name__}: {e}"[:300].replace(a.get("api_secret") or "\0", "…") + if not debug: + _put(db, last_error=error, last_error_at=store.now()) + return False, error, msgs + if not debug: + _put(db, last_sent_at=store.now(), last_sent=[n for n, _ in events], last_error=None) + return not msgs, None, msgs + + +def _claim(db, day): + """True for the one process that gets to send `day`'s events.""" + if _state(db).get("day") == day: + return False + client_id(db) + got = db.rows("UPDATE $r SET day = $d WHERE day != $d RETURN AFTER", r=ROW, d=day) + return bool(got) + + +def tick(db, cfg, now=None): + """Once a day, while analytics is on and set up: send what happened since the last time (docs/analytics.md).""" + if not configured(cfg): + return [] + from . import __version__ + + now = now or dt.datetime.now(dt.timezone.utc) + today = now.date() + if not _claim(db, today.isoformat()): + return [] + st, events = _state(db), [] + if not st.get("installed"): + events.append(("lens_install", {"version": __version__, "platform": _platform()})) + wiz = db.one("SELECT done_at, skipped FROM $r", r=R("setup", "wizard")) or {} + if wiz.get("done_at") and not st.get("setup_done"): + events.append(("lens_setup_done", {"skipped": 1 if wiz.get("skipped") else 0})) + if st.get("version") and st["version"] != __version__: + events.append(("lens_upgrade", {"from_version": st["version"], "to_version": __version__})) + yesterday = today - dt.timedelta(days=1) + seen = _days(db, yesterday) + users = {w for (w, d) in seen if d == yesterday.isoformat()} + inactive_days = int(_section(cfg).get("inactive_days") or 30) + marks = {"installed": True, "setup_done": bool(wiz.get("done_at")), "version": __version__} + if users: + events.append( + ( + "lens_daily_active", + {"active_users": len(users), "actions": sum(n for (_, d), n in seen.items() if d == yesterday.isoformat())}, + ) + ) + start = dt.datetime.combine(yesterday, dt.time(), dt.timezone.utc) + for f in _features(db, start, start + dt.timedelta(days=1))[:20]: + events.append(("lens_feature_used", f)) + if st.get("inactive_since"): + away = (yesterday - dt.date.fromisoformat(st["inactive_since"])).days + events.append(("lens_return", {"days_away": max(away, inactive_days)})) + marks["inactive_since"] = None + elif not st.get("inactive_since"): + last = (db.one("SELECT at FROM activity WHERE kind = 'in' AND actor != NONE ORDER BY at DESC LIMIT 1") or {}).get("at") + last = last or (db.one("SELECT at FROM audit_log WHERE action = 'login' ORDER BY at DESC LIMIT 1") or {}).get("at") + since = dt.date.fromisoformat(_day(last)) if last else None + if since and (today - since).days >= inactive_days: + events.append(("lens_inactive", {"days": (today - since).days})) + marks["inactive_since"] = since.isoformat() + ok, error, _ = send(db, cfg, events, day=yesterday) + if ok: + _put(db, **marks) + else: + log.info("analytics: %s", error) + db.q("UPDATE $r SET day = NONE", r=ROW) # try again on the next tick + return [n for n, _ in events] + + +def test(db, cfg): + """Send one lens_test event to GA's validation server (it's checked, not recorded); (ok, error, messages).""" + a = _section(cfg) + if not a.get("measurement_id") or not isinstance(a.get("api_secret"), str) or not a.get("api_secret"): + return False, "set the measurement id and the API secret first", [] + from . import __version__ + + return send(db, cfg, [("lens_test", {"version": __version__})], debug=True) + + +def status(db, cfg): + """Whether analytics is on and how the last send went, for Settings.""" + a, st = _section(cfg), _state(db) + return { + "enabled": bool(a.get("enabled")), + "configured": configured(cfg), + "measurement_id": a.get("measurement_id"), + "client_id": st.get("client_id"), + "last_sent_at": st.get("last_sent_at"), + "last_sent": st.get("last_sent") or [], + "last_error": st.get("last_error"), + "events": [{"name": k, "description": v, "on": k in set(a.get("events") or EVENTS)} for k, v in EVENTS.items()], + } diff --git a/fastapi_backend/app/domain/ops_tools.py b/fastapi_backend/app/domain/ops_tools.py index 8b494c627..991e7510d 100644 --- a/fastapi_backend/app/domain/ops_tools.py +++ b/fastapi_backend/app/domain/ops_tools.py @@ -38,9 +38,11 @@ "bridge", "matterbridge", "telemetry", + "analytics", "sensors", ) -ALWAYS_ASK = {"telemetry", "sensors", "files"} # sensors open ports on the network; files moves where new files go +# telemetry and analytics send data elsewhere; sensors open ports on the network; files moves where new files go +ALWAYS_ASK = {"telemetry", "analytics", "sensors", "files"} ADMIN_TOOLS = [ ( @@ -138,6 +140,7 @@ def status(db, cfg, base): "max_upload_mb": view["storage"]["max_upload_mb"], "jobs": q, "telemetry_on": view["telemetry"]["enabled"], + "google_analytics_on": view["telemetry"]["analytics"]["enabled"], # engines and models the workers are still fetching for themselves, or couldn't (Settings → Components) "components_not_ready": fetching, } diff --git a/fastapi_backend/app/domain/routines.py b/fastapi_backend/app/domain/routines.py index 1cc0b7b9d..ac6b1ae6c 100644 --- a/fastapi_backend/app/domain/routines.py +++ b/fastapi_backend/app/domain/routines.py @@ -24,6 +24,7 @@ from . import ( activity, + analytics, budgets, fedora, jobs, @@ -601,6 +602,11 @@ def run_due(db, cfg, log=print, now=None): stamp = _iso(now) sweep(db, now) activity.tidy(db, cfg) # the activity ledger past activity.keep_days, once an hour + try: + analytics.tick(db, cfg, now) # once a day, while Google Analytics is on (docs/analytics.md) + except Exception as e: # noqa: BLE001 - the schedule carries on + if log: + log(f"analytics: {type(e).__name__}: {e}") try: budgets.sweep(db, cfg, every=budgets.SWEEP_SECONDS) # budgets that crossed warn_at or their cap: warned once except Exception as e: # noqa: BLE001 - the schedule carries on diff --git a/fastapi_backend/app/domain/settings.py b/fastapi_backend/app/domain/settings.py index 4941151ea..9c2c49478 100644 --- a/fastapi_backend/app/domain/settings.py +++ b/fastapi_backend/app/domain/settings.py @@ -47,6 +47,7 @@ "matterbridge": None, "notifications": None, "telemetry": None, + "analytics": None, # repo is a startup setting only (what gets installed isn't chosen in the web app) "updates": ("check", "check_hours", "auto_install"), "activity": None, @@ -117,6 +118,7 @@ "bridge": ("token",), "matterbridge": ("slack_token", "discord_token", "telegram_token", "matrix_password"), "telemetry": ("headers",), + "analytics": ("api_secret",), "fedora": ("password",), "tunnel": ("token", "api_token"), "documents": ("anytopdf_token",), @@ -144,6 +146,7 @@ ("tunnel", "mode"): {"off", "quick", "token", "managed"}, ("files", "store"): {"local", "connection"}, ("documents", "converter"): {"auto", "anytopdf", "lens"}, + ("analytics", "region"): {"global", "eu"}, } # Settings the environment (.env) sets, which win over archive.yaml and the app and show as locked there: the # break-glass allowed hosts, the model provider so an install can be configured without the setup wizard, and @@ -167,6 +170,9 @@ ("telemetry", "enabled"): "LENS_TELEMETRY", ("telemetry", "endpoint"): "LENS_TELEMETRY_ENDPOINT", ("telemetry", "headers"): "LENS_TELEMETRY_HEADERS", + ("analytics", "enabled"): "LENS_ANALYTICS", + ("analytics", "measurement_id"): "LENS_GA_MEASUREMENT_ID", + ("analytics", "api_secret"): "LENS_GA_API_SECRET", ("fedora", "enabled"): "LENS_FEDORA", ("fedora", "url"): "LENS_FEDORA_URL", ("fedora", "user"): "LENS_FEDORA_USER", @@ -396,6 +402,8 @@ def _check(section, key, value, default): return _notify_setting(key, value) if section == "telemetry": return _telemetry_setting(key, value) + if section == "analytics" and key != "region": + return _analytics_setting(key, value) if section == "updates": return _updates_setting(key, value) if section == "activity": @@ -898,6 +906,31 @@ def _telemetry_setting(key, value): return value +def _analytics_setting(key, value): + from . import analytics + + if key == "enabled": + if not isinstance(value, bool): + raise ValueError("analytics.enabled is true or false") + return value + if key == "measurement_id": + if value in (None, ""): + return None + v = value.strip().upper() if isinstance(value, str) else "" + if not analytics.MEASUREMENT_RX.match(v): + raise ValueError("analytics.measurement_id is a GA4 measurement id, like G-ABC123XYZ") + return v + if key == "events": + if not (isinstance(value, list) and all(isinstance(x, str) for x in value)) or set(value) - set(analytics.EVENTS): + raise ValueError(f"analytics.events is a list of: {', '.join(analytics.EVENTS)}") + return [e for e in analytics.EVENTS if e in value] + if key == "inactive_days": + if not (isinstance(value, int) and not isinstance(value, bool) and 1 <= value <= 365): + raise ValueError("analytics.inactive_days is a whole number of days from 1 to 365") + return value + raise ValueError(f"unknown setting analytics.{key}") + + def _origin(url): try: u = urllib.parse.urlsplit(url or "") diff --git a/fastapi_backend/app/domain/setup.py b/fastapi_backend/app/domain/setup.py index a4ab3255a..fe79ebda0 100644 --- a/fastapi_backend/app/domain/setup.py +++ b/fastapi_backend/app/domain/setup.py @@ -178,6 +178,12 @@ def view(db, cfg): "enabled": bool(shown["telemetry"].get("enabled")), "endpoint": shown["telemetry"].get("endpoint"), "locked": [k for k in settings.locked("telemetry") if k in ("enabled", "endpoint")], + "analytics": { + "enabled": bool(shown["analytics"].get("enabled")), + "measurement_id": shown["analytics"].get("measurement_id"), + "secret_set": isinstance(shown["analytics"].get("api_secret"), dict), + "locked": settings.locked("analytics"), + }, }, "oauth": { "enabled": bool(shown["tokens"].get("oauth_enabled", True)), @@ -219,6 +225,27 @@ def save_telemetry(db, cfg, enabled, endpoint, user=None): return sorted(keep) +def save_analytics(db, cfg, enabled, measurement_id=None, api_secret=None, user=None): + """Opt in to Google Analytics (or stay out), leaving out what the environment sets. On needs a measurement id and + an API secret, given here or saved before.""" + locked = settings.locked("analytics") + values = {"enabled": bool(enabled)} + if (measurement_id or "").strip(): + values["measurement_id"] = measurement_id.strip() + if (api_secret or "").strip(): + values["api_secret"] = api_secret.strip() + keep = {k: v for k, v in values.items() if k not in locked} + if enabled: + shown = settings.effective(db, cfg, reveal=False)["analytics"] + if not (values.get("measurement_id") or shown.get("measurement_id")): + raise ValueError("Google Analytics needs a GA4 measurement id, like G-ABC123XYZ") + if not (values.get("api_secret") or shown.get("api_secret")): + raise ValueError("Google Analytics needs a Measurement Protocol API secret (Admin → Data streams in GA)") + if keep: + settings.save(db, cfg, "analytics", keep, user) + return sorted(keep) + + def save_oauth(db, cfg, enabled, access_minutes=None, refresh_days=None, user=None): """Whether apps and MCP clients may sign people in through OAuth, and how long their tokens last.""" values = {"oauth_enabled": bool(enabled)} diff --git a/fastapi_backend/app/domain/store.py b/fastapi_backend/app/domain/store.py index cbae2c2df..5dfaf4dcc 100644 --- a/fastapi_backend/app/domain/store.py +++ b/fastapi_backend/app/domain/store.py @@ -482,6 +482,24 @@ def labels(): "service_name": "lens", "prices": {}, }, + # usage analytics (docs/analytics.md): the numbers are worked out locally from the activity ledger; Google Analytics + # is off unless an admin turns it on with a GA4 measurement id and a Measurement Protocol API secret (sealed) + "analytics": { + "enabled": False, + "measurement_id": None, + "api_secret": None, + "region": "global", + "events": [ + "lens_install", + "lens_setup_done", + "lens_daily_active", + "lens_feature_used", + "lens_upgrade", + "lens_inactive", + "lens_return", + ], + "inactive_days": 30, + }, # new releases (docs/updates.md): check asks GitHub for repo's latest release every check_hours, for the web app's # banner; auto_install installs one as soon as it's seen, through the machine's upgrade agent (install.sh sets it up) "updates": {"check": True, "check_hours": 24, "auto_install": False, "repo": "adeelahmad/lense"}, diff --git a/fastapi_backend/app/schemas/admin.py b/fastapi_backend/app/schemas/admin.py index 6a843ad4b..bba0f0bc6 100644 --- a/fastapi_backend/app/schemas/admin.py +++ b/fastapi_backend/app/schemas/admin.py @@ -223,6 +223,91 @@ class TelemetryTestResult(ResponseModel): ms: int | None = None +class AnalyticsEvent(ResponseModel): + name: str + description: str + # analytics.events names it + on: bool + + +class AnalyticsStatus(ResponseModel): + """Whether usage statistics go to Google Analytics, and how the last send went (docs/analytics.md).""" + + enabled: bool + # on, with a measurement id and an API secret + configured: bool + measurement_id: str | None = None + # this install's random id in Google Analytics + client_id: str | None = None + last_sent_at: str | None = None + last_sent: list[str] = [] + last_error: str | None = None + events: list[AnalyticsEvent] + + +class AnalyticsTestResult(ResponseModel): + ok: bool + error: str | None = None + # what Google Analytics' validation server said about the test event + messages: list[str] = [] + + +class AnalyticsDay(ResponseModel): + day: str + active_users: int + actions: int + + +class AnalyticsChurn(ResponseModel): + # people active in the period before this one + previous_active: int + # of those, active again in this one + retained: int + # of those, not active in this one + churned: int + # churned / previous_active; null with no one before + rate: float | None = None + # active now, not in the period before + returning: int + + +class AnalyticsCohort(ResponseModel): + # the Monday of the week these accounts were made + week: str + users: int + # the share of them active in that week and each week after + retention: list[float | None] + + +class AnalyticsFeature(ResponseModel): + feature: str + actions: int + users: int + + +class AnalyticsMetrics(ResponseModel): + """Active people, churn, retention and feature use, worked out on this machine from the activity ledger and + sign-ins (docs/analytics.md#the-numbers).""" + + days: int + since: str + until: str + daily: list[AnalyticsDay] + dau: int + wau: int + mau: int + # average daily active / monthly active + stickiness: float | None = None + accounts: int + active_users: int + new_users: int + churn: AnalyticsChurn + retention: list[AnalyticsCohort] + features: list[AnalyticsFeature] + last_active_day: str | None = None + days_since_active: int | None = None + + class FileStoreTry(RequestModel): """Where to try keeping files, before saving it (Settings → Storage); nothing given tries what is saved.""" diff --git a/fastapi_backend/app/schemas/setup.py b/fastapi_backend/app/schemas/setup.py index 4f9498f74..fba566d81 100644 --- a/fastapi_backend/app/schemas/setup.py +++ b/fastapi_backend/app/schemas/setup.py @@ -30,11 +30,22 @@ class SetupStorageView(ResponseModel): watches: int +class SetupAnalyticsView(ResponseModel): + enabled: bool + measurement_id: str | None = None + # an API secret is saved (it is never shown) + secret_set: bool + # what .env sets (LENS_ANALYTICS, LENS_GA_MEASUREMENT_ID, LENS_GA_API_SECRET): shown locked + locked: list[str] + + class SetupTelemetryView(ResponseModel): enabled: bool endpoint: str | None = None # what .env sets (LENS_TELEMETRY, LENS_TELEMETRY_ENDPOINT): shown locked locked: list[str] + # usage statistics sent to the admin's own Google Analytics property (docs/analytics.md); off by default + analytics: SetupAnalyticsView class SetupOAuthView(ResponseModel): @@ -85,6 +96,11 @@ class SetupTelemetry(RequestModel): enabled: bool = False # the OTLP/HTTP address of a collector, like http://localhost:4318; needed to turn telemetry on endpoint: str | None = Field(default=None, max_length=500) + # Google Analytics (docs/analytics.md): left out, unchanged + analytics: bool | None = None + measurement_id: str | None = Field(default=None, max_length=40) + # empty: leave the saved secret as it is + api_secret: str | None = Field(default=None, max_length=200) class SetupOAuth(RequestModel): diff --git a/fastapi_backend/tests/api/test_analytics.py b/fastapi_backend/tests/api/test_analytics.py new file mode 100644 index 000000000..b3e50d3c3 --- /dev/null +++ b/fastapi_backend/tests/api/test_analytics.py @@ -0,0 +1,204 @@ +"""Usage analytics: the numbers worked out locally, and Google Analytics off by default, opt-in, counts only.""" + +from __future__ import annotations + +import datetime as dt + +import httpx +import pytest + +from app.domain import __version__, analytics, routines, store +from tests.helpers import login, make_user + +NOW = dt.datetime(2026, 10, 10, 12, tzinfo=dt.UTC) + + +def admin(client, db): + make_user(db, "root@x.io", "root password 1", admin=True) + return login(client, "root@x.io", "root password 1") + + +def act(db, who, day, action="POST /api/v1/namespaces/{ns}/recordings", n=1): + for _ in range(n): + db.q("CREATE activity CONTENT $d", d={"at": f"{day}T10:00:00.000+00:00", "kind": "in", "actor": who, "action": action, "ok": True}) + + +def account(db, uid, created): + db.q("UPSERT $r CONTENT $d", r=store.R("account", uid), d={"email": f"u{uid}@x.io", "created_at": f"{created}T09:00:00+00:00"}) + + +class Sent(list): + state: dict + + +@pytest.fixture +def ga(monkeypatch): + """A fake Google Analytics: records each request; `status` sets its answer.""" + sent = Sent() + state = {"status": 204, "messages": []} + + def post(url, params=None, json=None, timeout=None): + sent.append({"url": url, "params": params, "body": json}) + return httpx.Response(state["status"], json={"validationMessages": state["messages"]}, request=httpx.Request("POST", url)) + + monkeypatch.setattr(analytics.httpx, "post", post) + sent.state = state + return sent + + +def on(cfg, **more): + cfg["analytics"].update(enabled=True, measurement_id="G-TEST1234", api_secret="s3cret", **more) + return cfg + + +def names(sent): + return [e["name"] for s in sent for e in s["body"]["events"]] + + +def test_feature_names_come_from_route_templates(): + assert analytics.feature("POST /api/v1/namespaces/{ns}/recordings") == "recordings" + assert analytics.feature("PUT /api/v1/chats/{cid}") == "chats" + assert analytics.feature("POST /api/v1/namespaces") == "namespaces" + assert analytics.feature("POST /api/v1/{weird}") == "other" + assert analytics.feature("") == "other" + + +def test_metrics_active_churn_retention(db): + today = NOW.date() + + def d(n): + return (today - dt.timedelta(days=n)).isoformat() + + # 1 and 2 were active in the previous 7 days; only 1 came back; 3 is new this week + account(db, 1, d(40)) + account(db, 2, d(40)) + account(db, 3, d(2)) + act(db, 1, d(10)) + act(db, 2, d(9)) + act(db, 1, d(1), "PUT /api/v1/chats/{cid}", n=2) + act(db, 3, d(0)) + db.q("CREATE audit_log CONTENT $d", d={"at": f"{d(0)}T08:00:00+00:00", "account": 1, "action": "login"}) + m = analytics.metrics(db, days=7, now=NOW) + assert (m["since"], m["until"], len(m["daily"])) == (d(6), d(0), 7) + assert m["daily"][-1] == {"day": d(0), "active_users": 2, "actions": 2} + assert (m["dau"], m["wau"], m["mau"], m["active_users"], m["accounts"], m["new_users"]) == (2, 2, 3, 2, 3, 1) + assert m["churn"] == {"previous_active": 2, "retained": 1, "churned": 1, "rate": 0.5, "returning": 1} + assert m["stickiness"] == round((1 + 2) / 7 / 3, 4) + assert [f["feature"] for f in m["features"]] == ["chats", "recordings"] + assert m["features"][0] == {"feature": "chats", "actions": 2, "users": 1} + assert (m["last_active_day"], m["days_since_active"]) == (d(0), 0) + week = [c for c in m["retention"] if c["users"] == 1] + assert week and week[-1]["retention"][0] == 1.0 # 3 was active in its sign-up week + + +def test_off_by_default_sends_nothing(db, cfg, ga): + act(db, 1, (NOW.date() - dt.timedelta(days=1)).isoformat()) + assert cfg["analytics"]["enabled"] is False + assert analytics.tick(db, cfg, NOW) == [] + routines.run_due(db, cfg, log=None, now=NOW) + assert ga == [] + # on without a secret isn't set up either + cfg["analytics"].update(enabled=True, measurement_id="G-TEST1234") + assert analytics.tick(db, cfg, NOW) == [] + assert ga == [] + + +def test_daily_events_once_a_day_counts_only(db, cfg, ga): + on(cfg) + yesterday = (NOW.date() - dt.timedelta(days=1)).isoformat() + act(db, 7, yesterday, n=3) + act(db, 8, yesterday, "PUT /api/v1/chats/{cid}") + db.q("UPSERT setup:wizard CONTENT {done_at: '2026-10-01T00:00:00+00:00', skipped: false}") + got = analytics.tick(db, cfg, NOW) + assert got == ["lens_install", "lens_setup_done", "lens_daily_active", "lens_feature_used", "lens_feature_used"] + assert analytics.tick(db, cfg, NOW + dt.timedelta(hours=3)) == [] # once a day + (req,) = ga + assert req["url"] == "https://www.google-analytics.com/mp/collect" + assert req["params"] == {"measurement_id": "G-TEST1234", "api_secret": "s3cret"} + body = req["body"] + assert body["client_id"] == analytics.client_id(db) + assert body["user_properties"]["lens_version"] == {"value": __version__} + ev = {e["name"]: e["params"] for e in body["events"]} + assert ev["lens_daily_active"]["active_users"] == 2 and ev["lens_daily_active"]["actions"] == 4 + assert ev["lens_daily_active"]["engagement_time_msec"] == 1 + feats = [e["params"] for e in body["events"] if e["name"] == "lens_feature_used"] + assert [(f["feature"], f["actions"], f["users"]) for f in feats] == [("recordings", 3, 1), ("chats", 1, 1)] + # nothing personal: no emails, account ids or paths + text = str(body) + assert "x.io" not in text and "{cid}" not in text and "'actor'" not in text + # the next day: no install or setup again; a new version is an upgrade + db.q("UPDATE $r SET version = '0.0.1'", r=analytics.ROW) + assert analytics.tick(db, cfg, NOW + dt.timedelta(days=1)) == ["lens_upgrade"] + assert ga[-1]["body"]["events"][0]["params"]["from_version"] == "0.0.1" + + +def test_inactive_then_return(db, cfg, ga): + on(cfg, inactive_days=5) + act(db, 1, (NOW.date() - dt.timedelta(days=9)).isoformat()) + assert analytics.tick(db, cfg, NOW) == ["lens_install", "lens_inactive"] + assert ga[-1]["body"]["events"][1]["params"]["days"] == 9 + assert "lens_inactive" not in analytics.tick(db, cfg, NOW + dt.timedelta(days=1)) # once per stretch + act(db, 1, (NOW.date() + dt.timedelta(days=1)).isoformat()) + got = analytics.tick(db, cfg, NOW + dt.timedelta(days=2)) + assert "lens_return" in got and "lens_daily_active" in got + + +def test_events_can_be_left_out_and_failures_retry(db, cfg, ga): + on(cfg, events=["lens_install"]) + act(db, 1, (NOW.date() - dt.timedelta(days=1)).isoformat()) + ga.state["status"] = 500 + analytics.tick(db, cfg, NOW) + assert names(ga) == ["lens_install"] + assert "500" in analytics.status(db, cfg)["last_error"] + ga.state["status"] = 204 + analytics.tick(db, cfg, NOW + dt.timedelta(minutes=5)) # the failed day is tried again + assert names(ga) == ["lens_install", "lens_install"] + st = analytics.status(db, cfg) + assert st["last_error"] is None and st["last_sent"] == ["lens_install"] + + +def test_settings_api_status_and_test(client, db, ga): + h = admin(client, db) + a = client.get("/api/v1/settings", headers=h).json()["analytics"] + assert a["values"]["enabled"] is False and a["values"]["api_secret"] == {"secret": True, "set": False} + + def put(body): + return client.put("/api/v1/settings/analytics", json=body, headers=h) + + for bad in ({"measurement_id": "UA-1234-1"}, {"events": ["page_view"]}, {"inactive_days": 0}, {"region": "mars"}, {"enabled": "yes"}): + assert put(bad).status_code == 400, bad + r = client.post("/api/v1/settings/analytics/test", headers=h).json() + assert not r["ok"] and "measurement id" in r["error"] + assert put({"measurement_id": "g-test1234", "api_secret": "s3cret", "region": "eu"}).status_code == 200 + a = client.get("/api/v1/settings", headers=h).json()["analytics"]["values"] + assert a["measurement_id"] == "G-TEST1234" and a["api_secret"] == {"secret": True, "set": True} + ga.state["messages"] = [{"description": "event name is bad"}] + r = client.post("/api/v1/settings/analytics/test", headers=h).json() + assert r == {"ok": False, "error": None, "messages": ["event name is bad"]} + assert ga[-1]["url"] == "https://region1.google-analytics.com/debug/mp/collect" + assert ga[-1]["body"]["events"][0]["name"] == "lens_test" + s = client.get("/api/v1/settings/analytics/status", headers=h).json() + assert s["enabled"] is False and s["configured"] is False and len(s["events"]) == len(analytics.EVENTS) + assert all(e["on"] for e in s["events"]) + m = client.get("/api/v1/admin/analytics?days=7", headers=h).json() + assert m["days"] == 7 and m["accounts"] == 1 and m["dau"] == 1 # the sign-in above + make_user(db, "ed@x.io", "ed password 12") + h2 = login(client, "ed@x.io", "ed password 12") + assert client.get("/api/v1/admin/analytics", headers=h2).status_code == 403 + + +def test_setup_wizard_opt_in(client, db): + h = admin(client, db) + r = client.put("/api/v1/setup/telemetry", json={"enabled": False, "analytics": True}, headers=h) + assert r.status_code == 400 and "measurement id" in r.json()["detail"] + r = client.put( + "/api/v1/setup/telemetry", + json={"enabled": False, "analytics": True, "measurement_id": "G-TEST1234", "api_secret": "s3cret"}, + headers=h, + ) + assert r.status_code == 200 and "analytics.enabled" in r.json()["saved"] + v = client.get("/api/v1/setup", headers=h).json()["telemetry"]["analytics"] + assert v == {"enabled": True, "measurement_id": "G-TEST1234", "secret_set": True, "locked": []} + # off again keeps the id and secret, and no secret is needed to turn it back on + client.put("/api/v1/setup/telemetry", json={"enabled": False, "analytics": False}, headers=h) + assert client.put("/api/v1/setup/telemetry", json={"enabled": False, "analytics": True}, headers=h).status_code == 200 diff --git a/fastapi_backend/tests/api/test_telemetry.py b/fastapi_backend/tests/api/test_telemetry.py index dc90423b6..852142373 100644 --- a/fastapi_backend/tests/api/test_telemetry.py +++ b/fastapi_backend/tests/api/test_telemetry.py @@ -212,16 +212,31 @@ def test_environment_turns_it_on_or_keeps_it_off(client, db, collector, monkeypa def test_setup_wizard_opt_in(client, db, collector): h = admin(client, db) v = client.get("/api/v1/setup", headers=h).json()["telemetry"] - assert v == {"enabled": False, "endpoint": None, "locked": []} + assert v == { + "enabled": False, + "endpoint": None, + "locked": [], + "analytics": {"enabled": False, "measurement_id": None, "secret_set": False, "locked": []}, + } r = client.put("/api/v1/setup/telemetry", json={"enabled": True}, headers=h) assert r.status_code == 400 and "endpoint" in r.json()["detail"] assert client.put("/api/v1/setup/telemetry", json={"enabled": True, "endpoint": "nope"}, headers=h).status_code == 400 r = client.put("/api/v1/setup/telemetry", json={"enabled": True, "endpoint": collector}, headers=h) assert r.json()["saved"] == ["enabled", "endpoint"] - assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == {"enabled": True, "endpoint": collector, "locked": []} + assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == { + "enabled": True, + "endpoint": collector, + "locked": [], + "analytics": {"enabled": False, "measurement_id": None, "secret_set": False, "locked": []}, + } # off keeps the address for later client.put("/api/v1/setup/telemetry", json={"enabled": False}, headers=h) - assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == {"enabled": False, "endpoint": collector, "locked": []} + assert client.get("/api/v1/setup", headers=h).json()["telemetry"] == { + "enabled": False, + "endpoint": collector, + "locked": [], + "analytics": {"enabled": False, "measurement_id": None, "secret_set": False, "locked": []}, + } audit = client.get("/api/v1/audit", headers=h).json() assert any(a["action"] == "settings.save" and a["target"] == "telemetry" for a in audit) diff --git a/mkdocs.yml b/mkdocs.yml index d1f90077a..289851ddd 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -79,6 +79,7 @@ nav: - Chat rooms: chat-rooms.md - Sensors: sensors.md - Telemetry: telemetry.md + - Usage analytics: analytics.md - Activity and costs: activity.md - Budgets: budgets.md - IIIF: iiif.md diff --git a/nextjs-frontend/__tests__/setupWizard.test.tsx b/nextjs-frontend/__tests__/setupWizard.test.tsx index 0fcb1de53..36b3bb9ff 100644 --- a/nextjs-frontend/__tests__/setupWizard.test.tsx +++ b/nextjs-frontend/__tests__/setupWizard.test.tsx @@ -41,7 +41,12 @@ const VIEW = (over: Partial = {}): SetupView => ({ max_upload_mb: 4096, watches: 0, }, - telemetry: { enabled: false, endpoint: null, locked: [] }, + telemetry: { + enabled: false, + endpoint: null, + locked: [], + analytics: { enabled: false, measurement_id: null, secret_set: false, locked: [] }, + }, oauth: { enabled: true, access_minutes: 60, refresh_days: 30 }, ...over, }); diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index 32ab2f424..a50ac936a 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -96,6 +96,17 @@ export type { AddVaultPasskeyErrors, AddVaultPasskeyResponse, AddVaultPasskeyResponses, + AnalyticsChurn, + AnalyticsCohort, + AnalyticsDay, + AnalyticsEvent, + AnalyticsFeature, + AnalyticsMetrics, + AnalyticsStatus, + AnalyticsStatusData, + AnalyticsStatusResponse, + AnalyticsStatusResponses, + AnalyticsTestResult, AnswerCheck, AnswerData, AnswerError, @@ -2378,6 +2389,7 @@ export type { SetRecordingContentTypeResponse, SetRecordingContentTypeResponses, SetupAdmin, + SetupAnalyticsView, SetupData, SetupError, SetupErrors, @@ -2535,6 +2547,9 @@ export type { TemplateVersionCreate, TemplateVersionInfo, TermSuggestion, + TestAnalyticsData, + TestAnalyticsResponse, + TestAnalyticsResponses, TestBridgeData, TestBridgeResponse, TestBridgeResponses, @@ -2861,6 +2876,11 @@ export type { UploadLimitsResponse, UploadLimitsResponses, UploadStart, + UsageMetricsData, + UsageMetricsError, + UsageMetricsErrors, + UsageMetricsResponse, + UsageMetricsResponses, UserCreate, UserPublic, UserUpdate, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index 6e5801f34..bacf4113b 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -34,6 +34,8 @@ import type { AddVaultPasskeyData, AddVaultPasskeyErrors, AddVaultPasskeyResponses, + AnalyticsStatusData, + AnalyticsStatusResponses, AnswerData, AnswerErrors, AnswerResponses, @@ -1221,6 +1223,8 @@ import type { TakeBackupResponses, TelemetryStatusData, TelemetryStatusResponses, + TestAnalyticsData, + TestAnalyticsResponses, TestBridgeData, TestBridgeResponses, TestConversionNodeData, @@ -1405,6 +1409,9 @@ import type { UpdateWorkflowResponses, UploadLimitsData, UploadLimitsResponses, + UsageMetricsData, + UsageMetricsErrors, + UsageMetricsResponses, VaultOptionsData, VaultOptionsErrors, VaultOptionsResponses, @@ -2621,6 +2628,49 @@ export class Admin { }); } + /** + * Analytics Status + * + * Whether usage statistics go to Google Analytics, which events, and how the last send went. Off by default. + */ + public static analyticsStatus( + options?: Options, + ): RequestResult { + return (options?.client ?? client).get({ + url: "/api/v1/settings/analytics/status", + ...options, + }); + } + + /** + * Test Analytics + * + * Send one lens_test event to Google Analytics' validation server (checked there, not recorded), on or off. + */ + public static testAnalytics( + options?: Options, + ): RequestResult { + return (options?.client ?? client).post({ + url: "/api/v1/settings/analytics/test", + ...options, + }); + } + + /** + * Usage Metrics + * + * Daily, weekly and monthly active people, churn, retention by sign-up week and feature use over the last `days` + * days, worked out here from the activity ledger and sign-ins; nothing is sent anywhere. + */ + public static usageMetrics( + options?: Options, + ): RequestResult { + return (options?.client ?? client).get({ + url: "/api/v1/admin/analytics", + ...options, + }); + } + /** * List Audit * @@ -3060,7 +3110,8 @@ export class Setup { /** * Save Telemetry * - * Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector). Off unless chosen. + * Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector), and usage statistics + * to your own Google Analytics property when `analytics` is true. Each is off unless chosen. */ public static saveTelemetry( options: Options, diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index fd83bbb05..34e1d4dc1 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -288,6 +288,240 @@ export type ActivityTotals = { [key: string]: unknown; }; +/** + * AnalyticsChurn + */ +export type AnalyticsChurn = { + /** + * Previous Active + */ + previous_active: number; + /** + * Retained + */ + retained: number; + /** + * Churned + */ + churned: number; + /** + * Rate + */ + rate?: number | null; + /** + * Returning + */ + returning: number; + [key: string]: unknown; +}; + +/** + * AnalyticsCohort + */ +export type AnalyticsCohort = { + /** + * Week + */ + week: string; + /** + * Users + */ + users: number; + /** + * Retention + */ + retention: Array; + [key: string]: unknown; +}; + +/** + * AnalyticsDay + */ +export type AnalyticsDay = { + /** + * Day + */ + day: string; + /** + * Active Users + */ + active_users: number; + /** + * Actions + */ + actions: number; + [key: string]: unknown; +}; + +/** + * AnalyticsEvent + */ +export type AnalyticsEvent = { + /** + * Name + */ + name: string; + /** + * Description + */ + description: string; + /** + * On + */ + on: boolean; + [key: string]: unknown; +}; + +/** + * AnalyticsFeature + */ +export type AnalyticsFeature = { + /** + * Feature + */ + feature: string; + /** + * Actions + */ + actions: number; + /** + * Users + */ + users: number; + [key: string]: unknown; +}; + +/** + * AnalyticsMetrics + * + * Active people, churn, retention and feature use, worked out on this machine from the activity ledger and + * sign-ins (docs/analytics.md#the-numbers). + */ +export type AnalyticsMetrics = { + /** + * Days + */ + days: number; + /** + * Since + */ + since: string; + /** + * Until + */ + until: string; + /** + * Daily + */ + daily: Array; + /** + * Dau + */ + dau: number; + /** + * Wau + */ + wau: number; + /** + * Mau + */ + mau: number; + /** + * Stickiness + */ + stickiness?: number | null; + /** + * Accounts + */ + accounts: number; + /** + * Active Users + */ + active_users: number; + /** + * New Users + */ + new_users: number; + churn: AnalyticsChurn; + /** + * Retention + */ + retention: Array; + /** + * Features + */ + features: Array; + /** + * Last Active Day + */ + last_active_day?: string | null; + /** + * Days Since Active + */ + days_since_active?: number | null; + [key: string]: unknown; +}; + +/** + * AnalyticsStatus + * + * Whether usage statistics go to Google Analytics, and how the last send went (docs/analytics.md). + */ +export type AnalyticsStatus = { + /** + * Enabled + */ + enabled: boolean; + /** + * Configured + */ + configured: boolean; + /** + * Measurement Id + */ + measurement_id?: string | null; + /** + * Client Id + */ + client_id?: string | null; + /** + * Last Sent At + */ + last_sent_at?: string | null; + /** + * Last Sent + */ + last_sent?: Array; + /** + * Last Error + */ + last_error?: string | null; + /** + * Events + */ + events: Array; + [key: string]: unknown; +}; + +/** + * AnalyticsTestResult + */ +export type AnalyticsTestResult = { + /** + * Ok + */ + ok: boolean; + /** + * Error + */ + error?: string | null; + /** + * Messages + */ + messages?: Array; + [key: string]: unknown; +}; + /** * AnswerCheck */ @@ -13742,6 +13976,29 @@ export type SetupAdmin = { [key: string]: unknown; }; +/** + * SetupAnalyticsView + */ +export type SetupAnalyticsView = { + /** + * Enabled + */ + enabled: boolean; + /** + * Measurement Id + */ + measurement_id?: string | null; + /** + * Secret Set + */ + secret_set: boolean; + /** + * Locked + */ + locked: Array; + [key: string]: unknown; +}; + /** * SetupFinish */ @@ -13955,6 +14212,18 @@ export type SetupTelemetry = { * Endpoint */ endpoint?: string | null; + /** + * Analytics + */ + analytics?: boolean | null; + /** + * Measurement Id + */ + measurement_id?: string | null; + /** + * Api Secret + */ + api_secret?: string | null; }; /** @@ -13973,6 +14242,7 @@ export type SetupTelemetryView = { * Locked */ locked: Array; + analytics: SetupAnalyticsView; [key: string]: unknown; }; @@ -19193,6 +19463,68 @@ export type TestTelemetryResponses = { export type TestTelemetryResponse = TestTelemetryResponses[keyof TestTelemetryResponses]; +export type AnalyticsStatusData = { + body?: never; + path?: never; + query?: never; + url: "/api/v1/settings/analytics/status"; +}; + +export type AnalyticsStatusResponses = { + /** + * Successful Response + */ + 200: AnalyticsStatus; +}; + +export type AnalyticsStatusResponse = AnalyticsStatusResponses[keyof AnalyticsStatusResponses]; + +export type TestAnalyticsData = { + body?: never; + path?: never; + query?: never; + url: "/api/v1/settings/analytics/test"; +}; + +export type TestAnalyticsResponses = { + /** + * Successful Response + */ + 200: AnalyticsTestResult; +}; + +export type TestAnalyticsResponse = TestAnalyticsResponses[keyof TestAnalyticsResponses]; + +export type UsageMetricsData = { + body?: never; + path?: never; + query?: { + /** + * Days + */ + days?: number; + }; + url: "/api/v1/admin/analytics"; +}; + +export type UsageMetricsErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type UsageMetricsError = UsageMetricsErrors[keyof UsageMetricsErrors]; + +export type UsageMetricsResponses = { + /** + * Successful Response + */ + 200: AnalyticsMetrics; +}; + +export type UsageMetricsResponse = UsageMetricsResponses[keyof UsageMetricsResponses]; + export type ListAuditData = { body?: never; path?: never; diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index cfe128052..fcb2cf488 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -2732,6 +2732,96 @@ } } }, + "/api/v1/settings/analytics/status": { + "get": { + "tags": [ + "admin" + ], + "summary": "Analytics Status", + "description": "Whether usage statistics go to Google Analytics, which events, and how the last send went. Off by default.", + "operationId": "analytics_status", + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AnalyticsStatus" + } + } + } + } + } + } + }, + "/api/v1/settings/analytics/test": { + "post": { + "tags": [ + "admin" + ], + "summary": "Test Analytics", + "description": "Send one lens_test event to Google Analytics' validation server (checked there, not recorded), on or off.", + "operationId": "test_analytics", + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AnalyticsTestResult" + } + } + } + } + } + } + }, + "/api/v1/admin/analytics": { + "get": { + "tags": [ + "admin" + ], + "summary": "Usage Metrics", + "description": "Daily, weekly and monthly active people, churn, retention by sign-up week and feature use over the last `days`\ndays, worked out here from the activity ledger and sign-ins; nothing is sent anywhere.", + "operationId": "usage_metrics", + "parameters": [ + { + "name": "days", + "in": "query", + "required": false, + "schema": { + "type": "integer", + "maximum": 180, + "minimum": 1, + "default": 30, + "title": "Days" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AnalyticsMetrics" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/audit": { "get": { "tags": [ @@ -3812,7 +3902,7 @@ "setup" ], "summary": "Save Telemetry", - "description": "Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector). Off unless chosen.", + "description": "Opt in to telemetry: traces and metrics sent only to this endpoint (an OTLP collector), and usage statistics\nto your own Google Analytics property when `analytics` is true. Each is off unless chosen.", "operationId": "save_telemetry", "requestBody": { "content": { @@ -24755,6 +24845,381 @@ ], "title": "ActivityTotals" }, + "AnalyticsChurn": { + "properties": { + "previous_active": { + "type": "integer", + "title": "Previous Active" + }, + "retained": { + "type": "integer", + "title": "Retained" + }, + "churned": { + "type": "integer", + "title": "Churned" + }, + "rate": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Rate" + }, + "returning": { + "type": "integer", + "title": "Returning" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "previous_active", + "retained", + "churned", + "returning" + ], + "title": "AnalyticsChurn" + }, + "AnalyticsCohort": { + "properties": { + "week": { + "type": "string", + "title": "Week" + }, + "users": { + "type": "integer", + "title": "Users" + }, + "retention": { + "items": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ] + }, + "type": "array", + "title": "Retention" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "week", + "users", + "retention" + ], + "title": "AnalyticsCohort" + }, + "AnalyticsDay": { + "properties": { + "day": { + "type": "string", + "title": "Day" + }, + "active_users": { + "type": "integer", + "title": "Active Users" + }, + "actions": { + "type": "integer", + "title": "Actions" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "day", + "active_users", + "actions" + ], + "title": "AnalyticsDay" + }, + "AnalyticsEvent": { + "properties": { + "name": { + "type": "string", + "title": "Name" + }, + "description": { + "type": "string", + "title": "Description" + }, + "on": { + "type": "boolean", + "title": "On" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "name", + "description", + "on" + ], + "title": "AnalyticsEvent" + }, + "AnalyticsFeature": { + "properties": { + "feature": { + "type": "string", + "title": "Feature" + }, + "actions": { + "type": "integer", + "title": "Actions" + }, + "users": { + "type": "integer", + "title": "Users" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "feature", + "actions", + "users" + ], + "title": "AnalyticsFeature" + }, + "AnalyticsMetrics": { + "properties": { + "days": { + "type": "integer", + "title": "Days" + }, + "since": { + "type": "string", + "title": "Since" + }, + "until": { + "type": "string", + "title": "Until" + }, + "daily": { + "items": { + "$ref": "#/components/schemas/AnalyticsDay" + }, + "type": "array", + "title": "Daily" + }, + "dau": { + "type": "integer", + "title": "Dau" + }, + "wau": { + "type": "integer", + "title": "Wau" + }, + "mau": { + "type": "integer", + "title": "Mau" + }, + "stickiness": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "null" + } + ], + "title": "Stickiness" + }, + "accounts": { + "type": "integer", + "title": "Accounts" + }, + "active_users": { + "type": "integer", + "title": "Active Users" + }, + "new_users": { + "type": "integer", + "title": "New Users" + }, + "churn": { + "$ref": "#/components/schemas/AnalyticsChurn" + }, + "retention": { + "items": { + "$ref": "#/components/schemas/AnalyticsCohort" + }, + "type": "array", + "title": "Retention" + }, + "features": { + "items": { + "$ref": "#/components/schemas/AnalyticsFeature" + }, + "type": "array", + "title": "Features" + }, + "last_active_day": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Last Active Day" + }, + "days_since_active": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "title": "Days Since Active" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "days", + "since", + "until", + "daily", + "dau", + "wau", + "mau", + "accounts", + "active_users", + "new_users", + "churn", + "retention", + "features" + ], + "title": "AnalyticsMetrics", + "description": "Active people, churn, retention and feature use, worked out on this machine from the activity ledger and\nsign-ins (docs/analytics.md#the-numbers)." + }, + "AnalyticsStatus": { + "properties": { + "enabled": { + "type": "boolean", + "title": "Enabled" + }, + "configured": { + "type": "boolean", + "title": "Configured" + }, + "measurement_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Measurement Id" + }, + "client_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Client Id" + }, + "last_sent_at": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Last Sent At" + }, + "last_sent": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Last Sent", + "default": [] + }, + "last_error": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Last Error" + }, + "events": { + "items": { + "$ref": "#/components/schemas/AnalyticsEvent" + }, + "type": "array", + "title": "Events" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "enabled", + "configured", + "events" + ], + "title": "AnalyticsStatus", + "description": "Whether usage statistics go to Google Analytics, and how the last send went (docs/analytics.md)." + }, + "AnalyticsTestResult": { + "properties": { + "ok": { + "type": "boolean", + "title": "Ok" + }, + "error": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Error" + }, + "messages": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Messages", + "default": [] + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "ok" + ], + "title": "AnalyticsTestResult" + }, "AnswerCheck": { "properties": { "claims": { @@ -48150,6 +48615,44 @@ ], "title": "SetupAdmin" }, + "SetupAnalyticsView": { + "properties": { + "enabled": { + "type": "boolean", + "title": "Enabled" + }, + "measurement_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Measurement Id" + }, + "secret_set": { + "type": "boolean", + "title": "Secret Set" + }, + "locked": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Locked" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "enabled", + "secret_set", + "locked" + ], + "title": "SetupAnalyticsView" + }, "SetupFinish": { "properties": { "skipped": { @@ -48500,6 +49003,41 @@ } ], "title": "Endpoint" + }, + "analytics": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Analytics" + }, + "measurement_id": { + "anyOf": [ + { + "type": "string", + "maxLength": 40 + }, + { + "type": "null" + } + ], + "title": "Measurement Id" + }, + "api_secret": { + "anyOf": [ + { + "type": "string", + "maxLength": 200 + }, + { + "type": "null" + } + ], + "title": "Api Secret" } }, "additionalProperties": false, @@ -48529,13 +49067,17 @@ }, "type": "array", "title": "Locked" + }, + "analytics": { + "$ref": "#/components/schemas/SetupAnalyticsView" } }, "additionalProperties": true, "type": "object", "required": [ "enabled", - "locked" + "locked", + "analytics" ], "title": "SetupTelemetryView" }, From e0e76ee433321d951b64c92b8b64078232f718e8 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 10 Oct 2026 20:09:05 +0000 Subject: [PATCH 2/5] feat(analytics): Settings page with usage numbers and the wizard's Google Analytics opt-in Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH --- .../__tests__/setupWizard.test.tsx | 47 +++++- .../components/settings/bodies.tsx | 90 ++++++++++++ nextjs-frontend/components/settings/model.ts | 70 +++++++++ .../components/settings/settings-shell.tsx | 2 + .../components/settings/usage-analytics.tsx | 139 ++++++++++++++++++ .../components/setup/setup-wizard.tsx | 110 +++++++++++++- 6 files changed, 449 insertions(+), 9 deletions(-) create mode 100644 nextjs-frontend/components/settings/usage-analytics.tsx diff --git a/nextjs-frontend/__tests__/setupWizard.test.tsx b/nextjs-frontend/__tests__/setupWizard.test.tsx index 36b3bb9ff..4cc1d8b54 100644 --- a/nextjs-frontend/__tests__/setupWizard.test.tsx +++ b/nextjs-frontend/__tests__/setupWizard.test.tsx @@ -1,5 +1,5 @@ import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; -import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import { fireEvent, render, screen, waitFor, within } from "@testing-library/react"; import "@testing-library/jest-dom"; import type { ReactNode } from "react"; @@ -148,7 +148,13 @@ describe("SetupWizard", () => { await waitFor(() => expect(Setup.finish).toHaveBeenCalledWith(expect.objectContaining({ body: { skipped: false } })), ); - expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({ enabled: false, endpoint: null }); + expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({ + enabled: false, + endpoint: null, + analytics: false, + measurement_id: null, + api_secret: null, + }); await waitFor(() => expect(replace).toHaveBeenCalledWith("/")); }); @@ -159,7 +165,9 @@ describe("SetupWizard", () => { m(Setup.finish).mockImplementation(() => ok({ ok: true })); wrap(); fireEvent.click(await screen.findByRole("button", { name: /Telemetry/ })); - fireEvent.click(await screen.findByRole("radio", { name: "On" })); + fireEvent.click( + within(await screen.findByRole("radiogroup", { name: "Send telemetry" })).getByRole("radio", { name: "On" }), + ); const save = screen.getByRole("button", { name: "Save and finish" }); expect(save).toBeDisabled(); // on needs somewhere to send to fireEvent.change(screen.getByLabelText("OTLP endpoint"), { target: { value: "localhost:4318" } }); @@ -170,7 +178,38 @@ describe("SetupWizard", () => { expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({ enabled: false, endpoint: "http://localhost:4318" }); fireEvent.click(save); await waitFor(() => expect(Setup.finish).toHaveBeenCalled()); - expect(m(Setup.saveTelemetry).mock.calls[1][0].body).toEqual({ enabled: true, endpoint: "http://localhost:4318" }); + expect(m(Setup.saveTelemetry).mock.calls[1][0].body).toEqual({ + enabled: true, + endpoint: "http://localhost:4318", + analytics: false, + measurement_id: null, + api_secret: null, + }); + }); + + it("opts in to Google Analytics with a measurement id and secret", async () => { + m(Setup.getSetup).mockImplementation(() => ok(VIEW({ namespace: { existing: ["media"], locked: false } }))); + m(Setup.saveTelemetry).mockImplementation(() => ok({ ok: true })); + m(Setup.finish).mockImplementation(() => ok({ ok: true })); + wrap(); + fireEvent.click(await screen.findByRole("button", { name: /Telemetry/ })); + const group = await screen.findByRole("radiogroup", { name: "Share usage statistics with Google Analytics" }); + fireEvent.click(within(group).getByRole("radio", { name: "On" })); + const save = screen.getByRole("button", { name: "Save and finish" }); + expect(save).toBeDisabled(); // on needs an id and a secret + fireEvent.change(screen.getByLabelText("Measurement id"), { target: { value: "UA-1" } }); + fireEvent.change(screen.getByLabelText("Measurement Protocol API secret"), { target: { value: "s3cret" } }); + expect(save).toBeDisabled(); + fireEvent.change(screen.getByLabelText("Measurement id"), { target: { value: "G-TEST1234" } }); + fireEvent.click(save); + await waitFor(() => expect(Setup.finish).toHaveBeenCalled()); + expect(m(Setup.saveTelemetry).mock.calls[0][0].body).toEqual({ + enabled: false, + endpoint: null, + analytics: true, + measurement_id: "G-TEST1234", + api_secret: "s3cret", + }); }); it("lets apps sign in for a set time, or turns it off", async () => { diff --git a/nextjs-frontend/components/settings/bodies.tsx b/nextjs-frontend/components/settings/bodies.tsx index d7d16bf49..6e21e96ce 100644 --- a/nextjs-frontend/components/settings/bodies.tsx +++ b/nextjs-frontend/components/settings/bodies.tsx @@ -14,6 +14,7 @@ import { RIGHTS } from "@/components/iiif/rights"; import { hubText } from "@/components/sensors/sensor-model"; import { SecretSetting, SettingField, ZoneBar, type FieldState } from "@/components/settings/fields"; import { SignInProviders } from "@/components/settings/sign-in-providers"; +import { UsageNumbers } from "@/components/settings/usage-analytics"; import { AddConnection } from "@/components/storage/add-connection"; import { NOT_STORAGE, fileStoreTry } from "@/components/storage/file-store"; import { @@ -379,6 +380,8 @@ export function SectionBody({ ctx }: { ctx: BodyCtx }) { return ; case "telemetry": return ; + case "analytics": + return ; case "fedora": return ; case "sensors": @@ -610,6 +613,93 @@ function TelemetryBody({ ctx }: { ctx: BodyCtx }) { ); } +function AnalyticsBody({ ctx }: { ctx: BodyCtx }) { + const client = useApiClient(); + const a = ctx.view.analytics; + const saved = a?.values ?? {}; + const secret = (saved.api_secret ?? {}) as { set?: boolean }; + const apiSecret = ctx.state("analytics.api_secret"); + const locked = a?.locked ?? []; + const status = useQuery({ + queryKey: ["analytics-status", a?.updated_at ?? null], + queryFn: () => data(Admin.analyticsStatus({ client })), + refetchInterval: 60_000, + }); + const test = useMutation({ mutationFn: () => data(Admin.testAnalytics({ client })) }); + const s = status.data; + const on = Boolean(saved.enabled); + const id = (saved.measurement_id as string | null | undefined) ?? null; + return ( + <> + + {on && s?.configured ? ( + + {s.last_error + ? s.last_error + : s.last_sent_at + ? `Last sent ${new Date(s.last_sent_at).toLocaleString()}: ${(s.last_sent ?? []).join(", ")}` + : "Nothing sent yet; the first events go with the next daily check."} + + ) : on ? ( + + Set the measurement id and the API secret below; until then nothing is sent. + + ) : ( + + Turn it on here, in the setup wizard or with LENS_ANALYTICS=on in .env. + Events go only to the GA4 property whose id and secret you set. + + )} + +
+ + +
+ apiSecret.onChange(x)} + /> + + +

+ What is sent, once a day: a random id for this install{s?.client_id ? ` (${s.client_id})` : ""}, the Lens + version and platform, and counts. Never names, emails, account ids, titles, file names, namespaces, prompts or + any text. In GA, the Retention report and a cohort exploration on lens_daily_active give churn across your + installs. +

+
+ + + {ctx.dirty + ? "Uses the saved settings, not your unsaved changes" + : "GA’s validation server checks one lens_test event without recording it"} + +
+ {test.data && + (test.data.ok ? ( + + ) : ( + + {test.data.error ?? (test.data.messages ?? []).join(" · ")} + + ))} + {test.isError && {test.error.message}} + + ); +} + function EncryptionBody({ ctx }: { ctx: BodyCtx }) { const client = useApiClient(); const progress = useQuery({ diff --git a/nextjs-frontend/components/settings/model.ts b/nextjs-frontend/components/settings/model.ts index 6c59496a1..7c17f56b0 100644 --- a/nextjs-frontend/components/settings/model.ts +++ b/nextjs-frontend/components/settings/model.ts @@ -74,6 +74,7 @@ export type SectionId = | "mail" | "bridge" | "telemetry" + | "analytics" | "fedora" | "sensors" | "uploads" @@ -239,6 +240,13 @@ export const SECTIONS: SectionSpec[] = [ description: "Off unless you turn it on. Traces and metrics from the server and its workers, sent only to an OpenTelemetry collector you run or choose; nothing goes anywhere else.", }, + { + id: "analytics", + label: "Usage analytics", + backend: ["analytics"], + description: + "Active people, churn, retention and feature use, worked out on this server. Optionally, counted events for this install sent to your own Google Analytics property; off unless you turn it on.", + }, { id: "fedora", label: "Fedora repository", @@ -1866,6 +1874,59 @@ export const FIELDS: FieldSpec[] = [ max: 20, hint: "A failed send waits 30 seconds, then four times longer each time, up to 6 hours", }, + // Usage analytics (Google Analytics opt-in) + { + section: "analytics", + key: "enabled", + label: "Send usage events to Google Analytics", + kind: "switch", + hint: "Off by default; needs the measurement id and API secret below", + }, + { + section: "analytics", + key: "measurement_id", + label: "Measurement id", + kind: "text", + nullable: true, + mono: true, + placeholder: "G-ABC123XYZ", + hint: "Your GA4 property’s web data stream id", + }, + { section: "analytics", key: "api_secret", label: "Measurement Protocol API secret", kind: "secret" }, + { + section: "analytics", + key: "region", + label: "Collection", + kind: "select", + options: [ + { value: "global", label: "Global" }, + { value: "eu", label: "EU only (region1.google-analytics.com)" }, + ], + }, + { + section: "analytics", + key: "events", + label: "Events sent", + kind: "checks", + options: [ + { value: "lens_install", label: "Install", hint: "Analytics first runs here" }, + { value: "lens_setup_done", label: "Setup done", hint: "The setup wizard was finished" }, + { value: "lens_daily_active", label: "Daily active", hint: "People and actions the day before" }, + { value: "lens_feature_used", label: "Feature used", hint: "Each feature used the day before, with counts" }, + { value: "lens_upgrade", label: "Upgrade", hint: "From and to which version" }, + { value: "lens_inactive", label: "Inactive", hint: "Nobody used Lens for the days below" }, + { value: "lens_return", label: "Return", hint: "Used again after going inactive" }, + ], + }, + { + section: "analytics", + key: "inactive_days", + label: "Inactive after (days)", + kind: "int", + min: 1, + max: 365, + hint: "Days without anyone using Lens before it counts as churned", + }, // Telemetry (opt-in) { section: "telemetry", @@ -2397,6 +2458,9 @@ export function crossErrors(values: Record): Record = { mail: Mail, bridge: MessagesSquare, telemetry: Activity, + analytics: BarChart3, fedora: Archive, sensors: RadioTower, uploads: Upload, diff --git a/nextjs-frontend/components/settings/usage-analytics.tsx b/nextjs-frontend/components/settings/usage-analytics.tsx new file mode 100644 index 000000000..deda8866c --- /dev/null +++ b/nextjs-frontend/components/settings/usage-analytics.tsx @@ -0,0 +1,139 @@ +"use client"; + +import { useQuery } from "@tanstack/react-query"; + +import { Admin } from "@/app/openapi-client"; +import type { AnalyticsMetrics } from "@/app/openapi-client/types.gen"; +import { Banner } from "@/components/ui/banner"; +import { Skeleton } from "@/components/ui/states"; +import { data, useApiClient } from "@/lib/api/browser"; + +function pct(v: number | null | undefined) { + return v == null ? "—" : `${Math.round(v * 1000) / 10}%`; +} + +function Tile({ label, value, hint }: { label: string; value: string | number; hint?: string }) { + return ( +
+ {label} + {value} + {hint && {hint}} +
+ ); +} + +/** Daily active people as a row of bars, one per day. */ +function DailyBars({ days }: { days: AnalyticsMetrics["daily"] }) { + const top = Math.max(1, ...days.map((d) => d.active_users)); + return ( +
+ Active people each day +
+ {days.map((d) => ( + + ))} +
+
+ {days[0]?.day} + {days[days.length - 1]?.day} +
+
+ ); +} + +/** Active people, churn, retention and feature use, worked out on this server (docs/analytics.md). */ +export function UsageNumbers({ days = 30 }: { days?: number }) { + const client = useApiClient(); + const q = useQuery({ + queryKey: ["usage-analytics", days], + queryFn: () => data(Admin.usageMetrics({ client, query: { days } })), + }); + if (q.isError) return {q.error.message}; + const m = q.data; + if (!m) return ; + const c = m.churn; + const weeks = Math.max(0, ...m.retention.map((r) => r.retention.length)); + return ( +
+

+ The last {m.days} days ({m.since} to {m.until}), from the activity ledger and sign-ins on this server. Nothing + here is sent anywhere. +

+
+ + + + + + + + +
+ + {m.retention.length > 0 && ( +
+ + + + + + + {Array.from({ length: weeks }, (_, i) => ( + + ))} + + + + {m.retention.map((r) => ( + + + + {Array.from({ length: weeks }, (_, i) => ( + + ))} + + ))} + +
Retention by sign-up week
Week ofPeople + W{i} +
{r.week}{r.users} + {i < r.retention.length ? pct(r.retention[i]) : ""} +
+
+ )} + {m.features.length > 0 && ( +
+ Features used +
    + {m.features.slice(0, 16).map((f) => ( +
  • + {f.feature} {f.actions} +
  • + ))} +
+
+ )} +
+ ); +} diff --git a/nextjs-frontend/components/setup/setup-wizard.tsx b/nextjs-frontend/components/setup/setup-wizard.tsx index c94939a3b..d82d21639 100644 --- a/nextjs-frontend/components/setup/setup-wizard.tsx +++ b/nextjs-frontend/components/setup/setup-wizard.tsx @@ -42,6 +42,12 @@ const LLM_ENV: Record = { api_key: "LENS_LLM_API_KEY", }; const TELEMETRY_ENV: Record = { enabled: "LENS_TELEMETRY", endpoint: "LENS_TELEMETRY_ENDPOINT" }; +const ANALYTICS_ENV: Record = { + enabled: "LENS_ANALYTICS", + measurement_id: "LENS_GA_MEASUREMENT_ID", + api_secret: "LENS_GA_API_SECRET", +}; +const GA_ID_RX = /^G-[A-Z0-9]{4,20}$/i; const NS = /^[a-z0-9][a-z0-9_-]{0,40}$/; const URL_RX = /^https?:\/\/[^\s]+$/; @@ -882,7 +888,18 @@ function TelemetryStep({ const [endpoint, setEndpoint] = useState(t.endpoint ?? ""); const [tested, setTested] = useState(null); useEffect(() => setTested(null), [endpoint]); - const body = () => ({ enabled: on === "on", endpoint: endpoint.trim() || null }); + const ga = t.analytics; + const gaLocked = new Set(ga.locked); + const [share, setShare] = useState(ga.enabled ? "on" : "off"); + const [gaId, setGaId] = useState(ga.measurement_id ?? ""); + const [gaSecret, setGaSecret] = useState(""); + const body = () => ({ + enabled: on === "on", + endpoint: endpoint.trim() || null, + analytics: share === "on", + measurement_id: gaId.trim() || null, + api_secret: gaSecret.trim() || null, + }); const save = useMutation({ mutationFn: () => data(Setup.saveTelemetry({ client, body: body() })) }); const test = useMutation({ mutationFn: async () => { @@ -895,6 +912,12 @@ function TelemetryStep({ const wantsEndpoint = on === "on" && !locked.has("endpoint"); const badEndpoint = Boolean(endpoint.trim()) && !URL_RX.test(endpoint.trim()); const missing = wantsEndpoint && !endpoint.trim(); + const badGaId = Boolean(gaId.trim()) && !GA_ID_RX.test(gaId.trim()); + const gaMissing = + share === "on" && + !gaLocked.has("enabled") && + ((!gaId.trim() && !gaLocked.has("measurement_id")) || + (!gaSecret.trim() && !ga.secret_set && !gaLocked.has("api_secret"))); return ( <> @@ -960,6 +983,85 @@ function TelemetryStep({

)} + + ) : ( + "Counted events for this install in your own GA4 property: active people, features used, upgrades. Off by default; Settings → Usage analytics." + ) + } + > + {() => ( + undefined : setShare} + options={[ + { value: "off", label: "Off" }, + { value: "on", label: "On" }, + ]} + className="w-full max-w-[220px]" + /> + )} + + {share === "on" && ( + <> + : "Like G-ABC123XYZ" + } + error={badGaId ? "A GA4 measurement id starts with G-" : undefined} + > + {({ id, describedBy, invalid }) => ( + setGaId(e.target.value)} + disabled={gaLocked.has("measurement_id")} + autoComplete="off" + spellCheck={false} + /> + )} + + + ) : ga.secret_set ? ( + "Saved; leave empty to keep it" + ) : ( + "In GA: Admin → Data streams → your stream → Measurement Protocol API secrets" + ) + } + > + {({ id, describedBy, invalid }) => ( + setGaSecret(e.target.value)} + disabled={gaLocked.has("api_secret")} + autoComplete="off" + spellCheck={false} + /> + )} + +

+ Sent once a day: a random id for this install, the version and counts. Never names, emails, titles, files or + text. +

+ + )} {tested && (tested.ok ? ( The collector took a test span in {tested.ms} ms. @@ -985,10 +1087,8 @@ function TelemetryStep({ From f2f3ce1cfab85f451631dd9231c2d4adbedb6ca1 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 10 Oct 2026 20:09:51 +0000 Subject: [PATCH 3/5] docs(analytics): the Settings page is built Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH --- docs/analytics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analytics.md b/docs/analytics.md index 18b5714a1..715ac79a9 100644 --- a/docs/analytics.md +++ b/docs/analytics.md @@ -4,7 +4,7 @@ Lens works out how it is being used (active people, churn, retention, which feat and can also send a few counted events to a **Google Analytics 4** property you own, to see the same across every install you run. -Status: **built** (backend and API). The Settings page and the wizard toggle are **in progress**. +Status: **built**: the numbers, Settings → Usage analytics, the wizard toggle and the Google Analytics events. ## The numbers @@ -85,6 +85,6 @@ telemetry does. ## Refine later -- A Settings page with charts of the numbers, and the wizard toggle (in progress). +- Choose the period in Settings (it shows the last 30 days). - Per-namespace numbers. - Optional per-person pseudonymous ids in GA, for multi-person installs. From bdf598f96873d2355c5abe63870622969db8d1a3 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 10 Oct 2026 20:15:48 +0000 Subject: [PATCH 4/5] feat(analytics): built-in Analytics page, with each day's events kept locally The daily events are counted and kept on the server whether or not Google Analytics is on; GA is an optional export of them. A new Analytics page (admins) shows active people, churn, retention, feature use and the event log for 7 to 180 days. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH --- docs/analytics.md | 20 ++- fastapi_backend/app/api/v1/routes/admin.py | 8 ++ fastapi_backend/app/domain/analytics.py | 99 ++++++++++---- fastapi_backend/app/schemas/admin.py | 12 ++ fastapi_backend/tests/api/test_analytics.py | 26 +++- .../__tests__/analyticsPage.test.tsx | 92 +++++++++++++ nextjs-frontend/__tests__/navigation.test.ts | 2 + nextjs-frontend/app/(app)/analytics/page.tsx | 10 ++ nextjs-frontend/app/openapi-client/index.ts | 6 + nextjs-frontend/app/openapi-client/sdk.gen.ts | 18 +++ .../app/openapi-client/types.gen.ts | 63 +++++++++ .../components/analytics/analytics-page.tsx | 126 ++++++++++++++++++ .../usage-numbers.tsx} | 0 .../components/app-shell/nav-config.ts | 2 + .../components/settings/bodies.tsx | 10 +- nextjs-frontend/openapi.json | 92 +++++++++++++ 16 files changed, 549 insertions(+), 37 deletions(-) create mode 100644 nextjs-frontend/__tests__/analyticsPage.test.tsx create mode 100644 nextjs-frontend/app/(app)/analytics/page.tsx create mode 100644 nextjs-frontend/components/analytics/analytics-page.tsx rename nextjs-frontend/components/{settings/usage-analytics.tsx => analytics/usage-numbers.tsx} (100%) diff --git a/docs/analytics.md b/docs/analytics.md index 715ac79a9..0fd2f0a59 100644 --- a/docs/analytics.md +++ b/docs/analytics.md @@ -1,10 +1,12 @@ # Usage analytics -Lens works out how it is being used (active people, churn, retention, which features get used) on your own machine, -and can also send a few counted events to a **Google Analytics 4** property you own, to see the same across every -install you run. +Lens has built-in analytics: the **Analytics** page (admins) shows how Lens is being used, with active people, churn, +retention, which features get used and the events it counts each day. All of it is worked out and kept on your own +machine. Optionally, the same daily events also go to a **Google Analytics 4** property you own, to see them across +every install you run. -Status: **built**: the numbers, Settings → Usage analytics, the wizard toggle and the Google Analytics events. +Status: **built**: the Analytics page, the daily events, the Google Analytics export, its settings and the wizard +toggle. ## The numbers @@ -26,6 +28,13 @@ collected for them and nothing leaves the machine. A person is active on a day when they changed something through the API or signed in. With `activity.reads` on, reads count too. The ledger keeps `activity.keep_days` (365), which is as far back as the numbers can go. +## Events + +Once a day, a worker (or `lens watch`) counts the day before and keeps the result in the `usage_event` table, with +Google Analytics on or off. The Analytics page lists them, and so does `GET /api/v1/admin/analytics/events`. Each one +shows whether it was sent to Google Analytics, is waiting to be sent, or is kept here only. Events are kept for +`activity.keep_days`. + ## Google Analytics **Off unless you turn it on.** It needs a GA4 property of your own: its **measurement id** (`G-…`) and a @@ -37,7 +46,8 @@ reads count too. The ledger keeps `activity.keep_days` (365), which is as far ba - **.env**: `LENS_ANALYTICS=on`, `LENS_GA_MEASUREMENT_ID=G-…`, `LENS_GA_API_SECRET=…`. What .env sets wins over Settings, which show those fields locked. -Turned on, one process (a worker, or `lens watch`) sends once a day, for the install as a whole: +Turned on, the daily events are also sent, for the install as a whole. A send that fails is tried again 10 minutes +later. Events kept while Google Analytics was off are never sent later. These are the events: | Event | When | Parameters | | --- | --- | --- | diff --git a/fastapi_backend/app/api/v1/routes/admin.py b/fastapi_backend/app/api/v1/routes/admin.py index 2f203aa4c..aecdc9d6c 100644 --- a/fastapi_backend/app/api/v1/routes/admin.py +++ b/fastapi_backend/app/api/v1/routes/admin.py @@ -59,6 +59,7 @@ TelemetryStatus, TelemetryTestResult, TunnelStatus, + UsageEvent, ) from app.schemas.auth import AccountToken from app.schemas.common import Ok @@ -303,6 +304,13 @@ def usage_metrics(user: AdminReader, db: Db, days: int = Query(30, ge=1, le=180) return AnalyticsMetrics.model_validate(analytics.metrics(db, days)) +@router.get("/admin/analytics/events") +def usage_events(user: AdminReader, db: Db, limit: int = Query(200, ge=1, le=1000)) -> list[UsageEvent]: + """The events Lens counted each day (install, daily active, feature used, upgrade, inactive...), newest first, and + whether each went to Google Analytics. Kept here whether or not Google Analytics is on.""" + return [UsageEvent.model_validate(r) for r in analytics.recent_events(db, limit)] + + @router.get("/audit") def list_audit(user: AdminReader, db: Db, limit: int = Query(200, ge=1, le=1000)) -> list[AuditEntry]: """Who changed what, newest first.""" diff --git a/fastapi_backend/app/domain/analytics.py b/fastapi_backend/app/domain/analytics.py index 66afd15d1..8f0bff801 100644 --- a/fastapi_backend/app/domain/analytics.py +++ b/fastapi_backend/app/domain/analytics.py @@ -5,10 +5,12 @@ (activity.py) and sign-ins in the audit log. Nothing new is collected for them and nothing leaves the machine: daily, weekly and monthly active people, stickiness, churn and retention by sign-up week, and which features get used. +**The events** in EVENTS are worked out once a day and kept here (`usage_event`), so the Analytics page shows them +with Google Analytics off. + **Google Analytics** is off by default (analytics.enabled), and needs a GA4 measurement id and a Measurement Protocol API secret, set in Settings → Usage analytics, the setup wizard, or .env (LENS_ANALYTICS, LENS_GA_MEASUREMENT_ID, -LENS_GA_API_SECRET). Turned on, one process sends once a day the events in EVENTS that analytics.events names, for this -install as a whole: the install's random id (made here, never a person, host or address), the Lens version and +LENS_GA_API_SECRET). Turned on, the kept events that analytics.events names also go to GA, for this install as a whole: the install's random id (made here, never a person, host or address), the Lens version and platform, and counts. Never a name, email, title, file name, namespace, prompt or text. Churn across installs is then GA's retention report over lens_daily_active; lens_inactive marks an install nobody has used for analytics.inactive_days, which is how a self-hosted install that quietly stopped shows up. @@ -31,6 +33,7 @@ ROW = R("app_service", "analytics") HOSTS = {"global": "www.google-analytics.com", "eu": "region1.google-analytics.com"} # tests point it at a fake BATCH = 25 # GA4 takes up to 25 events a request +RETRY_SECONDS = 600 MEASUREMENT_RX = re.compile(r"^G-[A-Z0-9]{4,20}$") # The events Lens can send, and what each carries. Names are prefixed so none clashes with GA's reserved ones. @@ -271,35 +274,59 @@ def _claim(db, day): def tick(db, cfg, now=None): - """Once a day, while analytics is on and set up: send what happened since the last time (docs/analytics.md).""" - if not configured(cfg): - return [] + """Once a day: work out the day before's events and keep them here (`usage_event`, docs/analytics.md#events), + whether or not Google Analytics is on; with it on and set up, send them too. Every tick sends what is still unsent + (a send that failed is tried again a few minutes later). Returns the names of the events made this time.""" from . import __version__ now = now or dt.datetime.now(dt.timezone.utc) today = now.date() - if not _claim(db, today.isoformat()): - return [] + made = [] + if _claim(db, today.isoformat()): + events, marks = _today(db, cfg, today, __version__) + a = _section(cfg) + wanted = configured(cfg) and set(a.get("events") or EVENTS) + yesterday = (today - dt.timedelta(days=1)).isoformat() + at = store.now() + for n, (name, params) in enumerate(events): + db.q( + "CREATE usage_event CONTENT $d", + d={ + "at": at, + "n": n, + "day": yesterday, + "name": name, + "params": params, + "sent": False if wanted and name in wanted else None, + }, + ) + _put(db, **marks) + keep = int(((cfg or {}).get("activity") or {}).get("keep_days") or 365) + db.q("DELETE usage_event WHERE day < $d", d=(today - dt.timedelta(days=keep)).isoformat()) + made = [n for n, _ in events] + if configured(cfg): + flush_unsent(db, cfg, now) + return made + + +def _today(db, cfg, today, version): + """The events for the day before `today`, and what to remember so none is made twice.""" st, events = _state(db), [] if not st.get("installed"): - events.append(("lens_install", {"version": __version__, "platform": _platform()})) + events.append(("lens_install", {"version": version, "platform": _platform()})) wiz = db.one("SELECT done_at, skipped FROM $r", r=R("setup", "wizard")) or {} if wiz.get("done_at") and not st.get("setup_done"): events.append(("lens_setup_done", {"skipped": 1 if wiz.get("skipped") else 0})) - if st.get("version") and st["version"] != __version__: - events.append(("lens_upgrade", {"from_version": st["version"], "to_version": __version__})) + if st.get("version") and st["version"] != version: + events.append(("lens_upgrade", {"from_version": st["version"], "to_version": version})) yesterday = today - dt.timedelta(days=1) seen = _days(db, yesterday) users = {w for (w, d) in seen if d == yesterday.isoformat()} inactive_days = int(_section(cfg).get("inactive_days") or 30) - marks = {"installed": True, "setup_done": bool(wiz.get("done_at")), "version": __version__} + marks = {"installed": True, "setup_done": bool(wiz.get("done_at")), "version": version} if users: - events.append( - ( - "lens_daily_active", - {"active_users": len(users), "actions": sum(n for (_, d), n in seen.items() if d == yesterday.isoformat())}, - ) - ) + actions = sum(n for (_, d), n in seen.items() if d == yesterday.isoformat()) + events.append(("lens_daily_active", {"active_users": len(users), "actions": actions})) start = dt.datetime.combine(yesterday, dt.time(), dt.timezone.utc) for f in _features(db, start, start + dt.timedelta(days=1))[:20]: events.append(("lens_feature_used", f)) @@ -314,13 +341,37 @@ def tick(db, cfg, now=None): if since and (today - since).days >= inactive_days: events.append(("lens_inactive", {"days": (today - since).days})) marks["inactive_since"] = since.isoformat() - ok, error, _ = send(db, cfg, events, day=yesterday) - if ok: - _put(db, **marks) - else: - log.info("analytics: %s", error) - db.q("UPDATE $r SET day = NONE", r=ROW) # try again on the next tick - return [n for n, _ in events] + return events, marks + + +def flush_unsent(db, cfg, now=None): + """Send the kept events still waiting for Google Analytics, a day at a time; after a failure, wait RETRY_SECONDS.""" + now = now or dt.datetime.now(dt.timezone.utc) + st = _state(db) + if st.get("last_error_at") and st.get("last_error"): + try: + if (now - dt.datetime.fromisoformat(st["last_error_at"])).total_seconds() < RETRY_SECONDS: + return False + except ValueError: + pass + rows = db.rows("SELECT record::id(id) AS id, at, n, day, name, params FROM usage_event WHERE sent = false ORDER BY at, n LIMIT 500") + for day in sorted({r["day"] for r in rows}): + batch = [r for r in rows if r["day"] == day] + ok, error, _ = send(db, cfg, [(r["name"], r["params"] or {}) for r in batch], day=dt.date.fromisoformat(day)) + if not ok: + log.info("analytics: %s", error) + _put(db, last_error_at=now.isoformat()) + return False + db.q("UPDATE usage_event SET sent = true, sent_at = $t WHERE record::id(id) IN $ids", t=store.now(), ids=[r["id"] for r in batch]) + return True + + +def recent_events(db, limit=200): + """The kept events, newest first: what Lens counted each day, and whether each went to Google Analytics.""" + return db.rows( + "SELECT at, n, day, name, params, sent FROM usage_event ORDER BY at DESC, n LIMIT $n", + n=max(1, min(int(limit), 1000)), + ) def test(db, cfg): diff --git a/fastapi_backend/app/schemas/admin.py b/fastapi_backend/app/schemas/admin.py index bba0f0bc6..82f4f10e5 100644 --- a/fastapi_backend/app/schemas/admin.py +++ b/fastapi_backend/app/schemas/admin.py @@ -308,6 +308,18 @@ class AnalyticsMetrics(ResponseModel): days_since_active: int | None = None +class UsageEvent(ResponseModel): + """An event Lens counted for a day (docs/analytics.md#events).""" + + at: str + # the day it counts + day: str + name: str + params: dict[str, Any] = {} + # true: sent to Google Analytics; false: waiting to be sent; null: kept here only (GA off, or the event left out) + sent: bool | None = None + + class FileStoreTry(RequestModel): """Where to try keeping files, before saving it (Settings → Storage); nothing given tries what is saved.""" diff --git a/fastapi_backend/tests/api/test_analytics.py b/fastapi_backend/tests/api/test_analytics.py index b3e50d3c3..e09264bcb 100644 --- a/fastapi_backend/tests/api/test_analytics.py +++ b/fastapi_backend/tests/api/test_analytics.py @@ -91,15 +91,20 @@ def d(n): assert week and week[-1]["retention"][0] == 1.0 # 3 was active in its sign-up week -def test_off_by_default_sends_nothing(db, cfg, ga): +def test_events_kept_locally_with_ga_off(db, cfg, ga): act(db, 1, (NOW.date() - dt.timedelta(days=1)).isoformat()) assert cfg["analytics"]["enabled"] is False - assert analytics.tick(db, cfg, NOW) == [] - routines.run_due(db, cfg, log=None, now=NOW) + assert analytics.tick(db, cfg, NOW) == ["lens_install", "lens_daily_active", "lens_feature_used"] + routines.run_due(db, cfg, log=None, now=NOW) # once a day assert ga == [] - # on without a secret isn't set up either + kept = analytics.recent_events(db) + assert sorted(e["name"] for e in kept) == ["lens_daily_active", "lens_feature_used", "lens_install"] + assert all(e["sent"] is None and e["day"] == "2026-10-09" for e in kept) + # on without a secret isn't set up either, and turning it on later doesn't send what was kept while off cfg["analytics"].update(enabled=True, measurement_id="G-TEST1234") - assert analytics.tick(db, cfg, NOW) == [] + analytics.tick(db, cfg, NOW + dt.timedelta(days=1)) + on(cfg) + analytics.tick(db, cfg, NOW + dt.timedelta(days=1, hours=1)) assert ga == [] @@ -151,8 +156,14 @@ def test_events_can_be_left_out_and_failures_retry(db, cfg, ga): assert names(ga) == ["lens_install"] assert "500" in analytics.status(db, cfg)["last_error"] ga.state["status"] = 204 - analytics.tick(db, cfg, NOW + dt.timedelta(minutes=5)) # the failed day is tried again + analytics.tick(db, cfg, NOW + dt.timedelta(minutes=5)) # waits a while after a failure + assert names(ga) == ["lens_install"] + analytics.tick(db, cfg, NOW + dt.timedelta(minutes=11)) # then tries again assert names(ga) == ["lens_install", "lens_install"] + analytics.tick(db, cfg, NOW + dt.timedelta(minutes=20)) # and sends each once + assert len(ga) == 2 + kept = {e["name"]: e["sent"] for e in analytics.recent_events(db)} + assert kept == {"lens_install": True, "lens_daily_active": None, "lens_feature_used": None} st = analytics.status(db, cfg) assert st["last_error"] is None and st["last_sent"] == ["lens_install"] @@ -180,6 +191,9 @@ def put(body): s = client.get("/api/v1/settings/analytics/status", headers=h).json() assert s["enabled"] is False and s["configured"] is False and len(s["events"]) == len(analytics.EVENTS) assert all(e["on"] for e in s["events"]) + analytics.tick(db, client.app.state.settings.current()) + ev = client.get("/api/v1/admin/analytics/events", headers=h).json() + assert [e["name"] for e in ev] == ["lens_install"] and ev[0]["sent"] is None m = client.get("/api/v1/admin/analytics?days=7", headers=h).json() assert m["days"] == 7 and m["accounts"] == 1 and m["dau"] == 1 # the sign-in above make_user(db, "ed@x.io", "ed password 12") diff --git a/nextjs-frontend/__tests__/analyticsPage.test.tsx b/nextjs-frontend/__tests__/analyticsPage.test.tsx new file mode 100644 index 000000000..57aa3fb93 --- /dev/null +++ b/nextjs-frontend/__tests__/analyticsPage.test.tsx @@ -0,0 +1,92 @@ +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import "@testing-library/jest-dom"; +import type { ReactNode } from "react"; + +import { Admin } from "@/app/openapi-client"; +import { AnalyticsPage } from "@/components/analytics/analytics-page"; +import { TooltipProvider } from "@/components/ui/tooltip"; + +jest.mock("@/app/openapi-client", () => ({ + Admin: { usageMetrics: jest.fn(), usageEvents: jest.fn(), analyticsStatus: jest.fn() }, +})); +jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) })); +const archive = { admin: true }; +jest.mock("@/lib/hooks/session", () => ({ useArchive: () => archive })); + +const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } }); +const m = (f: unknown) => f as jest.Mock; + +const METRICS = (days: number) => ({ + days, + since: "2026-10-04", + until: "2026-10-10", + daily: [ + { day: "2026-10-09", active_users: 1, actions: 3 }, + { day: "2026-10-10", active_users: 2, actions: 2 }, + ], + dau: 2, + wau: 2, + mau: 3, + stickiness: 0.1429, + accounts: 3, + active_users: 2, + new_users: 1, + churn: { previous_active: 2, retained: 1, churned: 1, rate: 0.5, returning: 1 }, + retention: [{ week: "2026-10-05", users: 1, retention: [1] }], + features: [{ feature: "chats", actions: 2, users: 1 }], + last_active_day: "2026-10-10", + days_since_active: 0, +}); + +function wrap(ui: ReactNode) { + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + return render( + + {ui} + , + ); +} + +describe("AnalyticsPage", () => { + beforeEach(() => { + archive.admin = true; + m(Admin.usageMetrics).mockImplementation(({ query }: { query: { days: number } }) => ok(METRICS(query.days))); + m(Admin.usageEvents).mockImplementation(() => + ok([ + { + at: "2026-10-10T00:05:00+00:00", + day: "2026-10-09", + name: "lens_daily_active", + params: { active_users: 1 }, + sent: null, + }, + ]), + ); + m(Admin.analyticsStatus).mockImplementation(() => ok({ enabled: false, configured: false, events: [] })); + }); + + it("shows churn, retention and the kept events with Google Analytics off", async () => { + wrap(); + expect(await screen.findByText("50%")).toBeInTheDocument(); // churn rate + expect(screen.getByText("1 of 2 didn’t come back")).toBeInTheDocument(); + expect(screen.getByText("Retention by sign-up week")).toBeInTheDocument(); + expect(await screen.findByText("Daily active")).toBeInTheDocument(); + expect(screen.getByText("kept here only")).toBeInTheDocument(); + expect(screen.getByRole("link", { name: "Settings → Usage analytics" })).toHaveAttribute( + "href", + "/settings/analytics", + ); + fireEvent.click(screen.getByRole("radio", { name: "90 days" })); + await waitFor(() => + expect(Admin.usageMetrics).toHaveBeenCalledWith(expect.objectContaining({ query: { days: 90 } })), + ); + }); + + it("is for admins", () => { + archive.admin = false; + wrap(); + expect(screen.getByText("Analytics are for admins")).toBeInTheDocument(); + expect(Admin.usageMetrics).not.toHaveBeenCalled(); + }); +}); diff --git a/nextjs-frontend/__tests__/navigation.test.ts b/nextjs-frontend/__tests__/navigation.test.ts index 7f00825e1..02e8f0a66 100644 --- a/nextjs-frontend/__tests__/navigation.test.ts +++ b/nextjs-frontend/__tests__/navigation.test.ts @@ -8,6 +8,8 @@ describe("navigation", () => { expect(labels(true)).toEqual(expect.arrayContaining(["Settings", "Admin"])); expect(labels(false)).toContain("Settings"); expect(labels(false)).not.toContain("Admin"); + expect(labels(true)).toContain("Analytics"); + expect(labels(false)).not.toContain("Analytics"); expect(labels(true)).not.toContain("Speakers"); // Admins see every source as a sensor; members keep Sources, the watched folders feeding their namespaces. expect(labels(true)).toContain("Sensors"); diff --git a/nextjs-frontend/app/(app)/analytics/page.tsx b/nextjs-frontend/app/(app)/analytics/page.tsx new file mode 100644 index 000000000..38b65eb2f --- /dev/null +++ b/nextjs-frontend/app/(app)/analytics/page.tsx @@ -0,0 +1,10 @@ +import type { Metadata } from "next"; + +import { AnalyticsPage } from "@/components/analytics/analytics-page"; + +export const metadata: Metadata = { title: "Analytics" }; + +/** Built-in usage analytics: active people, churn, retention, features and the daily events (docs/analytics.md). */ +export default function Page() { + return ; +} diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index a50ac936a..7165c1257 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -2876,6 +2876,12 @@ export type { UploadLimitsResponse, UploadLimitsResponses, UploadStart, + UsageEvent, + UsageEventsData, + UsageEventsError, + UsageEventsErrors, + UsageEventsResponse, + UsageEventsResponses, UsageMetricsData, UsageMetricsError, UsageMetricsErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index bacf4113b..65fb5b0ed 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -1409,6 +1409,9 @@ import type { UpdateWorkflowResponses, UploadLimitsData, UploadLimitsResponses, + UsageEventsData, + UsageEventsErrors, + UsageEventsResponses, UsageMetricsData, UsageMetricsErrors, UsageMetricsResponses, @@ -2671,6 +2674,21 @@ export class Admin { }); } + /** + * Usage Events + * + * The events Lens counted each day (install, daily active, feature used, upgrade, inactive...), newest first, and + * whether each went to Google Analytics. Kept here whether or not Google Analytics is on. + */ + public static usageEvents( + options?: Options, + ): RequestResult { + return (options?.client ?? client).get({ + url: "/api/v1/admin/analytics/events", + ...options, + }); + } + /** * List Audit * diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 34e1d4dc1..cbcde94d4 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -16495,6 +16495,37 @@ export type UploadStart = { hold?: boolean; }; +/** + * UsageEvent + * + * An event Lens counted for a day (docs/analytics.md#events). + */ +export type UsageEvent = { + /** + * At + */ + at: string; + /** + * Day + */ + day: string; + /** + * Name + */ + name: string; + /** + * Params + */ + params?: { + [key: string]: unknown; + }; + /** + * Sent + */ + sent?: boolean | null; + [key: string]: unknown; +}; + /** * UserCreate */ @@ -19525,6 +19556,38 @@ export type UsageMetricsResponses = { export type UsageMetricsResponse = UsageMetricsResponses[keyof UsageMetricsResponses]; +export type UsageEventsData = { + body?: never; + path?: never; + query?: { + /** + * Limit + */ + limit?: number; + }; + url: "/api/v1/admin/analytics/events"; +}; + +export type UsageEventsErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type UsageEventsError = UsageEventsErrors[keyof UsageEventsErrors]; + +export type UsageEventsResponses = { + /** + * Response Admin-Usage Events + * + * Successful Response + */ + 200: Array; +}; + +export type UsageEventsResponse = UsageEventsResponses[keyof UsageEventsResponses]; + export type ListAuditData = { body?: never; path?: never; diff --git a/nextjs-frontend/components/analytics/analytics-page.tsx b/nextjs-frontend/components/analytics/analytics-page.tsx new file mode 100644 index 000000000..7883f27ab --- /dev/null +++ b/nextjs-frontend/components/analytics/analytics-page.tsx @@ -0,0 +1,126 @@ +"use client"; + +import { useQuery } from "@tanstack/react-query"; +import Link from "next/link"; +import { useState } from "react"; + +import { Admin } from "@/app/openapi-client"; +import { UsageNumbers } from "@/components/analytics/usage-numbers"; +import { SegmentedChoice } from "@/components/settings/controls"; +import { Banner } from "@/components/ui/banner"; +import { EmptyState, PageHeader, Skeleton } from "@/components/ui/states"; +import { data, useApiClient } from "@/lib/api/browser"; +import { useArchive } from "@/lib/hooks/session"; + +const PERIODS = [ + { value: "7", label: "7 days" }, + { value: "30", label: "30 days" }, + { value: "90", label: "90 days" }, + { value: "180", label: "180 days" }, +]; + +const EVENT_LABEL: Record = { + lens_install: "Install", + lens_setup_done: "Setup done", + lens_daily_active: "Daily active", + lens_feature_used: "Feature used", + lens_upgrade: "Upgrade", + lens_inactive: "Inactive", + lens_return: "Return", +}; + +function params(p: Record | undefined) { + return Object.entries(p ?? {}) + .map(([k, v]) => `${k.replaceAll("_", " ")} ${String(v)}`) + .join(" · "); +} + +/** The events Lens counted each day, and whether each went to Google Analytics. */ +function EventLog() { + const client = useApiClient(); + const q = useQuery({ + queryKey: ["usage-events"], + queryFn: () => data(Admin.usageEvents({ client, query: { limit: 200 } })), + }); + if (q.isError) return {q.error.message}; + if (!q.data) return ; + if (!q.data.length) + return ( +

+ No events yet. Lens counts the day before once a day, so the first ones show up tomorrow. +

+ ); + return ( +
+ + + + + + + + + + + {q.data.map((e, i) => ( + + + + + + + ))} + +
DayEventDetailsGoogle Analytics
{e.day}{EVENT_LABEL[e.name] ?? e.name}{params(e.params as Record)} + {e.sent === true ? "sent" : e.sent === false ? "waiting" : "kept here only"} +
+
+ ); +} + +/** Built-in analytics (docs/analytics.md): works with Google Analytics off; GA is an optional export. */ +export function AnalyticsPage() { + const { admin } = useArchive(); + const client = useApiClient(); + const [days, setDays] = useState("30"); + const ga = useQuery({ + queryKey: ["analytics-status"], + queryFn: () => data(Admin.analyticsStatus({ client })), + enabled: admin, + }); + if (!admin) + return ( +
+ Ask an admin of this archive. +
+ ); + const s = ga.data; + return ( +
+ } + className="mb-0" + /> + +
+

Events

+

+ {s?.configured ? ( + <>Each day’s events are also sent to Google Analytics ({s.measurement_id}). + ) : ( + <> + Kept on this server only. To also send them to your own Google Analytics property, turn it on in{" "} + + Settings → Usage analytics + + . + + )} +

+ +
+
+ ); +} diff --git a/nextjs-frontend/components/settings/usage-analytics.tsx b/nextjs-frontend/components/analytics/usage-numbers.tsx similarity index 100% rename from nextjs-frontend/components/settings/usage-analytics.tsx rename to nextjs-frontend/components/analytics/usage-numbers.tsx diff --git a/nextjs-frontend/components/app-shell/nav-config.ts b/nextjs-frontend/components/app-shell/nav-config.ts index cae4a1cbb..782b54350 100644 --- a/nextjs-frontend/components/app-shell/nav-config.ts +++ b/nextjs-frontend/components/app-shell/nav-config.ts @@ -1,6 +1,7 @@ import { Blocks, CalendarClock, + ChartLine, ChartNoAxesColumn, HardDriveDownload, House, @@ -61,6 +62,7 @@ export const NAV: NavItem[] = [ // Admins see every source as a sensor; members see the watched folders feeding their namespaces. { label: "Sensors", href: "/sensors", icon: RadioTower, adminOnly: true }, { label: "Sources", href: "/sources", icon: HardDriveDownload, membersOnly: true }, + { label: "Analytics", href: "/analytics", icon: ChartLine, adminOnly: true }, // Everyone has Settings: members find Speakers there, admins the archive's settings too. { label: "Settings", href: "/settings", icon: Settings, countKey: "reviews" }, { label: "Admin", href: "/admin", icon: Shield, adminOnly: true }, diff --git a/nextjs-frontend/components/settings/bodies.tsx b/nextjs-frontend/components/settings/bodies.tsx index 6e21e96ce..ab3ca2842 100644 --- a/nextjs-frontend/components/settings/bodies.tsx +++ b/nextjs-frontend/components/settings/bodies.tsx @@ -14,7 +14,6 @@ import { RIGHTS } from "@/components/iiif/rights"; import { hubText } from "@/components/sensors/sensor-model"; import { SecretSetting, SettingField, ZoneBar, type FieldState } from "@/components/settings/fields"; import { SignInProviders } from "@/components/settings/sign-in-providers"; -import { UsageNumbers } from "@/components/settings/usage-analytics"; import { AddConnection } from "@/components/storage/add-connection"; import { NOT_STORAGE, fileStoreTry } from "@/components/storage/file-store"; import { @@ -631,7 +630,14 @@ function AnalyticsBody({ ctx }: { ctx: BodyCtx }) { const id = (saved.measurement_id as string | null | undefined) ?? null; return ( <> - +

+ Active people, churn, retention and feature use are on the{" "} + + Analytics + {" "} + page, worked out on this server whether or not Google Analytics is on. This section only chooses whether the + daily events also go to your Google Analytics property. +

{on && s?.configured ? ( {s.last_error diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index fcb2cf488..ecd8860c7 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -2822,6 +2822,56 @@ } } }, + "/api/v1/admin/analytics/events": { + "get": { + "tags": [ + "admin" + ], + "summary": "Usage Events", + "description": "The events Lens counted each day (install, daily active, feature used, upgrade, inactive...), newest first, and\nwhether each went to Google Analytics. Kept here whether or not Google Analytics is on.", + "operationId": "usage_events", + "parameters": [ + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "type": "integer", + "maximum": 1000, + "minimum": 1, + "default": 200, + "title": "Limit" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "$ref": "#/components/schemas/UsageEvent" + }, + "title": "Response Admin-Usage Events" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/audit": { "get": { "tags": [ @@ -52908,6 +52958,48 @@ ], "title": "UploadStart" }, + "UsageEvent": { + "properties": { + "at": { + "type": "string", + "title": "At" + }, + "day": { + "type": "string", + "title": "Day" + }, + "name": { + "type": "string", + "title": "Name" + }, + "params": { + "additionalProperties": true, + "type": "object", + "title": "Params", + "default": {} + }, + "sent": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Sent" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "at", + "day", + "name" + ], + "title": "UsageEvent", + "description": "An event Lens counted for a day (docs/analytics.md#events)." + }, "UserCreate": { "properties": { "email": { From 6f23496226f961867be15717549195ce32a8a167 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 10 Oct 2026 21:14:30 +0000 Subject: [PATCH 5/5] docs(analytics): threat model and changelog entries for usage analytics Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HsT2KQQsLmAyBFPRxw1VVH --- CHANGELOG.md | 6 ++++++ threat-model.md | 5 +++-- threat-model.yaml | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10ea98810..a70255d03 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,12 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **Analytics.** A new Analytics page (admins) shows active people per day, week and month, stickiness, churn rate, + retention by sign-up week and feature use, worked out on the server from the activity ledger and sign-ins. Each day + Lens also keeps a few counted events (install, daily active, feature used, upgrade, inactive, return). They can + optionally go to your own Google Analytics 4 property too; that is off by default and is set in Settings → Usage + analytics or the setup wizard ([Usage analytics](docs/analytics.md)). + - **Prometheus metrics.** Lens can serve `/metrics` for Prometheus and Grafana: jobs by status and pipeline, imports, model calls, tokens, cost and failures by model, budgets, workers and disk. Off by default; turned on in Settings → Telemetry (or `LENS_PROMETHEUS=on`), it answers only a scraper with the scrape token, and follows `LENS_BIND`. A diff --git a/threat-model.md b/threat-model.md index 36f617348..58c8a55b9 100644 --- a/threat-model.md +++ b/threat-model.md @@ -199,7 +199,8 @@ flowchart LR and workers may run at once; throttle counters and the setup code live in each process's memory *(inferred, Q5)*. **Host side effects.** Lens makes no outbound connection except to configured sources, model servers, mail, -notification and telemetry targets; telemetry is off by default and there is no update check *(maintainer, 2026-10)*. +notification and telemetry targets; telemetry and the Google Analytics export (`analytics.py`: daily counts for the +install, no personal content) are off by default and there is no update check *(maintainer, 2026-10)*. | Behaviour | Observed | Tag | | --- | --- | --- | @@ -239,7 +240,7 @@ machine requires HTTPS, and a changed DB credential when SurrealDB runs as its o | `documents.web_networks`, `LENS_WEB_NETWORKS`; `notifications.networks` | empty | widen the outbound guard to listed private networks | Yes (operator and admin input) | *(documented, docs/configuration.md "loopback and cloud metadata addresses stay out")* | | Chromium sandbox | off as root or without user namespaces | the `netguard` proxy is the only barrier | Yes | *(documented, docs/configuration.md "runs with its sandbox where it can")* | | `sensors.enabled`, `sensors.mqtt_anonymous` | off (`bind` `0.0.0.0`); off | opens MQTT and syslog listeners; anonymous MQTT publish | Yes; anonymous is the admin's choice | *(documented, docs/sensors.md "Sensors are off until an admin turns them on")* | -| `tunnel.mode`, `local_llm.enabled`, `telemetry` | off | outbound tunnel (Cloudflare terminates TLS, Q30); local model server; OTLP export | Yes | *(documented, docs/remote-access.md "It's off until an admin turns it on")* | +| `tunnel.mode`, `local_llm.enabled`, `telemetry`, `analytics.enabled` | off | outbound tunnel (Cloudflare terminates TLS, Q30); local model server; OTLP export; GA4 Measurement Protocol export of daily counts | Yes | *(documented, docs/remote-access.md "It's off until an admin turns it on")* | | `components.auto` | on | unpinned-hash pip installs at runtime | Yes | *(maintainer, 2026-10)* | | `encryption.files` | on for new archives | file encryption at rest | Yes | *(documented, docs/encryption.md "A new archive turns this on at its first start")* | | `ai.extensions`, `ai.disabled_tools` | on, none | turn assistant extensions or named tools off | Yes | *(documented, docs/assistant.md "Extending the assistant")* | diff --git a/threat-model.yaml b/threat-model.yaml index 76ecb5aa5..71810ef05 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -2,7 +2,7 @@ # Authority order: prose > yaml > json. Regenerate whenever the prose changes. schema: threat-model-sidecar/v2 project: lens -prose_version: "threat-model.md@sha256:4525de4bf73248fe9511a6ad69c55ff7c021b11a9a3e46a285bf25fe5208ea99" +prose_version: "threat-model.md@sha256:067cef03edea230dfa3c5fec7f288dffea4e0c063c899c02b22ea41f3d8d2389" model_status: under-review triage_policy: strict confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8}