diff --git a/CHANGELOG.md b/CHANGELOG.md index 176d51f97..2f2c9a521 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **NSFW classification, with a model on your server.** Turn it on in Settings → NSFW for every namespace, or on a + namespace's page for that one (off by default). The new `nsfw` step scores a video's sampled frames, a document's + pages or an image with an Apache-2.0 model (fetched on first use, about 105 MB with ONNX Runtime) and flags the + resource when a picture scores at least the threshold. Flagged resources are blurred until someone chooses to see + them, left out of the Library, or shown with a label; editors can mark one NSFW or not by hand, and **Classify + existing** classifies what was there before. Agents use it through the API and MCP (`nsfw_settings`, `set_nsfw`, + `mark_nsfw`) ([NSFW classification](docs/nsfw.md)). + - **Apache Tika.** Lens can use an Apache Tika server (`--profile tika` in the compose files, or one of your own, set in Settings → Documents). Documents nothing here can make into a PDF, such as Word files on a lean server or a Raspberry Pi, are read by Tika as text, and every document's own title, authors, language and dates are read as diff --git a/docs/api.md b/docs/api.md index 5523d0134..dfc8ae034 100644 --- a/docs/api.md +++ b/docs/api.md @@ -255,6 +255,9 @@ DELETE /api/v1/namespaces/{name}/notifications/{tid} POST /api/v1/namespaces/{name}/notifications/{tid}/test POST /api/v1/namespaces/{name}/notifications/{tid}/secret GET /api/v1/namespaces/{name}/notifications/{tid}/deliveries +GET /api/v1/namespaces/{name}/nsfw +PUT /api/v1/namespaces/{name}/nsfw +POST /api/v1/namespaces/{name}/nsfw/classify GET /api/v1/namespaces/{name}/collections POST /api/v1/namespaces/{name}/collections GET /api/v1/namespaces/{name}/collections/{cid} @@ -294,6 +297,15 @@ removes it). `POST …/{tid}/test` sends a test message now and answers `{ok, co lists the latest 50 messages with their `status` (`pending`, `sending`, `sent`, `failed`, `dropped`), `attempts`, answer `code` and `error`. Changes are audited as `namespace.notification.create`, `.update`, `.delete` and `.secret`. +`/namespaces/{name}/nsfw` says how [NSFW classification](nsfw.md) works in a namespace (viewers): whether it's +`enabled`, the `threshold` and `display` that apply, what the namespace chose itself (`own`, `null` where it follows +the archive) and the archive's settings (`archive`), and how many resources are `classified`, `flagged` and +`unclassified`. `PUT` with `{enabled?, threshold?, display?, classify?}` changes the namespace's own choices (owners; +`null` follows the archive again, a new threshold flags or unflags what was classified at once), and `classify: true` +also queues the `nsfw` step for its videos, documents and images not classified yet, answering how many runs were +`queued`. Audited as `namespace.nsfw`. `POST …/nsfw/classify` queues them on its own (editors) and answers the `jobs`. +Both are a 409 where classification is off in the namespace. + ### Collections of a namespace Every recording lives in exactly one collection of its namespace, and collections nest (at most 8 deep, 5000 per @@ -340,6 +352,7 @@ GET /api/v1/resources/tags GET /api/v1/resources/origins GET /api/v1/resources/languages GET /api/v1/resources/objects +GET /api/v1/resources/nsfw POST /api/v1/resources/tags POST /api/v1/resources/collection GET /api/v1/resources/{rid} @@ -358,6 +371,7 @@ GET /api/v1/resources/{rid}/ip-groups PUT /api/v1/resources/{rid}/ip-groups/{gid} DELETE /api/v1/resources/{rid}/ip-groups/{gid} GET /api/v1/resources/{rid}/player +PUT /api/v1/resources/{rid}/nsfw GET /api/v1/resources/{rid}/embed-link GET /api/v1/resources/{rid}/audio GET /api/v1/resources/{rid}/wordcloud.svg @@ -398,6 +412,7 @@ repeat a parameter that takes several values (`?status=new&status=error`) to mat | `language` | language codes, ignoring case; `none` for recordings whose language isn't known | | `object` | kinds of object seen in them (`person`, `car` …; [Objects](#objects)), ignoring case | | `edited_by` | `me`: recordings you edited (corrected a line of the transcript, changed the catalogue record, or renamed) | +| `sensitive` | where [NSFW classification](nsfw.md) is on: `only` the ones flagged NSFW, `exclude` them, or `include` them. Without it, flagged ones are left out where their namespace hides them (`display: hide`) | | `sort` | `date`, `title`, `duration`, `speakers`, `status` or `importance`; `-` in front for descending (default `-date`). Recordings without the value come last either way | | `limit`, `offset` | one page (default 500 rows, at most 1000) | @@ -1043,12 +1058,25 @@ and each place it was found (`[t, x, y, w, h, score]`, at most 500). The kinds a `GET /resources?object=` filters on; search finds them by name (`source: object`), and its `object` filter and facet use them ([search](#search)). Moving or deleting a resource moves or deletes its objects. +### NSFW + +The `nsfw` step scores a video's sampled frames, a document's pages or an image for NSFW pictures with a model on the +server, where [NSFW classification](nsfw.md) is on in the resource's namespace (off by default); it comes after +`objects` in the standard pipeline, and is skipped, saying why, where it's off. A resource, its player and its Library +row carry `nsfw`: `{flagged, score, manual, blur, display, threshold, checked, of, at}`, and on the resource and its +player `hits`, its highest-scoring pictures (`{t, score, frame}`, a signed link); `null` where it's off in the +namespace or the resource wasn't classified. `PUT /resources/{rid}/nsfw {flagged}` (editors) marks it NSFW (`true`) +or not (`false`) by hand, over the model's score, or hands it back to the score (`null`); a 409 where it's off. +Audited as `recording.nsfw`. `GET /resources/nsfw` (`ns` for one namespace) says whether it's `on` in any namespace +you read, how many resources are `flagged`, and how many of those the Library leaves out (`hidden`). Search hits and +photo search results carry `nsfw: true` for the ones the web app blurs. + ### Descriptions The `describe` step has a model that can see images, the one an admin chose as `llm.vision_model`, say in a few sentences what each page of a document or an image shows, and each shot of a video (from its keyframe): the setting, the people (by what they do and wear, never who they might be), the things and any text in it. It comes after -`objects` in the standard pipeline, and describes at most `llm.describe_max` pages or shots of a resource. Without an +`nsfw` in the standard pipeline, and describes at most `llm.describe_max` pages or shots of a resource. Without an LLM, or with no `llm.vision_model`, it's skipped, saying why; a model that can't see makes it fail with what the server said ([Configuration](configuration.md#descriptions)). The player's `descriptions` lists them in order: `{idx, t0, t1, text, model, frame, paged}`: the shot or page, where it starts and ends (ms; on pages, the page from 0 diff --git a/docs/components.md b/docs/components.md index f2a387b5b..2c2460ccf 100644 --- a/docs/components.md +++ b/docs/components.md @@ -19,6 +19,7 @@ never fetch a speech or voice model. | Face models | `video.face_engine` is `opencv` (the default) | OpenCV, YuNet and SFace | | Face models | `video.face_engine` is `anytopdf` | SFace, for anytopdf to describe faces with (anytopdf too) | | Object model | `video.object_engine` is `yolox` (the default) or `anytopdf` | YOLOX-s, and ONNX Runtime for `yolox` | +| NSFW model | [NSFW classification](nsfw.md) is on for the archive or a namespace (off by default) | the 8-bit ONNX export of AdamCodd/vit-base-nsfw-detector (Apache-2.0, 88 MB), and ONNX Runtime | | Prompt-injection classifier | `ai.injection_check` is `classifier` | ProtectAI's deberta-v3-base-prompt-injection-v2 (Apache-2.0, about 740 MB), the tokenizers library and ONNX Runtime ([third-party content](assistant.md#third-party-content-and-prompt-injection)) | | yt-dlp | `documents.video_urls` is on | the yt-dlp program, release 2026.08.19 (Unlicense), to download videos and podcasts imported by address | | Chat model, embedding model | the LLM provider or embeddings server is Ollama | pulled on that server (`/api/pull`) | @@ -27,7 +28,7 @@ never fetch a speech or voice model. - Python packages go into `data_dir/python`, at the versions in `uv.lock`. Packages the image already has win. - Models go into `data_dir/models`: the Hugging Face, ModelScope and PyTorch caches point there (unless `HF_HOME`, `MODELSCOPE_CACHE` or `TORCH_HOME` say otherwise). They outlast container rebuilds and upgrades. -- Model files fetched by URL (YOLOX, YuNet, SFace) are checked against their SHA-256. +- Model files fetched by URL (YOLOX, YuNet, SFace, the NSFW model) are checked against their SHA-256. - A job step that needs something still being fetched waits for it instead of falling back or failing. If a fetch fails, the step runs as before (for transcription, falling back to another engine), and the fetch is tried again an hour later, when the settings change, or on **Check again now**. diff --git a/docs/configuration.md b/docs/configuration.md index b94744ea5..b850725bc 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -96,7 +96,7 @@ Start from `fastapi_backend/archive.example.yaml`, which documents every key. Th Admins can change, in Settings: transcription, speaker separation, speech providers, voice IDs, analysis, the LLM provider and key, a local model, the AI assistant, search (word search and search by meaning), reports and graph, -video, OCR, faces and objects, workers, sign-in, components, access and embedding (embed frame ancestors, transcript +video, OCR, faces and objects, [NSFW classification](nsfw.md), workers, sign-in, components, access and embedding (embed frame ancestors, transcript upload limit, allowed hosts, trusted proxies, session length), remote access, email, chat rooms, notifications, telemetry, the Fedora repository, sensors, file shares, uploads, storage, documents, API keys, and IIIF and metadata. The API refuses an allowed-host list that leaves out the address you are using. diff --git a/docs/mcp.md b/docs/mcp.md index 8e96a018d..965bb4c99 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -45,9 +45,9 @@ front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](aut | `search` | moments where words are said, shown on screen or written, best first: the recording, speaker, time, the matching text (matches in **bold**) and a link to that moment. Words, `"phrases"`, `OR`; by namespace, speaker or recording | | `find_similar` | the recordings, documents and images most like a text, a recording, a topic or an entity: why each is alike (meaning, key words, topics, entities in common) and the moments that show it, with links | | `list_namespaces` | the namespaces you can read, with their recordings, hours, speakers and your role | -| `list_recordings` | recordings, newest first, by namespace, title words, dates, speaker, tag or kind | +| `list_recordings` | recordings, newest first, by namespace, title words, dates, speaker, tag or kind, and whether they're flagged NSFW (`sensitive`) | | `list_speakers` | the speakers of your namespaces, the ones who talk most first | -| `get_recording` | one recording: when, how long, speakers, summary and key points, chapters, the entities and keywords most mentioned, its link | +| `get_recording` | one recording: when, how long, speakers, summary and key points, chapters, the entities and keywords most mentioned, its link, and whether it's flagged NSFW | | `get_transcript` | a recording's lines in order (speaker, time, text, link), a page at a time, from a line or a time window | | `fetch` | a recording's whole text at once, as `[m:ss] Speaker: …` lines | | `cite` | a citation for a moment (a line or a second, and a few lines from there): the words, who said them, the recording and date, the time, the link, and all of it as Markdown | @@ -68,6 +68,9 @@ front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](aut | `write_note` | write a note or a thing's page, marked as written by an assistant. Needs a write-scope token and editor access | | `create_podcast` | make a two-host learning [episode](podcasts.md) about recordings or lines in them, with an angle, a length and a style; it's made in the background. Needs a write-scope token and editor access where it goes | | `get_podcast` | an episode: how far it has got, its script with each line's speaker and the sources it cites (links), and what the fact-check changed | +| `nsfw_settings` | how [NSFW classification](nsfw.md) works in a namespace: on or off, the threshold, how flagged resources show, and how many are classified, flagged and not classified yet | +| `set_nsfw` | turn NSFW classification on or off, set its threshold and display, for a namespace (owners) or the archive (admins), and optionally classify what's there. Needs a write-scope token | +| `mark_nsfw` | mark a resource NSFW or not by hand, or hand it back to the model's score. Needs a write-scope token and editor access | | `list_files` | a folder of [the graph as files](file-tree.md): your namespaces, then their collections, topics (nested by narrower topics) and entities, each holding its assets' files | | `read_file` | a text file of that tree (an asset's `.txt` text or `.json` metadata), a page at a time; other files' download link | | `file_storage` | where Lens keeps its own files (notes' images and attachments) and the connections it could use; `check` tests it. Admins only | @@ -82,7 +85,8 @@ expect (`results` with `id`, `title`, `url`; `fetch` by that `id`). Every tool reads, except `suggest_topic`, which adds suggestions people accept or dismiss, `propose_graph_change`, which records an undoable change (proposed by default), `write_note`, which adds a note, `create_podcast`, which -queues an episode, and `set_file_storage`, which changes where new files are kept (admins). +queues an episode, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and +editors), and `set_file_storage`, which changes where new files are kept (admins). Importing, editing and curating stay in the web app and the API. ### What the archive says is data diff --git a/docs/nsfw.md b/docs/nsfw.md new file mode 100644 index 000000000..cc4fa05dd --- /dev/null +++ b/docs/nsfw.md @@ -0,0 +1,90 @@ +# NSFW classification + +Lens can tell which videos, documents and images have NSFW pictures in them (nudity and sexual content), with a model +that runs on your own server. It's **off** until an admin turns it on for every namespace, or a namespace's owner for +theirs. Pictures never leave the server for it, and no model provider is called, so it costs nothing per picture. + +## Turning it on + +- **Every namespace:** Settings → NSFW, **Classify NSFW pictures** (`nsfw.enabled`). +- **One namespace:** its page (Admin → Namespaces → the namespace → NSFW), or `PUT /api/v1/namespaces/{name}/nsfw` + (owners). A namespace can also stay off while the rest is on, and have its own threshold and display. + +Once it's on, the **nsfw** step in the standard pipeline classifies what's added from then on. Resources that were +there before are classified when someone asks: **Classify existing** on the namespace's page, `classify: true` with the +`PUT`, or `POST /api/v1/namespaces/{name}/nsfw/classify` (editors). A pipeline of your own runs it when you add the +**nsfw** step to it. + +The first time a resource waits at the step, the worker fetches the model and ONNX Runtime (about 105 MB together, +[Components](components.md)); until then nothing is downloaded, so a Raspberry Pi that never turns it on stays small. + +## What it does + +The nsfw step scores each of a video's sampled frames (from the `shots` step), each page of a document, or the image, +from 0 (safe for work) to 1, and keeps the highest. A long video or document has at most `nsfw.max_frames` pictures +scored, spread evenly over it. A resource is **flagged** when its highest score is at least the namespace's +threshold. Changing the threshold flags or unflags what was already classified at once, without running the model +again. + +The model is [AdamCodd/vit-base-nsfw-detector](https://huggingface.co/AdamCodd/vit-base-nsfw-detector) (Apache-2.0): +a Vision Transformer fine-tuned to tell NSFW from SFW pictures, as its author's 8-bit ONNX export, pinned to one +revision and checked against its SHA-256. It was trained to be strict: pictures with a lot of skin count as NSFW. It +is less sure of generated pictures. So a person can **mark a resource NSFW or not** by hand (editors), and their call +wins over the score until they hand it back to the model. + +## How flagged resources show + +`nsfw.display`, or the namespace's own: + +| Display | In the web app | +|---|---| +| `blur` (default) | its pictures (the video, frames, faces, pages and photo search results) are blurred until someone chooses to see them | +| `hide` | left out of the Library unless you ask for them, and blurred when shown | +| `show` | shown as they are, with a label | + +A flagged resource has an **NSFW** label in the Library, and its page says what was found, with **Show pictures** +and, for editors, **Not NSFW**. Its Details tab says what classification found on any classified resource, with +**Mark NSFW**. Every mark comes with Undo. Where it's on in a namespace you can read, the Library has an **NSFW** +filter: as each namespace shows them, including hidden ones, only flagged ones, or leaving them out +(`/library?namespace=…&nsfw=only` opens it so). + +Blurring and hiding are display preferences, not access control: anyone who may read a resource can still open it, +see its pictures and read its classification. + +## Settings + +| Setting | Default | | +|---|---|---| +| `nsfw.enabled` | `false` | classify in every namespace that doesn't say otherwise | +| `nsfw.threshold` | `0.8` | flag a resource when one of its pictures scores at least this (0.5 to 0.99) | +| `nsfw.display` | `blur` | `blur`, `hide` or `show` | +| `nsfw.max_frames` | `300` | the most pictures scored per resource (1 to 5000) | + +A namespace's own `enabled`, `threshold` and `display` win over these; `null` follows them again. + +## API and agents + +- `GET /api/v1/namespaces/{name}/nsfw`: what applies in a namespace, what it chose itself, the archive's settings, and + how many resources are classified, flagged and not classified yet. +- `PUT /api/v1/namespaces/{name}/nsfw {enabled?, threshold?, display?, classify?}` (owners). Audited as `namespace.nsfw`. +- `POST /api/v1/namespaces/{name}/nsfw/classify` (editors): queue the nsfw step for what isn't classified yet. +- `PUT /api/v1/resources/{rid}/nsfw {flagged}` (editors): `true`, `false`, or `null` to let the score decide. Audited + as `recording.nsfw`. +- Resources, their player and Library rows carry `nsfw`: `{flagged, score, manual, blur, display, threshold, checked, + of, at}`, and on a resource and its player `hits`, its highest-scoring pictures with signed links. It's `null` where + classification is off in the namespace or the resource wasn't classified. +- `GET /api/v1/resources?sensitive=only|exclude|include` filters the Library, and `GET /api/v1/resources/nsfw` says + whether it's on in any namespace you read, how many are flagged, and how many of those are hidden. Search hits and + photo search results carry `nsfw: true` for the ones the web app blurs. +- [MCP](mcp.md): `nsfw_settings`, `set_nsfw` (owners for a namespace, admins for the archive) and `mark_nsfw` + (editors); `get_recording` and `list_recordings` say what's flagged. + +The resource's job log says what the step found: `flagged NSFW: highest score 0.93 (threshold 0.8) over 1 page(s)`. + +## Refine later + +- Blur flagged pictures for visitors on public pages, embeds and IIIF, as faces can be pixelated. +- Keep the Library's NSFW filter in saved views. +- Skip sending flagged pictures to a cloud vision model (the `describe` step). +- Other models (a smaller one for a Raspberry Pi, one trained on generated pictures) to choose from. +- Scores per picture in the player, to jump to what was flagged. diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py index aa94b93f6..f90e878d1 100644 --- a/fastapi_backend/app/api/mcp_tools.py +++ b/fastapi_backend/app/api/mcp_tools.py @@ -42,6 +42,7 @@ library, monitoring, notebook, + nsfw, rdf, render, settings, @@ -531,6 +532,13 @@ def list_namespaces(ctx: Context) -> dict[str, Any]: Arg("speaker_id", "integer", "only recordings this speaker is in"), Arg("tag", "string", "only recordings with this tag", max_length=100), Arg("media", "string", "only this kind", enum=MEDIA), + Arg( + "sensitive", + "string", + "where NSFW classification is on: only those flagged NSFW, exclude them, or include them (without it, flagged ones " + "are left out where their namespace hides them)", + enum=("include", "only", "exclude"), + ), Arg("sort", "string", "order: date, title or duration; - first for descending", default="-date", enum=SORTS), Arg("limit", "integer", "how many", default=20, minimum=1, maximum=100), Arg("offset", "integer", "skip this many (for the next page)", default=0, minimum=0, maximum=10000), @@ -544,6 +552,7 @@ def list_recordings( speaker_id: int | None, tag: str | None, media: str | None, + sensitive: str | None, sort: str, limit: int, offset: int, @@ -563,6 +572,7 @@ def list_recordings( media=media, tags=[tag] if tag else None, cfg=ctx.cfg, + sensitive=sensitive, ) out: dict[str, Any] = { "total": total, @@ -578,6 +588,7 @@ def list_recordings( "speakers": r.get("speakers") or None, "tags": r.get("tags") or None, "language": r.get("language"), + "nsfw": True if (r.get("nsfw") or {}).get("flagged") else None, "url": ctx.link(r["id"]), } for r in rows @@ -673,6 +684,8 @@ def get_recording(ctx: Context, recording_id: int) -> dict[str, Any]: out["email"] = rec.email.model_dump(exclude_none=True) if rec.web: out["web_page"] = rec.web.model_dump(exclude_none=True) + if rec.nsfw: + out["nsfw"] = rec.nsfw.model_dump(exclude_none=True, include={"flagged", "score", "manual", "threshold", "checked", "of"}) return {k: v for k, v in out.items() if v is not None} @@ -1483,6 +1496,92 @@ def get_podcast(ctx: Context, episode_id: int) -> dict[str, Any]: ) +# ---------- NSFW classification ---------- +@tool( + "nsfw_settings", + "NSFW classification", + "Whether a local model classifies NSFW pictures (nudity and sexual content) in a namespace's videos, documents and " + "images; the threshold a resource is flagged at; how flagged resources show in the web app (blur, hide from the " + "Library, or show); what the namespace chose itself and the archive's settings it otherwise follows; and how many " + "are classified and flagged. Off unless an admin or the namespace's owner turns it on.", + Arg("namespace", "string", "the namespace (see list_namespaces)", required=True), +) +def nsfw_settings(ctx: Context, namespace: str) -> dict[str, Any]: + out = nsfw.status(ctx.db, ctx.cfg, ctx.acl.namespace(namespace)) + return {**out, "settings": f"{ctx.web}/admin/namespaces/{namespace}#nsfw"} + + +@tool( + "set_nsfw", + "Change NSFW classification", + "Turn NSFW classification on or off for a namespace, or give it its own threshold (0.5 to 0.99) or display (blur, " + "hide or show) (its owners); without a namespace, change the archive's settings that every namespace follows " + "unless it chose otherwise (admins). classify_existing also queues the namespace's videos, documents and images " + "not classified yet; new ones are classified by their pipeline. The model runs on this server and is fetched the " + "first time it's needed. Needs a token with the write scope.", + Arg("namespace", "string", "the namespace; leave it out for the archive's settings (admins)"), + Arg("enabled", "boolean", "on or off"), + Arg("threshold", "number", "flag a resource when one of its pictures scores at least this (0.5 to 0.99)"), + Arg("display", "string", "how flagged resources show in the web app", enum=nsfw.DISPLAYS), + Arg("classify_existing", "boolean", "also queue those not classified yet (needs a namespace)", default=False), + writes=True, +) +def set_nsfw( + ctx: Context, + namespace: str | None = None, + enabled: bool | None = None, + threshold: float | None = None, + display: str | None = None, + classify_existing: bool = False, +) -> dict[str, Any]: + if not ctx.user.can_write: + raise ToolError("this token is read-only: ask for a token with the write scope to change NSFW classification") + changes = {k: v for k, v in (("enabled", enabled), ("threshold", threshold), ("display", display)) if v is not None} + if not changes and not classify_existing: + raise ToolError("say what to change: enabled, threshold, display or classify_existing") + if namespace is None: + if classify_existing: + raise ToolError("classify_existing needs a namespace") + if not ctx.user.admin: + raise ToolError("only admins change the archive's NSFW settings; give a namespace you own") + settings.save(ctx.db, ctx.request.app.state.archive.base, "nsfw", changes, ctx.user.email) + auth.audit(ctx.db, ctx.user.as_audit(), "settings.save", "nsfw", sorted(changes) + ["mcp"]) + if "threshold" in changes: + nsfw.reflag(ctx.db, ctx.request.app.state.settings.current()) + return {"saved": changes, "archive": {k: (ctx.request.app.state.settings.current().get("nsfw") or {}).get(k) for k in changes}} + sid = ctx.acl.namespace(namespace, "owner") + nsfw.save_own(ctx.db, sid, changes) + if "threshold" in changes: + nsfw.reflag(ctx.db, ctx.cfg, [sid]) + out: dict[str, Any] = {"saved": changes} + if classify_existing: + if not nsfw.policy(ctx.db, ctx.cfg, sid)["enabled"]: + raise ToolError("NSFW classification is off in this namespace: turn it on first (enabled true)") + out["queued"] = len(nsfw.queue_unclassified(ctx.db, sid, ctx.user.email)) + auth.audit(ctx.db, ctx.user.as_audit(), "namespace.nsfw", namespace, {**changes, "via": "mcp"}) + return {**out, **nsfw.status(ctx.db, ctx.cfg, sid)} + + +@tool( + "mark_nsfw", + "Mark a recording NSFW or not", + "A person's own call on a recording, document or image, where NSFW classification is on in its namespace: nsfw or " + "not_nsfw wins over the model's score; model lets the score decide again. Editors, with a token that has the write " + "scope.", + Arg("recording_id", "integer", "the recording", required=True), + Arg("verdict", "string", "nsfw, not_nsfw, or model", required=True, enum=("nsfw", "not_nsfw", "model")), + writes=True, +) +def mark_nsfw(ctx: Context, recording_id: int, verdict: str) -> dict[str, Any]: + if not ctx.user.can_write: + raise ToolError("this token is read-only: ask for a token with the write scope to mark recordings") + ctx.acl.recording(recording_id, "editor") + flagged = {"nsfw": True, "not_nsfw": False, "model": None}[verdict] + got = nsfw.mark(ctx.db, ctx.cfg, recording_id, flagged, ctx.user.email) + auth.audit(ctx.db, ctx.user.as_audit(), "recording.nsfw", f"recording:{recording_id}", {"flagged": flagged, "via": "mcp"}) + return {"recording_id": recording_id, "nsfw": got, "url": ctx.link(recording_id)} + + # ---------- the graph as files ---------- FILE_CHARS = 100_000 # the most of a file read_file returns at once; offset pages through the rest diff --git a/fastapi_backend/app/api/v1/routes/admin.py b/fastapi_backend/app/api/v1/routes/admin.py index 8f4f98ae9..66b72af5b 100644 --- a/fastapi_backend/app/api/v1/routes/admin.py +++ b/fastapi_backend/app/api/v1/routes/admin.py @@ -30,6 +30,7 @@ matterbridge, monitoring, nfs_share, + nsfw, semantic, settings, slack, @@ -114,6 +115,8 @@ def update_settings(section: str, body: dict[str, Any], user: AdminWriter, reque if section == "encryption" and "files" in body and bool(body["files"]) != bool(files_were): cfg = request.app.state.settings.current() keyring.convert(db, cfg, bool(cfg["encryption"].get("files"))) # the files already kept follow, in the background + if section == "nsfw" and "threshold" in body: + nsfw.reflag(db, request.app.state.settings.current()) # what's flagged follows the new threshold return Ok() diff --git a/fastapi_backend/app/api/v1/routes/components.py b/fastapi_backend/app/api/v1/routes/components.py index be47e26aa..73787bba9 100644 --- a/fastapi_backend/app/api/v1/routes/components.py +++ b/fastapi_backend/app/api/v1/routes/components.py @@ -21,7 +21,7 @@ def list_components(user: AdminReader, db: Db, cfg: Cfg) -> Components: ahead=bool((cfg.get("components") or {}).get("ahead", False)), machine=machine.probe(cfg["data_dir"]), recommended=components.recommendation(cfg), - components=components.catalog(cfg), + components=components.catalog(cfg, db), workers=[ {k: w.get(k) for k in ("name", "host", "heartbeat_at", "machine")} | {"components": w.get("components") or {}} for w in jobs.workers(db) diff --git a/fastapi_backend/app/api/v1/routes/namespaces.py b/fastapi_backend/app/api/v1/routes/namespaces.py index a58694c3d..82ad7f71a 100644 --- a/fastapi_backend/app/api/v1/routes/namespaces.py +++ b/fastapi_backend/app/api/v1/routes/namespaces.py @@ -9,11 +9,12 @@ from fastapi import APIRouter, HTTPException, Query, Request from fastapi.responses import Response -from app.api.deps import Acl, AdminWriter, CurrentUser, Db, Writer, domain_errors, visitor_address +from app.api.deps import Acl, AdminWriter, Cfg, CurrentUser, Db, Writer, domain_errors, visitor_address from app.api.media import sign_urls -from app.domain import analyze, auth, ipgroups, library, pipelines, render, store +from app.domain import analyze, auth, ipgroups, library, nsfw, pipelines, render, store from app.domain.store import API, DB from app.schemas.common import Created, Ok +from app.schemas.jobs import JobsQueued from app.schemas.namespaces import ( IpGroup, IpGroupCreate, @@ -21,6 +22,8 @@ IpGroupUpdate, Namespace, NamespaceCreate, + NamespaceNsfw, + NamespaceNsfwUpdate, NamespaceStats, NamespaceUpdate, ) @@ -97,6 +100,43 @@ def update_namespace(name: str, body: NamespaceUpdate, acl: Acl, user: Writer, d return Ok() +@router.get("/{name}/nsfw") +def get_namespace_nsfw(name: str, acl: Acl, db: Db, cfg: Cfg) -> NamespaceNsfw: + """NSFW classification here (docs/nsfw.md): whether it's on, the threshold and how flagged resources show, what the + namespace chose for itself and the archive's settings, and how many resources are classified and flagged.""" + return NamespaceNsfw.model_validate(nsfw.status(db, cfg, acl.namespace(name))) + + +@router.put("/{name}/nsfw") +def update_namespace_nsfw(name: str, body: NamespaceNsfwUpdate, acl: Acl, user: Writer, db: Db, cfg: Cfg) -> NamespaceNsfw: + """Owners: turn NSFW classification on or off for this namespace, or give it its own threshold or display; null + follows the archive's setting again. A new threshold re-flags what's classified at once. `classify` also queues the + nsfw step for its videos, documents and images not classified yet. Audited as `namespace.nsfw`.""" + sid = acl.namespace(name, "owner") + changes = {k: getattr(body, k) for k in ("enabled", "threshold", "display") if k in body.model_fields_set} + with domain_errors(): + nsfw.save_own(db, sid, changes) + if "threshold" in changes: + nsfw.reflag(db, cfg, [sid]) + queued = None + if body.classify: + if not nsfw.policy(db, cfg, sid)["enabled"]: + raise HTTPException(409, "NSFW classification is off in this namespace: turn it on first") + queued = len(nsfw.queue_unclassified(db, sid, user.email)) + auth.audit(db, user.as_audit(), "namespace.nsfw", name, {**changes, **({"classify": queued} if body.classify else {})}) + return NamespaceNsfw.model_validate({**nsfw.status(db, cfg, sid), "queued": queued}) + + +@router.post("/{name}/nsfw/classify") +def classify_namespace_nsfw(name: str, acl: Acl, user: Writer, db: Db, cfg: Cfg) -> JobsQueued: + """Editors: queue the nsfw step for the namespace's videos, documents and images not classified yet (those added + while it's on are classified by their pipeline). 409 where classification is off.""" + sid = acl.namespace(name, "editor") + if not nsfw.policy(db, cfg, sid)["enabled"]: + raise HTTPException(409, "NSFW classification is off in this namespace: turn it on first") + return JobsQueued(jobs=nsfw.queue_unclassified(db, sid, user.email)) + + @router.get("/{name}/wordcloud.svg", response_class=Response, responses={200: {"content": {"image/svg+xml": {}}}}) def get_namespace_wordcloud(name: str, acl: Acl, db: Db) -> Response: """The namespace's word cloud. Accepts a bearer token or the signed link from the namespace list.""" diff --git a/fastapi_backend/app/api/v1/routes/recordings.py b/fastapi_backend/app/api/v1/routes/recordings.py index 4a2911719..96e85ee29 100644 --- a/fastapi_backend/app/api/v1/routes/recordings.py +++ b/fastapi_backend/app/api/v1/routes/recordings.py @@ -28,6 +28,7 @@ keyring, library, moving, + nsfw, render, sources, store, @@ -45,6 +46,8 @@ LanguageCount, MediaKind, NamespaceAccess, + NsfwCounts, + NsfwState, ObjectCount, OriginCount, Output, @@ -58,6 +61,7 @@ RecordingIpGroup, RecordingMove, RecordingMoved, + RecordingNsfwUpdate, RecordingSort, RecordingsPlace, RecordingsRetag, @@ -136,6 +140,11 @@ def list_recordings( edited_by: Literal["me"] | None = Query( None, description="me: recordings you edited (corrected the transcript, changed the catalogue record or renamed)" ), + sensitive: Literal["include", "only", "exclude"] | None = Query( + None, + description="where NSFW classification is on (docs/nsfw.md): only those flagged NSFW, exclude them, or include them. " + "Without it, flagged ones are left out where their namespace hides them (display hide)", + ), collection: int | None = Query(None, description="a collection: the recordings in it and in the collections inside it"), field: int | None = Query(None, description="a custom field (of resources): the recordings with a value for it, or with `value`"), value: str | None = Query( @@ -195,6 +204,7 @@ def list_recordings( cfg=cfg, field=by_field, objects=object, + sensitive=sensitive, ) response.headers["X-Total-Count"] = str(total) for r in rows: @@ -234,6 +244,14 @@ def list_objects(acl: Acl, user: CurrentUser, db: Db, ns: str | None = Query(Non return [ObjectCount.model_validate(x) for x in library.object_counts(db, spaces, within)] +@router.get("/nsfw") +def nsfw_counts(acl: Acl, user: CurrentUser, db: Db, cfg: Cfg, ns: str | None = Query(None, description="one namespace")) -> NsfwCounts: + """NSFW classification in the recordings you can read (or one namespace's): whether it's on in any of their + namespaces, how many are flagged, and how many of those the Library leaves out (docs/nsfw.md).""" + spaces, within = acl.scope(ns) + return NsfwCounts.model_validate(library.nsfw_counts(db, cfg, spaces, within)) + + @router.post("/collection") def place_recordings(body: RecordingsPlace, acl: Acl, user: Writer, db: Db, cfg: Cfg) -> Placed: """Move recordings into a collection of their namespace: editors of each recording (through the namespace or the @@ -293,9 +311,29 @@ def get_recording(rid: int, acl: Acl, db: Db, cfg: Cfg) -> Recording: d["attached_to"] = _attached_to(acl, db, r.get("attached_to")) if r.get("enrichment") and acl.member(r): # where a photo was taken: members only d["enrichment"] = {k: v for k, v in r["enrichment"].items() if k != "photo_fp"} + d["nsfw"] = _nsfw(db, cfg, r, rid) return Recording.model_validate(sign_urls(d, full=True)) +def _nsfw(db: DB, cfg: dict[str, Any], rec: dict[str, Any], rid: int) -> dict[str, Any] | None: + """A resource's NSFW classification under its namespace's policy, with its highest-scoring pictures.""" + b = nsfw.brief(rec.get("nsfw"), nsfw.policy(db, cfg, rec["space"])) + return {**b, "hits": nsfw.hits(rec.get("nsfw"), rid)} if b else None + + +@router.put("/{rid}/nsfw") +def mark_recording_nsfw(rid: int, body: RecordingNsfwUpdate, acl: Acl, user: Writer, db: Db, cfg: Cfg) -> NsfwState | None: + """Mark a resource NSFW or not by hand (editors), where NSFW classification is on in its namespace; your call wins + over the model's score until you send null. 409 where classification is off. Audited as `recording.nsfw`.""" + acl.recording(rid, "editor") + try: + got = nsfw.mark(db, cfg, rid, body.flagged, user.email) + except nsfw.Off as e: + raise HTTPException(409, str(e)) from None + auth.audit(db, user.as_audit(), "recording.nsfw", f"recording:{rid}", {"flagged": body.flagged}) + return NsfwState.model_validate(got) if got else None + + def _attached_to(acl: Access, db: DB, link: dict[str, Any] | None) -> dict[str, Any] | None: """The email an attachment came from, with its title, when the caller may see that email.""" if not link or not link.get("resource"): @@ -512,7 +550,9 @@ def decline_access_request(rid: int, account: int, acl: Acl, user: Writer, db: D def get_player(rid: int, acl: Acl, db: Db, cfg: Cfg, s: str = "") -> Player: """Player data. Works with a share link (``?s=``) as well as signed in; media links in it are signed.""" rec = acl.recording(rid, share=s) - return sign_urls(render.player_data(db, rid, audio_link(db, cfg, rid, s)), full=acl.member(rec)) + data = render.player_data(db, rid, audio_link(db, cfg, rid, s)) + data["nsfw"] = _nsfw(db, cfg, rec, rid) + return sign_urls(data, full=acl.member(rec)) @router.get("/{rid}/embed-link") diff --git a/fastapi_backend/app/api/v1/routes/search.py b/fastapi_backend/app/api/v1/routes/search.py index 47b1ce016..9810d39f1 100644 --- a/fastapi_backend/app/api/v1/routes/search.py +++ b/fastapi_backend/app/api/v1/routes/search.py @@ -9,7 +9,7 @@ from app.api.deps import Acl, Cfg, CurrentUser, Db from app.api.media import sign_urls from app.domain import graph as graphmod -from app.domain import graph_history, hierarchy, photos, render, similar, store +from app.domain import graph_history, hierarchy, nsfw, photos, render, similar, store from app.domain import search as searchmod from app.schemas.search import Graph, Mention, PhotoResults, SearchResults, SimilarResults, TermSuggestion @@ -69,6 +69,9 @@ def search_transcripts( cfg=cfg, mode=mode, ) + blur = nsfw.blurred(db, cfg, [h.get("recording_id") for h in res.get("hits") or []]) + for h in res.get("hits") or []: + h["nsfw"] = h.get("recording_id") in blur return sign_urls(res, full=True) @@ -104,9 +107,11 @@ def search_photos( x["recording"]: x.get("thumb") for x in db.rows("SELECT recording, thumb FROM page WHERE recording IN $r AND idx = 0", r=rids) } names = store.space_names(db) + blur = nsfw.blurred(db, cfg, rids) out = [ { **h, + "nsfw": h["recording"] in blur, "title": recs.get(h["recording"], {}).get("title"), "namespace": names.get(recs.get(h["recording"], {}).get("space")), "thumb": f"{store.API}/recordings/{h['recording']}/frames/{thumbs[h['recording']]}" if thumbs.get(h["recording"]) else None, diff --git a/fastapi_backend/app/domain/components.py b/fastapi_backend/app/domain/components.py index 16537d8cb..375626a68 100644 --- a/fastapi_backend/app/domain/components.py +++ b/fastapi_backend/app/domain/components.py @@ -69,6 +69,12 @@ "0ba9fbfa01b5270c96627c4ef784da859931e02f04419c829e83484087c34e79", 39, ), + # AdamCodd/vit-base-nsfw-detector (Apache-2.0), its author's 8-bit ONNX export, at a fixed revision (nsfw.py) + "vit-base-nsfw-detector-int8.onnx": ( + "https://huggingface.co/AdamCodd/vit-base-nsfw-detector/resolve/8587de998f441aac03fdd57a85d2e4cb808c7d64/onnx/model_quantized.onnx", + "432763a6899ebc418c55f784b98f90565e5fc694c778d2ffcb0294b12f6a7404", + 89, + ), } @@ -272,6 +278,10 @@ def serves(self, cfg, m): """The job steps that wait for it: what makes it fetched on first use.""" return self.steps + def wanted_by(self, db, cfg, m): + """needed(), for a component whose need can also come from what's in the database (a namespace's own choice).""" + return self.needed(cfg, m) + def advice(self): """How to get it when it isn't here, for the app.""" return self.hint @@ -433,6 +443,32 @@ def fetch(self, cfg, m, say): fetch_file(cfg, "yolox_s.onnx", say) +class Nsfw(Component): + """The NSFW model and ONNX Runtime (nsfw.py), when Settings → NSFW or a namespace turns classification on.""" + + kind = "model" + + def needed(self, cfg, m): + from . import nsfw + + return nsfw.wanted(None, cfg) + + def wanted_by(self, db, cfg, m): + from . import nsfw + + return nsfw.wanted(db, cfg) + + def present(self, cfg): + from . import nsfw + + return importable("onnxruntime") and bool(nsfw.model_path(cfg)) + + def fetch(self, cfg, m, say): + if not importable("onnxruntime"): + pip_install(cfg, "objects", say) # the objects extra is ONNX Runtime alone + fetch_file(cfg, "vit-base-nsfw-detector-int8.onnx", say) + + class InjectionModel(Component): """The prompt-injection classifier (injection_model.py), fetched when ai.injection_check asks for it.""" @@ -695,6 +731,14 @@ def detail(self, cfg): VOICES, Faces("faces", "Face models", "finds and recognises faces in video (YuNet and SFace)", steps={"faces"}, size_mb=80), Objects("objects", "Object model", "finds objects in frames and pages (YOLOX-s)", steps={"objects"}, size_mb=50), + Nsfw( + "nsfw", + "NSFW model", + "tells whether frames and pages are safe for work (nsfw.enabled, or a namespace's own switch)", + steps={"nsfw"}, + size_mb=105, + license="Apache-2.0", + ), LAYA, InjectionModel( "injection-model", @@ -753,7 +797,7 @@ def wanted(self, cfg, m): continue if c.steps and not c.steps & self.can and c.id not in also: continue - if c.id in also or c.needed(cfg, m): + if c.id in also or c.wanted_by(self.db, cfg, m): out.append(c) return out @@ -894,14 +938,14 @@ def _poked(db, since): return bool(at and at[0] and at[0] > since) -def catalog(cfg): +def catalog(cfg, db=None): """Every component, for the app: what it's for, whether this server's settings need it, and (for programs) whether the API's machine has it.""" m = machine.probe(cfg["data_dir"]) out = [] for c in COMPONENTS: try: - needed = c.needed(cfg, m) + needed = c.wanted_by(db, cfg, m) except Exception: # noqa: BLE001 needed = False row = { diff --git a/fastapi_backend/app/domain/jobs.py b/fastapi_backend/app/domain/jobs.py index 1a61594f2..d3a9479ca 100644 --- a/fastapi_backend/app/domain/jobs.py +++ b/fastapi_backend/app/domain/jobs.py @@ -17,14 +17,14 @@ from . import activity, analyze, ingest, keyring, pipelines, render, speakers as spk, store, telemetry R = store.R -PIPELINE = ["transcribe", "diarize", "shots", "ocr", "faces", "objects", "describe", "analyze", "embed", "summarize", "report"] +PIPELINE = ["transcribe", "diarize", "shots", "ocr", "faces", "objects", "nsfw", "describe", "analyze", "embed", "summarize", "report"] AFTER_IMPORT = ["analyze", "embed", "summarize", "report"] ACTIVE = ["queued", "running", "held"] # held: over budget, waiting for someone to pick (budgets.py) FIELDS = ( "record::id(id) AS id, recording, space, batch, pipeline, steps, step_index, next_step, status, worker, error, attempts, " "created_by, created_at, started_at, finished_at, updated_at, cancel_requested, log_total, cost_usd, tokens, cost_estimate, hold" ) -MEDIA_STEPS = {"transcribe", "diarize", "shots", "ocr", "faces", "objects", "describe"} # they take longer the longer the recording +MEDIA_STEPS = {"transcribe", "diarize", "shots", "ocr", "faces", "objects", "nsfw", "describe"} # they take longer the longer the recording FILED = ("audio", "document", "image") # sources with a file of their own for the steps to work on TIMINGS = 25 # recent timings kept per kind of step, for estimates @@ -169,6 +169,12 @@ def _objects(db, cfg, rid, say, spec=None): objects.step_objects(db, cfg, rid, say) +def _nsfw(db, cfg, rid, say, spec=None): + from . import nsfw + + nsfw.step(db, cfg, rid, say) + + def _describe(db, cfg, rid, say, spec=None): from . import descriptions @@ -184,6 +190,7 @@ def _describe(db, cfg, rid, say, spec=None): "ocr": _ocr, "faces": _faces, "objects": _objects, + "nsfw": _nsfw, "describe": _describe, "analyze": _analyze, "embed": _embed, @@ -904,6 +911,8 @@ def __init__(self, db, cfg_fn, name=None, steps=None, log=None): self.can |= VIDEO_STEPS if self.can & {"analyze", "summarize"}: # embedding needs what they need; lists written before it existed run it too self.can.add("embed") + if not steps and "objects" in self.can: # NSFW classification runs on ONNX Runtime too; older lists run it as well + self.can.add("nsfw") if not steps and log and (missing := sorted(set(STEPS) - self.can)): log(f"worker {self.name}: workers.steps leaves out {', '.join(missing)}; jobs with those steps wait for another worker") self.was_paused = False diff --git a/fastapi_backend/app/domain/library.py b/fastapi_backend/app/domain/library.py index f292e59c9..9609795d0 100644 --- a/fastapi_backend/app/domain/library.py +++ b/fastapi_backend/app/domain/library.py @@ -8,7 +8,7 @@ import re from collections import Counter, defaultdict -from . import access as acc, metadata, render, store +from . import access as acc, metadata, nsfw, render, store R = store.R STATUSES = ("new", "transcribed", "diarized", "analyzed", "error") @@ -26,7 +26,7 @@ } FIELDS = ( "record::id(id) AS id, title, recorded_at, duration_ms, status, error, source, stats, summary, space, collection, media, " - "access, access_parts, featured, tags, language, path, remote, engine" + "access, access_parts, featured, tags, language, path, remote, engine, nsfw" ) # where a recording came from (origin_of): a connected source (source:), or one of these ORIGINS = { @@ -195,6 +195,7 @@ def where( cfg=None, field=None, objects=None, + sensitive=None, ): """The WHERE clause and its parameters for these filters. Filters combine with AND, the values of one filter with OR. @@ -207,6 +208,8 @@ def where( (edited_by). collections: collection ids (a collection and the ones inside it). within: {namespace: collection ids} for namespaces someone sees only some collections of (in_scope). field: (condition, value) on a custom field, from fields.filter_condition. objects: any of these kinds of object seen in it (ignoring case; objects.py). + sensitive (nsfw.py, where NSFW classification is on): only those flagged NSFW, exclude them, or include them; + without it (and with cfg), flagged ones are left out where their namespace hides them. """ base, p = in_scope(spaces, within) w = [base] @@ -299,6 +302,19 @@ def where( if kinds: p["kinds"] = kinds w.append("objects CONTAINSANY $kinds") + if sensitive not in (None, "include", "only", "exclude"): + raise ValueError("sensitive is include, only or exclude") + if sensitive in ("only", "exclude") or (sensitive is None and cfg is not None): + on, hide = nsfw.flagged_spaces(db, cfg or {}, sorted({*spaces, *(within or {})})) + if sensitive == "only": + p["nsfw_on"] = on + w.append("(nsfw.flagged = true AND space IN $nsfw_on)") + elif sensitive == "exclude" and on: + p["nsfw_on"] = on + w.append("(nsfw.flagged != true OR space NOTINSIDE $nsfw_on)") + elif sensitive is None and hide: + p["nsfw_hide"] = hide + w.append("(nsfw.flagged != true OR space NOTINSIDE $nsfw_hide)") if edited is not None: p["edited"] = recs(sorted(edited)) w.append("id IN $edited") @@ -374,14 +390,17 @@ def summaries(db, rows, cfg=None): ) origins = Origins(db, cfg) if cfg is not None and rows else None came = {r["id"]: origins.of(r) for r in rows} if origins else {} + flags = nsfw.briefs(db, cfg, rows) called = origins.names(set(came.values())) if origins else {} out = [] for r in rows: if origins: r["origin"] = came[r["id"]] r["origin_name"] = called[r["origin"]] - for k in ("_k", "_none", "access_parts", "path", "remote", "engine"): + for k in ("_k", "_none", "access_parts", "path", "remote", "engine", "nsfw"): r.pop(k, None) + if flags.get(r["id"]): + r["nsfw"] = flags[r["id"]] a = access[r["id"]] r.update(access=a["access"], open=a["open"], featured=a["featured"], tags=r.get("tags") or []) st, sm = r.pop("stats", None) or {}, r.pop("summary", None) or {} @@ -462,6 +481,17 @@ def object_counts(db, spaces, within=None): return [{"object": k, "recordings": n} for k, n in sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))] +def nsfw_counts(db, cfg, spaces, within=None): + """NSFW classification in these namespaces' recordings (and those of the collections in `within`): whether it's on + in any of them, how many are flagged, and how many of those the Library leaves out (display hide).""" + on, hide = nsfw.flagged_spaces(db, cfg, sorted({*spaces, *(within or {})})) + if not on: + return {"on": False, "flagged": 0, "hidden": 0} + cond, p = in_scope(spaces, within) + rows = db.values(f"SELECT VALUE space FROM recording WHERE {cond} AND nsfw.flagged = true AND space IN $on", on=on, **p) + return {"on": True, "flagged": len(rows), "hidden": sum(1 for s in rows if s in hide)} + + def tag_counts(db, spaces, within=None): """The tags on these namespaces' recordings (and those of the collections in `within`) with how many recordings have each, most used first; each spelled as most of its recordings spell it.""" diff --git a/fastapi_backend/app/domain/moving.py b/fastapi_backend/app/domain/moving.py index 6cb5151f1..8019f5041 100644 --- a/fastapi_backend/app/domain/moving.py +++ b/fastapi_backend/app/domain/moving.py @@ -29,6 +29,7 @@ keyring, metadata as md, notebook, + nsfw, render, sources, speakers as spk, @@ -278,6 +279,7 @@ def _move(db, cfg, rid, dst, rec, names, home, rediarize, revoke_shares, by): deletion.orphans(db, old_speakers, old_faces) # unnamed ones nothing else has any more job = jobs.enqueue(db, rid, (["diarize"] if rediarize else []) + ["analyze", "embed", "report"], by=by) md.touched(db, cfg, rid) # harvesters see an Update: the manifest's collection changed + nsfw.reflag(db, cfg, [dst], rid) # flagged NSFW or not by the new namespace's threshold return { "title": rec.get("title"), "from": names[src], diff --git a/fastapi_backend/app/domain/nsfw.py b/fastapi_backend/app/domain/nsfw.py new file mode 100644 index 000000000..f9c38fb3c --- /dev/null +++ b/fastapi_backend/app/domain/nsfw.py @@ -0,0 +1,372 @@ +"""NSFW classification (docs/nsfw.md): a local model scores how likely each picture of a resource is to be not safe +for work (nudity and sexual content), and a resource with a picture at or over the threshold is flagged, so the web +app can blur it or leave it out of the Library. + +Off by default. An admin turns it on for every namespace (Settings → NSFW, `nsfw.enabled`), or a namespace's owner +for theirs (PUT /namespaces/{name}/nsfw), which can also have its own threshold and display, or stay off while the rest +is on. The nsfw step then scores a video's sampled frames, a document's pages or an image, on this machine: no picture +leaves it, and no model provider is called. + +The model is AdamCodd/vit-base-nsfw-detector (Apache-2.0): a Vision Transformer fine-tuned to tell NSFW from SFW +pictures, as its author's 8-bit ONNX export (88 MB), run by ONNX Runtime on the CPU. components.py fetches both on first +use: the first time a job waits at the nsfw step with classification on somewhere. It was trained to be strict +("sexy" pictures count as NSFW) and is less sure of generated pictures, so a person can mark a resource NSFW or not by +hand, and their call wins over the score. + +What a resource keeps (recording.nsfw): `score`, the highest of its pictures'; `flagged`, the person's call (`manual`) +if they made one, else score >= its namespace's threshold (kept in step when the threshold changes: reflag); `hits`, +the pictures that scored highest; `checked` and `of`, how many pictures were scored out of how many it has; `model` +and `at`. Where classification is off, what was kept stays but nothing acts on it. +""" + +from __future__ import annotations + +import pathlib +import threading + +import numpy as np + +from . import jobs, keyring, store + +R = store.R +MODEL = "vit-base-nsfw-detector-int8.onnx" # components.FILES fetches it into the data folder's models +MODEL_NAME = "vit-base-nsfw-detector" +MODELS = pathlib.Path("/opt/lens/models") # models put there by hand, or baked into an image +LABELS = ("sfw", "nsfw") # the model's id2label +SIZE = 384 # its input, when the model doesn't say +DISPLAYS = ("blur", "hide", "show") +THRESHOLD = (0.5, 0.99) +MAX_FRAMES = (1, 5000) +HITS = 12 # the highest-scoring pictures kept per resource +PICTURED_WHERE = "(source IN ['document', 'image'] OR media.kind = 'video')" # resources with pictures to classify +_SESSIONS: dict[str, object] = {} +_LOCK = threading.Lock() + + +class Off(ValueError): + """Classification is off in this namespace.""" + + +# ---------- who it's on for ---------- +def own(db, sid): + """What a namespace chose for itself: {enabled, threshold, display}, each only when it chose.""" + return dict((db.one("SELECT nsfw FROM $s", s=R("space", int(sid))) or {}).get("nsfw") or {}) + + +def _merge(cfg, mine): + g = cfg.get("nsfw") or {} + on = mine.get("enabled") + return { + "enabled": bool(g.get("enabled")) if on is None else bool(on), + "threshold": float(mine.get("threshold") or g.get("threshold") or 0.8), + "display": mine.get("display") or g.get("display") or "blur", + "max_frames": int(g.get("max_frames") or 300), + } + + +def policy(db, cfg, sid): + """What applies in a namespace: its own choices, else the archive's (`nsfw` settings).""" + return _merge(cfg, own(db, sid)) + + +def policies(db, cfg, sids): + """{namespace id: policy} for many namespaces, in one query.""" + sids = sorted({int(s) for s in sids if s is not None}) + if not sids: + return {} + rows = db.rows("SELECT record::id(id) AS id, nsfw FROM space WHERE id IN $ids", ids=[R("space", s) for s in sids]) + mine = {r["id"]: dict(r.get("nsfw") or {}) for r in rows} + return {s: _merge(cfg, mine.get(s) or {}) for s in sids} + + +def any_namespace(db): + """Whether a namespace turned it on for itself.""" + try: + return bool(db.values("SELECT VALUE id FROM space WHERE nsfw.enabled = true LIMIT 1")) + except Exception: # noqa: BLE001 - asked again on the next pass + return False + + +def wanted(db, cfg): + """Whether it's on anywhere: what has the model fetched (components.py).""" + return bool((cfg.get("nsfw") or {}).get("enabled")) or (db is not None and any_namespace(db)) + + +def check_threshold(value, name="threshold"): + lo, hi = THRESHOLD + if not (isinstance(value, (int, float)) and not isinstance(value, bool) and lo <= value <= hi): + raise ValueError(f"{name} is a number from {lo} to {hi}") + return round(float(value), 3) + + +def check_display(value, name="display"): + if value not in DISPLAYS: + raise ValueError(f"{name} is one of {', '.join(DISPLAYS)}") + return value + + +def save_own(db, sid, changes): + """Change what a namespace chose for itself; a key set to None goes back to the archive's setting.""" + mine = own(db, sid) + for k, v in (changes or {}).items(): + if k not in ("enabled", "threshold", "display"): + raise ValueError(f"there's no NSFW setting called {k}") + if v is None: + mine.pop(k, None) + elif k == "enabled": + if not isinstance(v, bool): + raise ValueError("enabled is true or false") + mine[k] = v + else: + mine[k] = check_threshold(v) if k == "threshold" else check_display(v) + db.q("UPDATE $s SET nsfw = $n", s=R("space", int(sid)), n=mine or None) + return mine + + +def reflag(db, cfg, sids=None, rid=None): + """Keep `flagged` in step with each namespace's threshold (after it changes, or a resource moves; `rid`: only that + one): a person's own call stays as it is.""" + spaces = sids if sids is not None else db.values("SELECT VALUE record::id(id) FROM space") + one = " AND id = $r" if rid is not None else "" + for sid, p in policies(db, cfg, spaces).items(): + db.q( + "UPDATE recording SET nsfw.flagged = nsfw.score >= $t WHERE space = $s AND nsfw.score != NONE AND nsfw.manual = NONE" + one, + s=sid, + t=p["threshold"], + r=R("recording", rid) if rid is not None else None, + ) + + +def brief(n, p): + """A resource's classification as the API shows it, under its namespace's policy; None where it's off or the + resource wasn't classified.""" + if not n or not p or not p.get("enabled") or (n.get("score") is None and n.get("manual") is None): + return None + flagged = bool(n.get("flagged")) + return store.clean( + { + "flagged": flagged, + "score": n.get("score"), + "manual": n.get("manual"), + "blur": flagged and p["display"] != "show", + "display": p["display"], + "threshold": p["threshold"], + "checked": n.get("checked"), + "of": n.get("of"), + "at": n.get("at"), + } + ) + + +def briefs(db, cfg, rows): + """{recording id: brief} for rows with `id`, `space` and `nsfw` (only those that have one).""" + ps = policies(db, cfg, {r["space"] for r in rows if r.get("nsfw")}) if cfg is not None else {} + out = {} + for r in rows: + b = brief(r.get("nsfw"), ps.get(r["space"])) + if b: + out[r["id"]] = b + return out + + +def blurred(db, cfg, rids): + """Which of these resources the web app blurs: flagged, where classification is on and doesn't show them as they are.""" + rids = sorted({int(r) for r in rids if r is not None}) + if not rids: + return set() + rows = db.rows( + "SELECT record::id(id) AS id, space, nsfw FROM recording WHERE id IN $r AND nsfw.flagged = true", + r=[R("recording", x) for x in rids], + ) + return {rid for rid, b in briefs(db, cfg, rows).items() if b.get("blur")} + + +def flagged_spaces(db, cfg, spaces): + """(namespaces where it's on, those that hide flagged resources) among `spaces`.""" + ps = policies(db, cfg, spaces) + on = sorted(s for s, p in ps.items() if p["enabled"]) + return on, sorted(s for s in on if ps[s]["display"] == "hide") + + +def mark(db, cfg, rid, flagged, user=None): + """A person's own call: NSFW (True), not (False), or None to go back to the score.""" + rec = db.one("SELECT space, nsfw FROM $r", r=R("recording", rid)) + if not rec: + raise KeyError(rid) + p = policy(db, cfg, rec["space"]) + if not p["enabled"]: + raise Off("NSFW classification is off in this namespace") + n = dict(rec.get("nsfw") or {}) + if flagged is None: + for k in ("manual", "manual_by", "manual_at"): + n.pop(k, None) + if n.get("score") is None: + n = {} + else: + n["flagged"] = n["score"] >= p["threshold"] + else: + n.update(manual=bool(flagged), flagged=bool(flagged), manual_by=user, manual_at=store.now()) + db.q("UPDATE $r SET nsfw = $n", r=R("recording", rid), n=store.clean(n) or None) + return brief(n, p) + + +def queue_unclassified(db, sid, by=None): + """Queue the nsfw step for a namespace's videos, documents and images that weren't classified yet.""" + rows = db.values(f"SELECT VALUE record::id(id) FROM recording WHERE space = $s AND nsfw.score = NONE AND {PICTURED_WHERE}", s=int(sid)) + return [jobs.add_steps(db, rid, ["nsfw"], by=by) for rid in rows] + + +def _count(db, where, **params): + got = db.values(f"RETURN array::len((SELECT VALUE id FROM recording WHERE {where}))", **params) + return int(got[0]) if got else 0 + + +def status(db, cfg, sid): + """A namespace's classification: what applies, what it chose itself, the archive's settings, and how far it got.""" + g, mine = cfg.get("nsfw") or {}, own(db, sid) + p = _merge(cfg, mine) + here = "space = $s AND " + PICTURED_WHERE + return { + "enabled": p["enabled"], + "threshold": p["threshold"], + "display": p["display"], + "own": mine, + "archive": {k: g.get(k) for k in ("enabled", "threshold", "display")}, + "classified": _count(db, here + " AND nsfw.score != NONE", s=int(sid)), + "flagged": _count(db, here + " AND nsfw.flagged = true", s=int(sid)), + "unclassified": _count(db, here + " AND nsfw.score = NONE", s=int(sid)), + } + + +# ---------- the model ---------- +def model_path(cfg): + """The model file: one put in MODELS by hand, else the one components.py fetched; None without one.""" + from . import components + + for d in (MODELS, components.models_dir(cfg)): + p = d / MODEL + if p.is_file(): + return str(p) + return None + + +def _session(path): + import onnxruntime as ort + + with _LOCK: # one per process: loading 88 MB for every resource would take longer than scoring it + if path not in _SESSIONS: + _SESSIONS[path] = ort.InferenceSession(path, providers=["CPUExecutionProvider"]) + return _SESSIONS[path] + + +def _pixels(img, size): + """The picture as the model was trained on it: RGB, resized to its input, scaled to [-1, 1], channels first.""" + from PIL import Image + + x = np.asarray(img.convert("RGB").resize((size[1], size[0]), Image.BILINEAR), dtype=np.float32) / 255.0 + return ((x - 0.5) / 0.5).transpose(2, 0, 1)[None] + + +class VitOnnx: + name = MODEL_NAME + + def __init__(self, cfg): + path = model_path(cfg) + if not path: + raise RuntimeError("the NSFW model isn't here yet (it's fetched the first time a job needs it)") + try: + import onnxruntime # noqa: F401 + except ImportError: + raise RuntimeError("ONNX Runtime isn't installed (it's fetched on first use, or pip install \"lens[objects]\")") from None + self.session = _session(path) + inp = self.session.get_inputs()[0] + self.input = inp.name + h, w = (list(inp.shape) + [None, None, None, None])[2:4] + self.size = (h if isinstance(h, int) else SIZE, w if isinstance(w, int) else SIZE) + + def score(self, path): + """How likely the picture at `path` is to be NSFW, from 0 to 1.""" + from PIL import Image + + with Image.open(path) as img: + x = _pixels(img, self.size) + logits = np.asarray(self.session.run(None, {self.input: x})[0][0], dtype=np.float64) + p = np.exp(logits - logits.max()) + return float(p[LABELS.index("nsfw")] / p.sum()) + + +def engine(cfg): + """(the classifier, None), or (None, why there's none).""" + try: + return VitOnnx(cfg), None + except RuntimeError as e: + return None, str(e) + + +def spread(items, n): + """At most n of the items, evenly spread from the first to the last.""" + if n <= 0 or len(items) <= n: + return list(items) + if n == 1: + return [items[0]] + return [items[round(i * (len(items) - 1) / (n - 1))] for i in range(n)] + + +# ---------- the step ---------- +def step(db, cfg, rid, say): + """The nsfw step: score a video's sampled frames, or a document's or an image's pages, and keep the highest.""" + from . import documents, video + + rec = db.one("SELECT space, source, media, samples, sample_ms, nsfw FROM $r", r=R("recording", rid)) or {} + p = policy(db, cfg, rec.get("space")) + if not p["enabled"]: + raise jobs.Skip("NSFW classification is off in this namespace (Settings → NSFW, or the namespace's page)") + paged = rec.get("source") in ("document", "image") + if not paged and (rec.get("media") or {}).get("kind") != "video": + raise jobs.Skip("it has no pictures (it isn't a video, a document or an image)") + if paged: + frames = [[pg["idx"], pg["image"]] for pg in documents.pages(db, rid) if pg.get("image")] + if not frames: + raise jobs.Skip("its pages weren't drawn") + else: + frames = rec.get("samples") or [] + if not frames: + raise jobs.Skip("it has no sampled frames yet (the shots step samples them)") + found, why = engine(cfg) + if not found: + raise jobs.Skip(why) + look, d, scored = spread(frames, p["max_frames"]), video.frames_dir(cfg, rid), [] + for t, name in look: + with keyring.plain_picture(db, cfg, d / name) as pic: + scored.append((t, name, found.score(pic))) + top = sorted(scored, key=lambda x: -x[2])[:HITS] + best = top[0][2] + manual = (rec.get("nsfw") or {}).get("manual") + flagged = manual if manual is not None else best >= p["threshold"] + keep = {k: v for k, v in (rec.get("nsfw") or {}).items() if k in ("manual", "manual_by", "manual_at")} + n = store.clean( + { + **keep, + "score": round(best, 4), + "flagged": bool(flagged), + "hits": [{"t": t, "frame": name, "score": round(s, 4)} for t, name, s in top], + "checked": len(look), + "of": len(frames), + "paged": paged or None, + "model": found.name, + "at": store.now(), + } + ) + db.q("UPDATE $r SET nsfw = $n", r=R("recording", rid), n=n) + what = "page(s)" if paged else "frame(s)" + of = f" of {len(frames)}" if len(look) < len(frames) else "" + verdict = "flagged NSFW" if flagged else "not NSFW" + say(f"{verdict}: highest score {best:.2f} (threshold {p['threshold']}) over {len(look)}{of} {what} ({found.name})") + + +def hits(n, rid): + """A resource's highest-scoring pictures, with links to them (signed by the route).""" + return [ + {"t": h["t"], "score": h["score"], "frame": f"{store.API}/recordings/{rid}/frames/{h['frame']}"} + for h in (n or {}).get("hits") or [] + ] diff --git a/fastapi_backend/app/domain/pipelines.py b/fastapi_backend/app/domain/pipelines.py index b5d519ed6..f0ab6a5df 100644 --- a/fastapi_backend/app/domain/pipelines.py +++ b/fastapi_backend/app/domain/pipelines.py @@ -1,10 +1,10 @@ """Pipelines: named, versioned lists of steps a recording goes through. Each namespace can choose its default; without one, recordings get the standard pipeline. Steps: transcribe (speech to text), read (a document's or an -image's pages and their text), diarize, analyze, embed (passages for search by meaning, -semantic.py), summarize, llm (a prompt template whose structured result is saved as a named output), report (built in, -or from a report template) and export (a template rendered to a file, optionally copied to a storage source) and -workflow (a workflow graph, workflows.py, that turns what the steps made into metadata). Any step can carry a -condition. +image's pages and their text), diarize, shots, ocr, faces, objects, nsfw (NSFW classification, nsfw.py), describe, +analyze, embed (passages for search by meaning, semantic.py), summarize, llm (a prompt template whose structured +result is saved as a named output), report (built in, or from a report template) and export (a template rendered to +a file, optionally copied to a storage source) and workflow (a workflow graph, workflows.py, that turns what the steps +made into metadata). Any step can carry a condition. Which pipeline a recording gets also depends on its content type (content_types.py): the one chosen for the run, else the namespace's override for the recording's subtype, else the subtype's own pipeline, else the namespace's default, @@ -23,13 +23,14 @@ from . import keyring, llm, render, sources, store, templates R = store.R -STANDARD = ["transcribe", "diarize", "shots", "ocr", "faces", "objects", "describe", "analyze", "embed", "summarize", "report"] +STANDARD = ["transcribe", "diarize", "shots", "ocr", "faces", "objects", "nsfw", "describe", "analyze", "embed", "summarize", "report"] TYPES = { - "transcribe", "read", "diarize", "shots", "ocr", "faces", "objects", "describe", "analyze", "embed", "summarize", "llm", "report", + "transcribe", "read", "diarize", "shots", "ocr", "faces", "objects", "nsfw", "describe", "analyze", "embed", "summarize", "llm", + "report", "export", "workflow", } # fmt: skip -ASSET_STEPS = ("transcribe", "read", "diarize", "shots", "ocr", "faces", "objects", "describe") # they make something of the media +ASSET_STEPS = ("transcribe", "read", "diarize", "shots", "ocr", "faces", "objects", "nsfw", "describe") # they make something of the media KEYS = {"type", "name", "when", "template", "version", "key", "filename", "destination", "model", "force", "workflow"} NODE_RX = re.compile(r"^[A-Za-z0-9_-]{1,40}$") WHEN = {"min_minutes": (int, float), "max_minutes": (int, float), "source": str, "languages": list} diff --git a/fastapi_backend/app/domain/settings.py b/fastapi_backend/app/domain/settings.py index b27ef7080..b222dbcd7 100644 --- a/fastapi_backend/app/domain/settings.py +++ b/fastapi_backend/app/domain/settings.py @@ -99,6 +99,7 @@ "object_engine", "object_min_score", ), + "nsfw": None, "server": ("embed_frame_ancestors", "max_upload_mb", "allowed_hosts", "session_hours", "secure_cookies", "trusted_proxies"), "uploads": None, "files": None, @@ -473,6 +474,8 @@ def _check(section, key, value, default): return _smb_setting(key, value) if section == "components": return _component_setting(key, value) + if section == "nsfw": + return _nsfw_setting(key, value) if section == "mail" and key != "security": return _mail_setting(key, value) if section == "bridge" and key not in ("answer", "enabled", "approve"): @@ -882,6 +885,25 @@ def _component_setting(key, value): raise ValueError(f"unknown setting components.{key}") +def _nsfw_setting(key, value): + from . import nsfw + + if key == "enabled": + if not isinstance(value, bool): + raise ValueError("nsfw.enabled is true or false") + return value + if key == "threshold": + return nsfw.check_threshold(value, "nsfw.threshold") + if key == "display": + return nsfw.check_display(value, "nsfw.display") + if key == "max_frames": + lo, hi = nsfw.MAX_FRAMES + if not (isinstance(value, int) and not isinstance(value, bool) and lo <= value <= hi): + raise ValueError(f"nsfw.max_frames is a whole number from {lo} to {hi}") + return value + raise ValueError(f"unknown setting nsfw.{key}") + + def _decision_setting(key, value): if key == "laya_model": from . import decide diff --git a/fastapi_backend/app/domain/store.py b/fastapi_backend/app/domain/store.py index 146a2a5fa..2478bf383 100644 --- a/fastapi_backend/app/domain/store.py +++ b/fastapi_backend/app/domain/store.py @@ -262,6 +262,7 @@ def labels(): "ocr", "faces", "objects", + "nsfw", "describe", "analyze", "embed", @@ -503,6 +504,11 @@ def labels(): "ultralytics_model": None, "object_min_score": 0.4, }, + # NSFW classification (nsfw.py, docs/nsfw.md): off until an admin turns it on here, or an owner for their namespace + # (which can also choose its own threshold and display). A local model (ViT, Apache-2.0, fetched on first use) scores + # each picture; a resource is flagged when one scores at least the threshold. display is how flagged resources show + # in the web app: blur (click to see), hide (left out of the Library unless asked for) or show. + "nsfw": {"enabled": False, "threshold": 0.8, "display": "blur", "max_frames": 300}, # rclone and local_roots are bootstrap-only on purpose: the web app must not be able to pick an executable # or open up arbitrary folders on the server. Local folders can only be watched inside local_roots. "sources": {"rclone": None, "local_roots": [], "check_seconds": 15, "cache_dir": None}, diff --git a/fastapi_backend/app/schemas/namespaces.py b/fastapi_backend/app/schemas/namespaces.py index 16156ef87..8ae60b3f9 100644 --- a/fastapi_backend/app/schemas/namespaces.py +++ b/fastapi_backend/app/schemas/namespaces.py @@ -68,6 +68,39 @@ class NamespaceUpdate(RequestModel): ) +NsfwDisplay = Literal["blur", "hide", "show"] + + +class NsfwChoices(ResponseModel): + enabled: bool | None = Field(None, description="on or off; null: as the archive's setting says") + threshold: float | None = Field(None, description="flag a picture at least this likely to be NSFW; null: the archive's") + display: NsfwDisplay | None = Field(None, description="blur, hide or show what's flagged; null: the archive's") + + +class NamespaceNsfw(ResponseModel): + """NSFW classification in a namespace (docs/nsfw.md): what applies, what the namespace chose for itself, and the + archive's settings it otherwise follows.""" + + enabled: bool = Field(description="its videos, documents and images are classified by the nsfw step") + threshold: float = Field(description="a resource is flagged when one of its pictures scores at least this (0.5 to 0.99)") + display: NsfwDisplay = Field(description="how flagged resources show in the web app: blurred, left out of the Library, or as they are") + own: NsfwChoices = Field(description="what the namespace chose for itself (owners change it)") + archive: NsfwChoices = Field(description="the archive's settings (Settings → NSFW, admins), for what the namespace didn't choose") + classified: int = Field(0, description="its resources classified so far") + flagged: int = Field(0, description="how many of them are flagged NSFW") + unclassified: int = Field(0, description="its videos, documents and images not classified yet") + queued: int | None = Field(None, description="with classify: how many runs were queued") + + +class NamespaceNsfwUpdate(RequestModel): + """Only the fields you send change; send null to follow the archive's setting again.""" + + enabled: bool | None = None + threshold: float | None = Field(None, ge=0.5, le=0.99) + display: NsfwDisplay | None = None + classify: bool = Field(False, description="also queue the nsfw step for its videos, documents and images not classified yet") + + class IpGroup(ResponseModel): """Address ranges whose visitors see all of some of the namespace's recordings without signing in (docs/access.md).""" diff --git a/fastapi_backend/app/schemas/recordings.py b/fastapi_backend/app/schemas/recordings.py index 91d220439..5611f505c 100644 --- a/fastapi_backend/app/schemas/recordings.py +++ b/fastapi_backend/app/schemas/recordings.py @@ -26,6 +26,36 @@ ] +NsfwDisplay = Literal["blur", "hide", "show"] + + +class NsfwHit(ResponseModel): + """One of a resource's pictures that scored highest.""" + + t: int = Field(description="where it is: ms into a video, or a page number from 0") + score: float = Field(description="how likely it is to be NSFW, from 0 to 1") + frame: str | None = Field(None, description="a signed link to the frame or page") + + +class NsfwState(ResponseModel): + """What NSFW classification found (docs/nsfw.md), where it's on in the resource's namespace.""" + + flagged: bool = Field(description="flagged NSFW: a person's call if they made one, else its highest score is at least the threshold") + score: float | None = Field(None, description="the highest score of its pictures, from 0 (safe for work) to 1; null when only marked") + manual: bool | None = Field(None, description="a person marked it NSFW (true) or not (false); null: the score decides") + blur: bool = Field(False, description="flagged, and its namespace blurs or hides what's flagged: the web app blurs its pictures") + display: NsfwDisplay = Field("blur", description="how its namespace shows what's flagged: blur, hide (left out of the Library) or show") + threshold: float = Field(0.8, description="its namespace's threshold") + checked: int | None = Field(None, description="how many of its frames or pages were scored") + of: int | None = Field(None, description="how many it has") + at: str | None = Field(None, description="when it was classified") + hits: list[NsfwHit] = Field(default_factory=list, description="its highest-scoring pictures (on the resource and its player)") + + +class RecordingNsfwUpdate(RequestModel): + flagged: bool | None = Field(description="true: NSFW; false: not NSFW; null: let the score decide again") + + class RecordingSummary(ResponseModel): id: int title: str | None = None @@ -58,6 +88,7 @@ class RecordingSummary(ResponseModel): role: Role | None = Field( None, description="your role on it: through its namespace or its collection (an admin of the collection is an owner)" ) + nsfw: NsfwState | None = Field(None, description="NSFW classification, where it's on in its namespace and it was classified") class OriginCount(ResponseModel): @@ -76,6 +107,14 @@ class ObjectCount(ResponseModel): recordings: int +class NsfwCounts(ResponseModel): + """NSFW classification in the recordings you can read (docs/nsfw.md).""" + + on: bool = Field(description="it's on in at least one of their namespaces") + flagged: int = Field(0, description="how many are flagged NSFW") + hidden: int = Field(0, description="how many of those the Library leaves out unless asked (their namespace hides them)") + + class ObjectTrack(ResponseModel): """A kind of object seen in a recording, and where.""" @@ -225,6 +264,7 @@ class Recording(ResponseModel): attached_to: AttachedTo | None = Field(None, description="an email's attachment made a resource of its own: that email") web: WebPage | None = Field(None, description="a web page captured as a document") enrichment: Enrichment | None = Field(None, description="the places and dates anytopdf found (analysis.anytopdf)") + nsfw: NsfwState | None = Field(None, description="NSFW classification, where it's on in its namespace and it was classified") class Page(ResponseModel): @@ -275,6 +315,7 @@ class Player(ResponseModel): ) faces_mode: str = Field("off", description="the namespace's face mode: off, detect or recognize") faces_pixelate: bool = Field(False, description="the namespace pixelates faces in the pictures visitors see") + nsfw: NsfwState | None = Field(None, description="NSFW classification, where it's on in its namespace and it was classified") class RecordingUpdate(RequestModel): diff --git a/fastapi_backend/app/schemas/search.py b/fastapi_backend/app/schemas/search.py index 4e5297a12..c52d66ee4 100644 --- a/fastapi_backend/app/schemas/search.py +++ b/fastapi_backend/app/schemas/search.py @@ -39,6 +39,9 @@ class SearchHit(ResponseModel): 'alike; shown at its line that fits best), or "both"', ) similarity: float | None = Field(None, description="found by meaning: how alike the passage is to the query (cosine, up to 1)") + nsfw: bool = Field( + False, description="its recording is flagged NSFW where its namespace blurs or hides that: the web app blurs its frame" + ) class FacetCount(ResponseModel): @@ -122,6 +125,7 @@ class PhotoHit(ResponseModel): namespace: str | None = None similarity: float = Field(description="CLIP's cosine similarity of the photo and the query; a good match is about 0.3") thumb: str | None = Field(None, description="a signed link to it, small") + nsfw: bool = Field(False, description="flagged NSFW where its namespace blurs or hides that: the web app blurs it") class PhotoResults(ResponseModel): diff --git a/fastapi_backend/tests/api/test_jobs.py b/fastapi_backend/tests/api/test_jobs.py index 2270e3896..ca791ccd6 100644 --- a/fastapi_backend/tests/api/test_jobs.py +++ b/fastapi_backend/tests/api/test_jobs.py @@ -375,7 +375,7 @@ def test_a_steps_list_from_before_the_video_steps_runs_them_too(db, cfg): w = jobs.Worker(db, lambda: {**cfg, "workers": {**cfg["workers"], "steps": old}}, name="old", log=said.append) assert w.can == set(jobs.STEPS) and not said w = jobs.Worker(db, lambda: {**cfg, "workers": {**cfg["workers"], "steps": [*old, "shots"]}}, name="chosen", log=said.append) - assert "ocr" not in w.can and "leaves out describe, faces, objects, ocr" in said[0] + assert "ocr" not in w.can and "leaves out describe, faces, nsfw, objects, ocr" in said[0] assert jobs.Worker(db, lambda: cfg, name="mac", steps=["transcribe"], log=said.append).can == {"transcribe"} and len(said) == 1 diff --git a/fastapi_backend/tests/api/test_mcp.py b/fastapi_backend/tests/api/test_mcp.py index 9b9d6d8f7..8f853d9a3 100644 --- a/fastapi_backend/tests/api/test_mcp.py +++ b/fastapi_backend/tests/api/test_mcp.py @@ -77,6 +77,8 @@ def test_clients_are_sent_to_sign_in(client, app, env): "propose_graph_change", "write_note", "create_podcast", + "set_nsfw", + "mark_nsfw", "set_file_storage", "set_monitoring", ] diff --git a/fastapi_backend/tests/api/test_nsfw.py b/fastapi_backend/tests/api/test_nsfw.py new file mode 100644 index 000000000..f60029708 --- /dev/null +++ b/fastapi_backend/tests/api/test_nsfw.py @@ -0,0 +1,267 @@ +"""NSFW classification (docs/nsfw.md): off by default; on for the archive or for one namespace, the nsfw step scores +a resource's pictures with a local model and keeps the highest; resources are flagged by their namespace's threshold +(kept in step when it changes), shown blurred or left out of the Library, marked by hand, classified after the fact, +and reached by agents over MCP. The model is read the way its export expects (ViT: RGB, 384 pixels, [-1, 1]).""" + +from __future__ import annotations + +import os +import shutil + +import numpy as np +import pytest +from PIL import Image + +from app.api import mcp_tools +from app.domain import components, jobs, machine, nsfw, settings, store +from tests.api.test_documents import _png, _upload +from tests.api.test_mcp import tool, tool_error +from tests.helpers import drain, login, make_user + +R = store.R + + +class FakeModel: + """Red pictures are NSFW, green ones nearly, anything else safe.""" + + name = "fake-vit" + + def score(self, path): + r, g, b = Image.open(path).convert("RGB").getpixel((2, 2)) + return 0.93 if r > 150 and g < 100 else 0.6 if g > 150 and r < 100 else 0.04 + + +@pytest.fixture +def model(monkeypatch): + monkeypatch.setattr(nsfw, "engine", lambda cfg: (FakeModel(), None)) + + +@pytest.fixture +def env(client, db): + make_user(db, "root@x.io", "root password 1", admin=True) + make_user(db, "own@x.io", "owner password 1", roles={"pods": "owner"}) + make_user(db, "ed@x.io", "editor password 1", roles={"pods": "editor"}) + make_user(db, "vi@x.io", "viewer password 1", roles={"pods": "viewer"}) + return { + "ha": login(client, "root@x.io", "root password 1"), + "ho": login(client, "own@x.io", "owner password 1"), + "he": login(client, "ed@x.io", "editor password 1"), + "hv": login(client, "vi@x.io", "viewer password 1"), + } + + +def _photo(client, h, colour, name, ns="pods"): + return _upload(client, h, _png(Image.new("RGB", (48, 32), colour)), name, ns=ns)["recording"] + + +def _log(db, rid): + return " ".join(x for j in db.rows("SELECT log FROM job WHERE recording = $r", r=rid) for x in j["log"]) + + +def _ids(client, h, **params): + return {r["id"] for r in client.get("/api/v1/recordings", params=params, headers=h).json()} + + +def test_off_by_default_nothing_is_fetched_or_classified(client, env, db, cfg, model): + assert store.DEFAULTS["nsfw"] == {"enabled": False, "threshold": 0.8, "display": "blur", "max_frames": 300} + rid = _photo(client, env["he"], "red", "a.png") + drain(db, cfg) + assert "nsfw skipped: NSFW classification is off in this namespace" in _log(db, rid) + assert db.one("SELECT nsfw FROM $r", r=R("recording", rid)).get("nsfw") is None + assert client.get(f"/api/v1/recordings/{rid}", headers=env["hv"]).json()["nsfw"] is None + assert not components.BY_ID["nsfw"].wanted_by(db, cfg, machine.probe(cfg["data_dir"])) + got = client.get("/api/v1/namespaces/pods/nsfw", headers=env["hv"]).json() + assert (got["enabled"], got["own"], got["unclassified"]) == (False, {"enabled": None, "threshold": None, "display": None}, 1) + r = client.put(f"/api/v1/recordings/{rid}/nsfw", json={"flagged": True}, headers=env["he"]) + assert r.status_code == 409 and "off" in r.json()["detail"] + assert client.post("/api/v1/namespaces/pods/nsfw/classify", headers=env["he"]).status_code == 409 + assert client.get("/api/v1/recordings/nsfw", headers=env["hv"]).json() == {"on": False, "flagged": 0, "hidden": 0} + + +def test_a_namespace_classifies_flags_blurs_and_hides(client, env, db, cfg, model): + hv, he = env["hv"], env["he"] + assert client.put("/api/v1/namespaces/pods/nsfw", json={"enabled": True}, headers=he).status_code == 403 # owners + got = client.put("/api/v1/namespaces/pods/nsfw", json={"enabled": True}, headers=env["ho"]).json() + assert (got["enabled"], got["own"]["enabled"], got["archive"]["enabled"], got["threshold"], got["display"]) == ( + True, + True, + False, + 0.8, + "blur", + ) + assert components.BY_ID["nsfw"].wanted_by(db, cfg, machine.probe(cfg["data_dir"])) # the model is wanted now + red, white, green = _photo(client, he, "red", "red.png"), _photo(client, he, "white", "white.png"), _photo(client, he, "lime", "g.png") + other = _photo(client, env["ha"], "red", "elsewhere.png", ns="calls") # off there + drain(db, cfg) + assert "flagged NSFW: highest score 0.93 (threshold 0.8) over 1 page(s) (fake-vit)" in _log(db, red) + assert "not NSFW: highest score 0.04" in _log(db, white) + kept = db.one("SELECT nsfw FROM $r", r=R("recording", red))["nsfw"] + assert (kept["score"], kept["flagged"], kept["checked"], kept["of"], kept["paged"], kept["model"]) == ( + 0.93, + True, + 1, + 1, + True, + "fake-vit", + ) + assert kept["hits"][0]["score"] == 0.93 and kept["hits"][0]["t"] == 0 + assert db.one("SELECT nsfw FROM $r", r=R("recording", other)).get("nsfw") is None + + # the resource, its player and the Library say so; the web app blurs it + rec = client.get(f"/api/v1/recordings/{red}", headers=hv).json()["nsfw"] + assert (rec["flagged"], rec["score"], rec["blur"], rec["display"], rec["manual"]) == (True, 0.93, True, "blur", None) + assert "sig=" in rec["hits"][0]["frame"] and client.get(rec["hits"][0]["frame"]).status_code == 200 + assert client.get(f"/api/v1/recordings/{red}/player", headers=hv).json()["nsfw"]["flagged"] is True + rows = {r["id"]: r for r in client.get("/api/v1/recordings", headers=hv).json()} + assert rows[red]["nsfw"]["blur"] is True and rows[white]["nsfw"]["flagged"] is False and rows[green]["nsfw"]["flagged"] is False + assert _ids(client, hv, sensitive="only") == {red} + assert red not in _ids(client, hv, sensitive="exclude") and white in _ids(client, hv, sensitive="exclude") + assert nsfw.blurred(db, cfg, [red, white, other]) == {red} + st = client.get("/api/v1/namespaces/pods/nsfw", headers=hv).json() + assert (st["classified"], st["flagged"], st["unclassified"]) == (3, 1, 0) + + # hidden: left out of the Library unless asked for + client.put("/api/v1/namespaces/pods/nsfw", json={"display": "hide"}, headers=env["ho"]) + assert red not in _ids(client, hv) and white in _ids(client, hv) + assert red in _ids(client, hv, sensitive="include") + assert client.get("/api/v1/recordings/nsfw", headers=hv).json() == {"on": True, "flagged": 1, "hidden": 1} + assert client.get("/api/v1/recordings/nsfw", params={"ns": "calls"}, headers=env["ha"]).json()["on"] is False + client.put("/api/v1/namespaces/pods/nsfw", json={"display": "show"}, headers=env["ho"]) + assert red in _ids(client, hv) + assert client.get(f"/api/v1/recordings/{red}", headers=hv).json()["nsfw"]["blur"] is False # flagged, shown as it is + + # a lower threshold flags the green one at once; a person's call wins over the score + got = client.put("/api/v1/namespaces/pods/nsfw", json={"threshold": 0.55, "display": None}, headers=env["ho"]).json() + assert (got["threshold"], got["display"], got["flagged"]) == (0.55, "blur", 2) + assert _ids(client, hv, sensitive="only") == {red, green} + assert client.put(f"/api/v1/recordings/{green}/nsfw", json={"flagged": False}, headers=hv).status_code == 403 # editors + assert client.put(f"/api/v1/recordings/{green}/nsfw", json={"flagged": False}, headers=he).json()["manual"] is False + client.put("/api/v1/namespaces/pods/nsfw", json={"threshold": 0.5}, headers=env["ho"]) + assert _ids(client, hv, sensitive="only") == {red} # still not, by hand + assert client.put(f"/api/v1/recordings/{green}/nsfw", json={"flagged": None}, headers=he).json()["flagged"] is True + assert client.put(f"/api/v1/recordings/{white}/nsfw", json={"flagged": True}, headers=he).json()["flagged"] is True + + # off again: what was found stays, but nothing acts on it + client.put("/api/v1/namespaces/pods/nsfw", json={"enabled": False}, headers=env["ho"]) + assert client.get(f"/api/v1/recordings/{red}", headers=hv).json()["nsfw"] is None + assert _ids(client, hv, sensitive="only") == set() + + +def test_the_archive_setting_and_a_namespace_staying_off(client, env, db, cfg, model): + ha = env["ha"] + for bad in ({"threshold": 0.2}, {"display": "pixelate"}, {"max_frames": 0}, {"enabled": "yes"}): + assert client.put("/api/v1/settings/nsfw", json=bad, headers=ha).status_code == 400, bad + assert client.put("/api/v1/settings/nsfw", json={"enabled": True}, headers=ha).status_code == 200 + eff = settings.effective(db, cfg) + assert nsfw.policy(db, eff, store.ns_id(db, "calls"))["enabled"] and components.BY_ID["nsfw"].needed(eff, None) + client.put("/api/v1/namespaces/pods/nsfw", json={"enabled": False}, headers=env["ho"]) + pods, calls = _photo(client, env["he"], "red", "p.png"), _photo(client, ha, "red", "c.png", ns="calls") + drain(db, eff) + assert "nsfw skipped" in _log(db, pods) and db.one("SELECT nsfw FROM $r", r=R("recording", calls))["nsfw"]["flagged"] is True + # the archive's threshold re-flags every namespace that follows it + assert client.put("/api/v1/settings/nsfw", json={"threshold": 0.95}, headers=ha).status_code == 200 + assert db.one("SELECT nsfw FROM $r", r=R("recording", calls))["nsfw"]["flagged"] is False + + +def test_what_was_there_before_is_classified_when_asked(client, env, db, cfg, model): + rid = _photo(client, env["he"], "red", "old.png") + drain(db, cfg) + got = client.put("/api/v1/namespaces/pods/nsfw", json={"enabled": True, "classify": True}, headers=env["ho"]).json() + assert (got["queued"], got["unclassified"]) == (1, 1) + drain(db, cfg) + assert db.one("SELECT nsfw FROM $r", r=R("recording", rid))["nsfw"]["flagged"] is True + assert client.post("/api/v1/namespaces/pods/nsfw/classify", headers=env["he"]).json()["jobs"] == [] # nothing left + # moved to a namespace with a stricter threshold, it's flagged by that one + calls = store.ns_id(db, "calls") + nsfw.save_own(db, calls, {"enabled": True, "threshold": 0.99}) + assert client.post(f"/api/v1/recordings/{rid}/move", json={"namespace": "calls"}, headers=env["ha"]).status_code == 200 + assert db.one("SELECT nsfw FROM $r", r=R("recording", rid))["nsfw"]["flagged"] is False + + +def test_agents_turn_it_on_read_and_mark(client, env, db, cfg, model): + he, ho, ha = env["he"], env["ho"], env["ha"] + rid = _photo(client, he, "red", "a.png") + drain(db, cfg) + assert tool(client, env["hv"], "nsfw_settings", namespace="pods")["enabled"] is False + assert tool_error(client, he, "set_nsfw", namespace="pods", enabled=True) # owners + assert not nsfw.policy(db, cfg, store.ns_id(db, "pods"))["enabled"] + assert "only admins" in tool_error(client, ho, "set_nsfw", enabled=True) + got = tool(client, ho, "set_nsfw", namespace="pods", enabled=True, threshold=0.7, classify_existing=True) + assert (got["saved"], got["queued"], got["enabled"], got["threshold"]) == ({"enabled": True, "threshold": 0.7}, 1, True, 0.7) + drain(db, cfg) + assert tool(client, env["hv"], "get_recording", recording_id=rid)["nsfw"] == { + "flagged": True, + "score": 0.93, + "threshold": 0.7, + "checked": 1, + "of": 1, + } + listed = tool(client, env["hv"], "list_recordings", sensitive="only")["recordings"] + assert [(r["recording_id"], r["nsfw"]) for r in listed] == [(rid, True)] + assert tool(client, he, "mark_nsfw", recording_id=rid, verdict="not_nsfw")["nsfw"]["flagged"] is False + assert tool(client, he, "mark_nsfw", recording_id=rid, verdict="model")["nsfw"]["flagged"] is True + assert tool(client, ha, "set_nsfw", display="hide")["saved"] == {"display": "hide"} + assert set(mcp_tools.TOOLS) >= {"nsfw_settings", "set_nsfw", "mark_nsfw"} + + +def test_workers_run_the_step_from_older_lists(db, cfg): + assert "nsfw" in jobs.Worker(db, lambda: cfg).can + older = {**cfg, "workers": {**cfg["workers"], "steps": [s for s in cfg["workers"]["steps"] if s != "nsfw"]}} + assert "nsfw" in jobs.Worker(db, lambda: older).can # lists with objects run it too + assert "nsfw" not in jobs.Worker(db, lambda: cfg, steps=["transcribe"]).can # a list given by name is kept + + +# ---------- the model ---------- +def _model_file(path, dynamic=True): + """A model shaped like the real export (pixel_values → logits [sfw, nsfw]) whose answer follows the red channel.""" + import onnx + from onnx import TensorProto, helper, numpy_helper + + w = np.array([[-4.0, 4.0], [0.0, 0.0], [0.0, 0.0]], np.float32) + nodes = [ + helper.make_node("ReduceMean", ["pixel_values"], ["mean"], axes=[2, 3], keepdims=0), + helper.make_node("Constant", [], ["w"], value=numpy_helper.from_array(w, name="wv")), + helper.make_node("MatMul", ["mean", "w"], ["logits"]), + ] + dims = ["batch_size", 3, "height", "width"] if dynamic else [1, 3, 64, 64] + graph = helper.make_graph( + nodes, + "vit-shaped", + [helper.make_tensor_value_info("pixel_values", TensorProto.FLOAT, dims)], + [helper.make_tensor_value_info("logits", TensorProto.FLOAT, ["batch_size" if dynamic else 1, 2])], + ) + m = helper.make_model(graph, opset_imports=[helper.make_opsetid("", 13)]) + m.ir_version = 8 + onnx.save(m, str(path)) + return str(path) + + +def test_the_model_is_read_as_its_export_expects(cfg, tmp_path, monkeypatch): + monkeypatch.setattr(nsfw, "MODELS", tmp_path / "none") + assert nsfw.engine(cfg) == (None, "the NSFW model isn't here yet (it's fetched the first time a job needs it)") + models = tmp_path / "data" / "models" + models.mkdir(parents=True) + _model_file(models / nsfw.MODEL) + found, why = nsfw.engine(cfg) + assert why is None and found.size == (384, 384) and found.name == "vit-base-nsfw-detector" + for colour, want in (("red", 0.9997), ("blue", 0.0003)): + Image.new("RGB", (90, 60), colour).save(tmp_path / f"{colour}.png") + assert round(found.score(tmp_path / f"{colour}.png"), 4) == want + x = nsfw._pixels(Image.new("RGB", (10, 20), (255, 0, 128)), (8, 4)) + assert x.shape == (1, 3, 8, 4) and [round(float(v), 3) for v in x[0, :, 0, 0]] == [1.0, -1.0, 0.004] + assert components.FILES[nsfw.MODEL][1] == "432763a6899ebc418c55f784b98f90565e5fc694c778d2ffcb0294b12f6a7404" + + +def test_long_resources_are_sampled_evenly(): + assert nsfw.spread(list(range(10)), 4) == [0, 3, 6, 9] + assert nsfw.spread([1, 2], 5) == [1, 2] and nsfw.spread([1, 2, 3], 1) == [1] + + +@pytest.mark.skipif(not os.environ.get("LENS_TEST_NSFW_MODEL"), reason="LENS_TEST_NSFW_MODEL: the real model file") +def test_the_real_model(cfg, tmp_path, monkeypatch): + monkeypatch.setattr(nsfw, "MODELS", tmp_path) + shutil.copy(os.environ["LENS_TEST_NSFW_MODEL"], tmp_path / nsfw.MODEL) + found, why = nsfw.engine(cfg) + assert why is None and found.size == (384, 384) + Image.new("RGB", (640, 480), (40, 90, 160)).save(tmp_path / "sky.jpg") + assert found.score(tmp_path / "sky.jpg") < 0.5 # a plain blue picture is safe for work diff --git a/mkdocs.yml b/mkdocs.yml index b9bdc218b..9610933ca 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -86,6 +86,7 @@ nav: - Sensors: sensors.md - Processing: processing.md - Video: video.md + - NSFW classification: nsfw.md - Notifications: notifications.md - Knowledge: - Graph and Cypher: graph.md diff --git a/nextjs-frontend/__tests__/library-model.test.ts b/nextjs-frontend/__tests__/library-model.test.ts index 334b2fe0a..7c15e7bd7 100644 --- a/nextjs-frontend/__tests__/library-model.test.ts +++ b/nextjs-frontend/__tests__/library-model.test.ts @@ -204,6 +204,7 @@ describe("the list query", () => { objects: ["car", "dog"], collection: 12, field: { id: 7, value: " Lecture " }, + sensitive: "only", }, "all", { key: "duration", dir: "desc" }, @@ -226,6 +227,7 @@ describe("the list query", () => { collection: 12, field: 7, value: "Lecture", + sensitive: "only", }); // a field with any value expect(libraryQuery({ ...NO_FILTERS, field: { id: 7, value: " " } }, "all", byDate, "pods", NOW)).toEqual({ diff --git a/nextjs-frontend/__tests__/nsfw.test.tsx b/nextjs-frontend/__tests__/nsfw.test.tsx new file mode 100644 index 000000000..6dc2f48e6 --- /dev/null +++ b/nextjs-frontend/__tests__/nsfw.test.tsx @@ -0,0 +1,244 @@ +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import "@testing-library/jest-dom"; + +import { Namespaces, Resources } from "@/app/openapi-client"; +import type { NsfwState } from "@/app/openapi-client/types.gen"; +import { NsfwNotice, NsfwPictures, RevealCover, useBlur } from "@/components/nsfw/nsfw"; +import { nsfwText, ownSwitch, parseThreshold, switchValue } from "@/components/nsfw/nsfw-model"; +import { NsfwSection } from "@/components/nsfw/nsfw-section"; +import { RecordingProvider, type RecordingCtx } from "@/components/recording/context"; +import { libraryQuery, NO_FILTERS } from "@/components/library/model"; +import { reprocessOptions } from "@/components/recording/jobs"; +import { TooltipProvider } from "@/components/ui/tooltip"; + +jest.mock("@/app/openapi-client", () => ({ + Namespaces: { + getNamespaceNsfw: jest.fn(), + updateNamespaceNsfw: jest.fn(), + classifyNamespaceNsfw: jest.fn(), + }, + Resources: { markRecordingNsfw: jest.fn() }, +})); +jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) })); +let editor = true; +jest.mock("@/lib/hooks/session", () => ({ + ...jest.requireActual("@/lib/hooks/session"), + useArchive: () => ({ can: () => editor }), +})); +const toast = jest.fn(); +jest.mock("@/components/ui/toast", () => ({ useToast: () => toast })); + +const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } }); +const m = (f: unknown) => f as jest.Mock; +const wrap = (ui: React.ReactNode) => { + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + return render( + + {ui} + , + ); +}; + +const FLAGGED: NsfwState = { + flagged: true, + score: 0.93, + manual: null, + blur: true, + display: "blur", + threshold: 0.8, + checked: 12, + of: 300, +}; + +beforeEach(() => { + jest.clearAllMocks(); + editor = true; +}); + +describe("what the web app says", () => { + it("describes what classification found", () => { + expect(nsfwText(FLAGGED)).toBe("Flagged NSFW: highest score 0.93 (threshold 0.80), 12 of 300 frames scored"); + expect(nsfwText({ ...FLAGGED, flagged: false, score: 0.04, checked: 1, of: 1 }, true)).toBe( + "Not NSFW: highest score 0.04 (threshold 0.80)", + ); + expect(nsfwText({ ...FLAGGED, manual: true, score: null })).toBe("Marked NSFW by hand"); + expect(nsfwText({ ...FLAGGED, flagged: false, manual: false, checked: 2, of: 2 }, true)).toBe( + "Marked not NSFW by hand; the model's highest score 0.93 (threshold 0.80), 2 of 2 pages scored", + ); + }); + + it("reads a namespace's own choices and a typed threshold", () => { + expect([ownSwitch({}), ownSwitch({ enabled: true }), ownSwitch({ enabled: false })]).toEqual([ + "archive", + "on", + "off", + ]); + expect([switchValue("archive"), switchValue("on"), switchValue("off")]).toEqual([null, true, false]); + expect([parseThreshold(""), parseThreshold(" 0.85 "), parseThreshold("0.4"), parseThreshold("x")]).toEqual([ + null, + 0.85, + undefined, + undefined, + ]); + }); + + it("filters the Library and offers the step where there are pictures", () => { + const byDate = { key: "date" as const, dir: "desc" as const }; + expect(libraryQuery({ ...NO_FILTERS, sensitive: "exclude" }, "all", byDate, null)).toEqual({ + sort: "-date", + sensitive: "exclude", + }); + expect(libraryQuery(NO_FILTERS, "all", byDate, null)).not.toHaveProperty("sensitive"); + const keys = (o: Parameters[0]) => reprocessOptions(o).map((x) => x.key); + expect(keys({ video: true, hasAudio: true })).toContain("nsfw"); + expect(keys({ video: false, hasAudio: false, paged: true })).toContain("nsfw"); + expect(keys({ video: false, hasAudio: true })).not.toContain("nsfw"); + }); +}); + +function Picture() { + const blur = useBlur(); + return ( +
+ {/* eslint-disable-next-line @next/next/no-img-element */} + page 1 + +
+ ); +} + +const ctx = (nsfw: NsfwState | null, canEdit = true) => + ({ id: 9, ns: "pods", rec: { id: 9, title: "Beach", nsfw }, paged: false, canEdit }) as unknown as RecordingCtx; + +describe("a flagged resource's page", () => { + it("blurs its pictures until someone shows them, and blurs them again", () => { + wrap( + + + + + + , + ); + expect(screen.getByText(/Flagged NSFW: highest score 0.93/)).toBeInTheDocument(); + expect(screen.getByAltText("page 1")).toHaveClass("blur-xl"); + fireEvent.click(screen.getByRole("button", { name: "Show NSFW pictures" })); + expect(screen.getByAltText("page 1")).not.toHaveClass("blur-xl"); + expect(screen.queryByRole("button", { name: "Show NSFW pictures" })).not.toBeInTheDocument(); + fireEvent.click(screen.getByRole("button", { name: "Blur them" })); + expect(screen.getByAltText("page 1")).toHaveClass("blur-xl"); + }); + + it("shows pictures as they are where the namespace shows flagged ones, or nothing was flagged", () => { + const shown = { ...FLAGGED, blur: false, display: "show" as const }; + wrap( + + + + + + , + ); + expect(screen.getByAltText("page 1")).not.toHaveClass("blur-xl"); + expect(screen.getByText(/Flagged NSFW/)).toBeInTheDocument(); + expect(screen.queryByRole("button", { name: "Show pictures" })).not.toBeInTheDocument(); + }); + + it("editors mark it not NSFW, with Undo", async () => { + m(Resources.markRecordingNsfw).mockReturnValue(ok({ ...FLAGGED, flagged: false, manual: false, blur: false })); + wrap( + + + + + , + ); + fireEvent.click(screen.getByRole("button", { name: "Not NSFW" })); + await waitFor(() => + expect(Resources.markRecordingNsfw).toHaveBeenCalledWith( + expect.objectContaining({ path: { rid: 9 }, body: { flagged: false } }), + ), + ); + await waitFor(() => expect(toast).toHaveBeenCalled()); + const t = toast.mock.calls[0][0]; + expect(t.title).toBe("Marked not NSFW"); + t.action.onClick(); + await waitFor(() => + expect(Resources.markRecordingNsfw).toHaveBeenLastCalledWith( + expect.objectContaining({ body: { flagged: null } }), + ), + ); + }); + + it("viewers can't mark it", () => { + wrap( + + + , + ); + expect(screen.getByRole("button", { name: "Not NSFW" })).toHaveAttribute("aria-disabled", "true"); + }); +}); + +const STATUS = { + enabled: false, + threshold: 0.8, + display: "blur", + own: { enabled: null, threshold: null, display: null }, + archive: { enabled: false, threshold: 0.8, display: "blur" }, + classified: 0, + flagged: 0, + unclassified: 4, +}; + +describe("a namespace's NSFW section", () => { + it("owners turn it on, set the threshold, and editors classify what was there before", async () => { + m(Namespaces.getNamespaceNsfw).mockReturnValue(ok(STATUS)); + const on = { ...STATUS, enabled: true, own: { ...STATUS.own, enabled: true } }; + m(Namespaces.updateNamespaceNsfw).mockReturnValue(ok(on)); + m(Namespaces.classifyNamespaceNsfw).mockReturnValue(ok({ ok: true, jobs: [1, 2, 3, 4] })); + wrap(); + fireEvent.click(await screen.findByRole("radio", { name: /^On/ })); + await waitFor(() => + expect(Namespaces.updateNamespaceNsfw).toHaveBeenCalledWith( + expect.objectContaining({ path: { name: "pods" }, body: { enabled: true } }), + ), + ); + await waitFor(() => + expect(toast).toHaveBeenCalledWith(expect.objectContaining({ title: "NSFW classification is on in pods" })), + ); + // Undo puts back what the namespace had chosen + toast.mock.calls[0][0].action.onClick(); + await waitFor(() => + expect(Namespaces.updateNamespaceNsfw).toHaveBeenLastCalledWith( + expect.objectContaining({ body: { enabled: null, threshold: null, display: null } }), + ), + ); + + m(Namespaces.updateNamespaceNsfw).mockReturnValue( + ok({ ...on, threshold: 0.9, own: { ...on.own, threshold: 0.9 } }), + ); + const box = screen.getByLabelText("Flag at score"); + fireEvent.change(box, { target: { value: "0.3" } }); + expect(screen.getByText("A score from 0.5 to 0.99, or empty")).toBeInTheDocument(); + fireEvent.change(box, { target: { value: "0.9" } }); + fireEvent.submit(box); + await waitFor(() => + expect(Namespaces.updateNamespaceNsfw).toHaveBeenLastCalledWith( + expect.objectContaining({ body: { threshold: 0.9 } }), + ), + ); + + fireEvent.click(await screen.findByRole("button", { name: "Classify existing (4)" })); + await waitFor(() => expect(Namespaces.classifyNamespaceNsfw).toHaveBeenCalled()); + }); + + it("is read-only for people who don't own the namespace", async () => { + editor = false; + m(Namespaces.getNamespaceNsfw).mockReturnValue(ok({ ...STATUS, enabled: true })); + wrap(); + expect(await screen.findByLabelText("Flag at score")).toBeDisabled(); + expect(screen.getByRole("button", { name: "Classify existing (4)" })).toHaveAttribute("aria-disabled", "true"); + }); +}); diff --git a/nextjs-frontend/__tests__/recording-jobs.test.ts b/nextjs-frontend/__tests__/recording-jobs.test.ts index 4757f7d9c..e209adccd 100644 --- a/nextjs-frontend/__tests__/recording-jobs.test.ts +++ b/nextjs-frontend/__tests__/recording-jobs.test.ts @@ -281,18 +281,20 @@ describe("reprocess picker (R9)", () => { "ocr", "faces", "objects", + "nsfw", "describe", "analyze", "embed", "summarize", "report", ]); - // a document's or an image's pages: faces, objects and descriptions, not shots or text on screen + // a document's or an image's pages: faces, objects, NSFW and descriptions, not shots or text on screen expect(reprocessOptions({ video: false, hasAudio: false, paged: true }).map((o) => o.key)).toEqual([ "transcribe", "diarize", "faces", "objects", + "nsfw", "describe", "analyze", "embed", diff --git a/nextjs-frontend/app/(app)/library/page.tsx b/nextjs-frontend/app/(app)/library/page.tsx index 56abc7d71..b0e6dc942 100644 --- a/nextjs-frontend/app/(app)/library/page.tsx +++ b/nextjs-frontend/app/(app)/library/page.tsx @@ -4,20 +4,30 @@ import { LibraryScreen } from "@/components/library/library-screen"; export const metadata: Metadata = { title: "Library" }; -type Props = { searchParams: Promise<{ namespace?: string | string[]; collection?: string | string[] }> }; +type Props = { + searchParams: Promise<{ namespace?: string | string[]; collection?: string | string[]; nsfw?: string | string[] }>; +}; + +const SENSITIVE = ["include", "only", "exclude"] as const; const one = (v?: string | string[]) => (Array.isArray(v) ? v[0] : v); -/** Library (L1–L6): every recording you can see, with live progress; ?namespace=…&collection=… opens a collection. */ +/** Library (L1–L6): every recording you can see, with live progress; ?namespace=…&collection=… opens a collection, and + * &nsfw=only (include, exclude) filters by NSFW classification. */ export default async function LibraryPage({ searchParams }: Props) { const p = await searchParams; const namespace = one(p.namespace); const collection = Number(one(p.collection)); + const nsfw = SENSITIVE.find((s) => s === one(p.nsfw)); return ( 0 ? collection : null } + ? { + namespace, + collection: Number.isInteger(collection) && collection > 0 ? collection : null, + sensitive: nsfw, + } : undefined } /> diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index 0e987d26b..690e988dd 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -269,6 +269,11 @@ export type { CheckUpdatesData, CheckUpdatesResponse, CheckUpdatesResponses, + ClassifyNamespaceNsfwData, + ClassifyNamespaceNsfwError, + ClassifyNamespaceNsfwErrors, + ClassifyNamespaceNsfwResponse, + ClassifyNamespaceNsfwResponses, ClearEntitySetupData, ClearEntitySetupError, ClearEntitySetupErrors, @@ -1084,6 +1089,11 @@ export type { GetNamespaceMetadataErrors, GetNamespaceMetadataResponse, GetNamespaceMetadataResponses, + GetNamespaceNsfwData, + GetNamespaceNsfwError, + GetNamespaceNsfwErrors, + GetNamespaceNsfwResponse, + GetNamespaceNsfwResponses, GetNamespaceRdfData, GetNamespaceRdfError, GetNamespaceRdfErrors, @@ -1861,6 +1871,11 @@ export type { MarketplaceCreate, MarketplacePlugin, MarketplaceSuggestion, + MarkRecordingNsfwData, + MarkRecordingNsfwError, + MarkRecordingNsfwErrors, + MarkRecordingNsfwResponse, + MarkRecordingNsfwResponses, MatterbridgeRoom, MatterbridgeStatus, Me, @@ -1930,6 +1945,8 @@ export type { NamespaceFaces, NamespaceMetadata, NamespaceMetadataUpdate, + NamespaceNsfw, + NamespaceNsfwUpdate, NamespaceStats, NamespaceUpdate, NewImportHookTokenData, @@ -1982,6 +1999,15 @@ export type { NotSameSpeakerErrors, NotSameSpeakerResponse, NotSameSpeakerResponses, + NsfwChoices, + NsfwCounts, + NsfwCountsData, + NsfwCountsError, + NsfwCountsErrors, + NsfwCountsResponse, + NsfwCountsResponses, + NsfwHit, + NsfwState, OAuthGrant, OAuthTokens, ObjectCount, @@ -2208,6 +2234,7 @@ export type { RecordingMetadataUpdate, RecordingMove, RecordingMoved, + RecordingNsfwUpdate, RecordingSpeaker, RecordingsPlace, RecordingsRetag, @@ -3043,6 +3070,11 @@ export type { UpdateNamespaceMetadataErrors, UpdateNamespaceMetadataResponse, UpdateNamespaceMetadataResponses, + UpdateNamespaceNsfwData, + UpdateNamespaceNsfwError, + UpdateNamespaceNsfwErrors, + UpdateNamespaceNsfwResponse, + UpdateNamespaceNsfwResponses, UpdateNamespaceResponse, UpdateNamespaceResponses, UpdateNoteData, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index 0b3515647..7a8d08851 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -104,6 +104,9 @@ import type { CheckMessageResponses, CheckUpdatesData, CheckUpdatesResponses, + ClassifyNamespaceNsfwData, + ClassifyNamespaceNsfwErrors, + ClassifyNamespaceNsfwResponses, ClearEntitySetupData, ClearEntitySetupErrors, ClearEntitySetupResponses, @@ -528,6 +531,9 @@ import type { GetNamespaceMetadataData, GetNamespaceMetadataErrors, GetNamespaceMetadataResponses, + GetNamespaceNsfwData, + GetNamespaceNsfwErrors, + GetNamespaceNsfwResponses, GetNamespaceRdfData, GetNamespaceRdfErrors, GetNamespaceRdfResponses, @@ -952,6 +958,9 @@ import type { MarkEntitiesNotSameData, MarkEntitiesNotSameErrors, MarkEntitiesNotSameResponses, + MarkRecordingNsfwData, + MarkRecordingNsfwErrors, + MarkRecordingNsfwResponses, MeData, MeResponses, MergeEntitiesData, @@ -989,6 +998,9 @@ import type { NotSameSpeakerData, NotSameSpeakerErrors, NotSameSpeakerResponses, + NsfwCountsData, + NsfwCountsErrors, + NsfwCountsResponses, OpenRecordingToIpGroupData, OpenRecordingToIpGroupErrors, OpenRecordingToIpGroupResponses, @@ -1483,6 +1495,9 @@ import type { UpdateNamespaceMetadataData, UpdateNamespaceMetadataErrors, UpdateNamespaceMetadataResponses, + UpdateNamespaceNsfwData, + UpdateNamespaceNsfwErrors, + UpdateNamespaceNsfwResponses, UpdateNamespaceResponses, UpdateNoteData, UpdateNoteErrors, @@ -3712,6 +3727,56 @@ export class Namespaces { }); } + /** + * Get Namespace Nsfw + * + * NSFW classification here (docs/nsfw.md): whether it's on, the threshold and how flagged resources show, what the + * namespace chose for itself and the archive's settings, and how many resources are classified and flagged. + */ + public static getNamespaceNsfw( + options: Options, + ): RequestResult { + return (options.client ?? client).get({ + url: "/api/v1/namespaces/{name}/nsfw", + ...options, + }); + } + + /** + * Update Namespace Nsfw + * + * Owners: turn NSFW classification on or off for this namespace, or give it its own threshold or display; null + * follows the archive's setting again. A new threshold re-flags what's classified at once. `classify` also queues the + * nsfw step for its videos, documents and images not classified yet. Audited as `namespace.nsfw`. + */ + public static updateNamespaceNsfw( + options: Options, + ): RequestResult { + return (options.client ?? client).put({ + url: "/api/v1/namespaces/{name}/nsfw", + ...options, + headers: { + "Content-Type": "application/json", + ...options.headers, + }, + }); + } + + /** + * Classify Namespace Nsfw + * + * Editors: queue the nsfw step for the namespace's videos, documents and images not classified yet (those added + * while it's on are classified by their pipeline). 409 where classification is off. + */ + public static classifyNamespaceNsfw( + options: Options, + ): RequestResult { + return (options.client ?? client).post({ + url: "/api/v1/namespaces/{name}/nsfw/classify", + ...options, + }); + } + /** * Get Namespace Wordcloud * @@ -4145,6 +4210,21 @@ export class Resources { }); } + /** + * Nsfw Counts + * + * NSFW classification in the recordings you can read (or one namespace's): whether it's on in any of their + * namespaces, how many are flagged, and how many of those the Library leaves out (docs/nsfw.md). + */ + public static nsfwCounts( + options?: Options, + ): RequestResult { + return (options?.client ?? client).get({ + url: "/api/v1/resources/nsfw", + ...options, + }); + } + /** * Place Recordings * @@ -4213,6 +4293,25 @@ export class Resources { }); } + /** + * Mark Recording Nsfw + * + * Mark a resource NSFW or not by hand (editors), where NSFW classification is on in its namespace; your call wins + * over the model's score until you send null. 409 where classification is off. Audited as `recording.nsfw`. + */ + public static markRecordingNsfw( + options: Options, + ): RequestResult { + return (options.client ?? client).put({ + url: "/api/v1/resources/{rid}/nsfw", + ...options, + headers: { + "Content-Type": "application/json", + ...options.headers, + }, + }); + } + /** * Move Recording * diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 843b729a3..1a5b31e5b 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -8889,6 +8889,92 @@ export type NamespaceMetadataUpdate = { } | null; }; +/** + * NamespaceNsfw + * + * NSFW classification in a namespace (docs/nsfw.md): what applies, what the namespace chose for itself, and the + * archive's settings it otherwise follows. + */ +export type NamespaceNsfw = { + /** + * Enabled + * + * its videos, documents and images are classified by the nsfw step + */ + enabled: boolean; + /** + * Threshold + * + * a resource is flagged when one of its pictures scores at least this (0.5 to 0.99) + */ + threshold: number; + /** + * Display + * + * how flagged resources show in the web app: blurred, left out of the Library, or as they are + */ + display: "blur" | "hide" | "show"; + /** + * what the namespace chose for itself (owners change it) + */ + own: NsfwChoices; + /** + * the archive's settings (Settings → NSFW, admins), for what the namespace didn't choose + */ + archive: NsfwChoices; + /** + * Classified + * + * its resources classified so far + */ + classified?: number; + /** + * Flagged + * + * how many of them are flagged NSFW + */ + flagged?: number; + /** + * Unclassified + * + * its videos, documents and images not classified yet + */ + unclassified?: number; + /** + * Queued + * + * with classify: how many runs were queued + */ + queued?: number | null; + [key: string]: unknown; +}; + +/** + * NamespaceNsfwUpdate + * + * Only the fields you send change; send null to follow the archive's setting again. + */ +export type NamespaceNsfwUpdate = { + /** + * Enabled + */ + enabled?: boolean | null; + /** + * Threshold + */ + threshold?: number | null; + /** + * Display + */ + display?: "blur" | "hide" | "show" | null; + /** + * Classify + * + * also queue the nsfw step for its videos, documents and images not classified yet + */ + classify?: boolean; +}; + /** * NamespaceStats */ @@ -10254,6 +10340,154 @@ export type NotifyTestResult = { [key: string]: unknown; }; +/** + * NsfwChoices + */ +export type NsfwChoices = { + /** + * Enabled + * + * on or off; null: as the archive's setting says + */ + enabled?: boolean | null; + /** + * Threshold + * + * flag a picture at least this likely to be NSFW; null: the archive's + */ + threshold?: number | null; + /** + * Display + * + * blur, hide or show what's flagged; null: the archive's + */ + display?: "blur" | "hide" | "show" | null; + [key: string]: unknown; +}; + +/** + * NsfwCounts + * + * NSFW classification in the recordings you can read (docs/nsfw.md). + */ +export type NsfwCounts = { + /** + * On + * + * it's on in at least one of their namespaces + */ + on: boolean; + /** + * Flagged + * + * how many are flagged NSFW + */ + flagged?: number; + /** + * Hidden + * + * how many of those the Library leaves out unless asked (their namespace hides them) + */ + hidden?: number; + [key: string]: unknown; +}; + +/** + * NsfwHit + * + * One of a resource's pictures that scored highest. + */ +export type NsfwHit = { + /** + * T + * + * where it is: ms into a video, or a page number from 0 + */ + t: number; + /** + * Score + * + * how likely it is to be NSFW, from 0 to 1 + */ + score: number; + /** + * Frame + * + * a signed link to the frame or page + */ + frame?: string | null; + [key: string]: unknown; +}; + +/** + * NsfwState + * + * What NSFW classification found (docs/nsfw.md), where it's on in the resource's namespace. + */ +export type NsfwState = { + /** + * Flagged + * + * flagged NSFW: a person's call if they made one, else its highest score is at least the threshold + */ + flagged: boolean; + /** + * Score + * + * the highest score of its pictures, from 0 (safe for work) to 1; null when only marked + */ + score?: number | null; + /** + * Manual + * + * a person marked it NSFW (true) or not (false); null: the score decides + */ + manual?: boolean | null; + /** + * Blur + * + * flagged, and its namespace blurs or hides what's flagged: the web app blurs its pictures + */ + blur?: boolean; + /** + * Display + * + * how its namespace shows what's flagged: blur, hide (left out of the Library) or show + */ + display?: "blur" | "hide" | "show"; + /** + * Threshold + * + * its namespace's threshold + */ + threshold?: number; + /** + * Checked + * + * how many of its frames or pages were scored + */ + checked?: number | null; + /** + * Of + * + * how many it has + */ + of?: number | null; + /** + * At + * + * when it was classified + */ + at?: string | null; + /** + * Hits + * + * its highest-scoring pictures (on the resource and its player) + */ + hits?: Array; + [key: string]: unknown; +}; + /** * OAuthGrant * @@ -10921,6 +11155,12 @@ export type PhotoHit = { * a signed link to it, small */ thumb?: string | null; + /** + * Nsfw + * + * flagged NSFW where its namespace blurs or hides that: the web app blurs it + */ + nsfw?: boolean; [key: string]: unknown; }; @@ -11361,6 +11601,10 @@ export type Player = { * the namespace pixelates faces in the pictures visitors see */ faces_pixelate?: boolean; + /** + * NSFW classification, where it's on in its namespace and it was classified + */ + nsfw?: NsfwState | null; [key: string]: unknown; }; @@ -12755,6 +12999,10 @@ export type Recording = { * the places and dates anytopdf found (analysis.anytopdf) */ enrichment?: Enrichment | null; + /** + * NSFW classification, where it's on in its namespace and it was classified + */ + nsfw?: NsfwState | null; [key: string]: unknown; }; @@ -13011,6 +13259,18 @@ export type RecordingMoved = { [key: string]: unknown; }; +/** + * RecordingNsfwUpdate + */ +export type RecordingNsfwUpdate = { + /** + * Flagged + * + * true: NSFW; false: not NSFW; null: let the score decide again + */ + flagged: boolean | null; +}; + /** * RecordingSpeaker */ @@ -13170,6 +13430,10 @@ export type RecordingSummary = { * your role on it: through its namespace or its collection (an admin of the collection is an owner) */ role?: "viewer" | "editor" | "owner" | null; + /** + * NSFW classification, where it's on in its namespace and it was classified + */ + nsfw?: NsfwState | null; [key: string]: unknown; }; @@ -14574,6 +14838,12 @@ export type SearchHit = { * found by meaning: how alike the passage is to the query (cosine, up to 1) */ similarity?: number | null; + /** + * Nsfw + * + * its recording is flagged NSFW where its namespace blurs or hides that: the web app blurs its frame + */ + nsfw?: boolean; [key: string]: unknown; }; @@ -22820,6 +23090,96 @@ export type UpdateNamespaceResponses = { export type UpdateNamespaceResponse = UpdateNamespaceResponses[keyof UpdateNamespaceResponses]; +export type GetNamespaceNsfwData = { + body?: never; + path: { + /** + * Name + */ + name: string; + }; + query?: never; + url: "/api/v1/namespaces/{name}/nsfw"; +}; + +export type GetNamespaceNsfwErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type GetNamespaceNsfwError = GetNamespaceNsfwErrors[keyof GetNamespaceNsfwErrors]; + +export type GetNamespaceNsfwResponses = { + /** + * Successful Response + */ + 200: NamespaceNsfw; +}; + +export type GetNamespaceNsfwResponse = GetNamespaceNsfwResponses[keyof GetNamespaceNsfwResponses]; + +export type UpdateNamespaceNsfwData = { + body: NamespaceNsfwUpdate; + path: { + /** + * Name + */ + name: string; + }; + query?: never; + url: "/api/v1/namespaces/{name}/nsfw"; +}; + +export type UpdateNamespaceNsfwErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type UpdateNamespaceNsfwError = UpdateNamespaceNsfwErrors[keyof UpdateNamespaceNsfwErrors]; + +export type UpdateNamespaceNsfwResponses = { + /** + * Successful Response + */ + 200: NamespaceNsfw; +}; + +export type UpdateNamespaceNsfwResponse = UpdateNamespaceNsfwResponses[keyof UpdateNamespaceNsfwResponses]; + +export type ClassifyNamespaceNsfwData = { + body?: never; + path: { + /** + * Name + */ + name: string; + }; + query?: never; + url: "/api/v1/namespaces/{name}/nsfw/classify"; +}; + +export type ClassifyNamespaceNsfwErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type ClassifyNamespaceNsfwError = ClassifyNamespaceNsfwErrors[keyof ClassifyNamespaceNsfwErrors]; + +export type ClassifyNamespaceNsfwResponses = { + /** + * Successful Response + */ + 200: JobsQueued; +}; + +export type ClassifyNamespaceNsfwResponse = ClassifyNamespaceNsfwResponses[keyof ClassifyNamespaceNsfwResponses]; + export type GetNamespaceWordcloudData = { body?: never; path: { @@ -23608,6 +23968,12 @@ export type ListRecordingsData = { * me: recordings you edited (corrected the transcript, changed the catalogue record or renamed) */ edited_by?: "me" | null; + /** + * Sensitive + * + * where NSFW classification is on (docs/nsfw.md): only those flagged NSFW, exclude them, or include them. Without it, flagged ones are left out where their namespace hides them (display hide) + */ + sensitive?: "include" | "only" | "exclude" | null; /** * Collection * @@ -23837,6 +24203,38 @@ export type ListObjectsResponses = { export type ListObjectsResponse = ListObjectsResponses[keyof ListObjectsResponses]; +export type NsfwCountsData = { + body?: never; + path?: never; + query?: { + /** + * Ns + * + * one namespace + */ + ns?: string | null; + }; + url: "/api/v1/resources/nsfw"; +}; + +export type NsfwCountsErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type NsfwCountsError = NsfwCountsErrors[keyof NsfwCountsErrors]; + +export type NsfwCountsResponses = { + /** + * Successful Response + */ + 200: NsfwCounts; +}; + +export type NsfwCountsResponse = NsfwCountsResponses[keyof NsfwCountsResponses]; + export type PlaceRecordingsData = { body: RecordingsPlace; path?: never; @@ -23952,6 +24350,38 @@ export type UpdateRecordingResponses = { export type UpdateRecordingResponse = UpdateRecordingResponses[keyof UpdateRecordingResponses]; +export type MarkRecordingNsfwData = { + body: RecordingNsfwUpdate; + path: { + /** + * Rid + */ + rid: number; + }; + query?: never; + url: "/api/v1/resources/{rid}/nsfw"; +}; + +export type MarkRecordingNsfwErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type MarkRecordingNsfwError = MarkRecordingNsfwErrors[keyof MarkRecordingNsfwErrors]; + +export type MarkRecordingNsfwResponses = { + /** + * Response Recordings-Mark Recording Nsfw + * + * Successful Response + */ + 200: NsfwState | null; +}; + +export type MarkRecordingNsfwResponse = MarkRecordingNsfwResponses[keyof MarkRecordingNsfwResponses]; + export type MoveRecordingData = { body: RecordingMove; path: { diff --git a/nextjs-frontend/components/admin/namespaces.tsx b/nextjs-frontend/components/admin/namespaces.tsx index 65cd8ec90..6ec48154e 100644 --- a/nextjs-frontend/components/admin/namespaces.tsx +++ b/nextjs-frontend/components/admin/namespaces.tsx @@ -15,6 +15,7 @@ import { ActivityPanel, BudgetPanel, CostCell, useBudgets, useCosts } from "@/co import { isUnreachable } from "@/components/errors/error-states"; import { NotificationsSection } from "@/components/notifications/notifications-section"; import { runtime } from "@/components/iiif/iiif-model"; +import { NsfwSection } from "@/components/nsfw/nsfw-section"; import { ChoiceCards } from "@/components/settings/controls"; import { VaultSection } from "@/components/vaults/vault-section"; import { RoleChip } from "@/components/ui/badge"; @@ -475,6 +476,7 @@ export function NamespaceDetail({ ns }: { ns: string }) { )} + {known && } diff --git a/nextjs-frontend/components/batches/format.ts b/nextjs-frontend/components/batches/format.ts index f839f0f9b..78001452d 100644 --- a/nextjs-frontend/components/batches/format.ts +++ b/nextjs-frontend/components/batches/format.ts @@ -35,7 +35,7 @@ export function confirmNumber(expected: string | null | undefined): string { } /** What a run would need that a transcript-only recording can't give: audio and video steps. */ -export const MEDIA_STEPS = new Set(["transcribe", "diarize", "shots", "ocr", "faces", "objects", "describe"]); +export const MEDIA_STEPS = new Set(["transcribe", "diarize", "shots", "ocr", "faces", "objects", "nsfw", "describe"]); /** True when every recording is transcript-only and every step needs media: nothing can run. */ export function nothingRunnable(est: Pick, steps: { type?: unknown }[]): boolean { diff --git a/nextjs-frontend/components/batches/run-on.tsx b/nextjs-frontend/components/batches/run-on.tsx index d04b65386..f77569144 100644 --- a/nextjs-frontend/components/batches/run-on.tsx +++ b/nextjs-frontend/components/batches/run-on.tsx @@ -39,6 +39,7 @@ const STEP_ORDER = [ "ocr", "faces", "objects", + "nsfw", "describe", "analyze", "embed", diff --git a/nextjs-frontend/components/library/cells.tsx b/nextjs-frontend/components/library/cells.tsx index 05390d8cc..0298023ca 100644 --- a/nextjs-frontend/components/library/cells.tsx +++ b/nextjs-frontend/components/library/cells.tsx @@ -1,6 +1,9 @@ "use client"; -import { BookOpenText, FileAudio, FileImage, FileText, FileVideo } from "lucide-react"; +import { BookOpenText, EyeOff, FileAudio, FileImage, FileText, FileVideo } from "lucide-react"; + +import type { NsfwState } from "@/app/openapi-client/types.gen"; +import { nsfwText } from "@/components/nsfw/nsfw-model"; import { Badge, EmotionBar, speakerColor } from "@/components/ui/badge"; import { Tooltip } from "@/components/ui/tooltip"; @@ -40,6 +43,22 @@ export function TextBadge() { ); } +/** "NSFW" marks a resource flagged NSFW where its namespace classifies them (docs/nsfw.md). */ +export function NsfwBadge({ state, paged }: { state?: NsfwState | null; paged?: boolean }) { + if (!state?.flagged) return null; + return ( + + + + NSFW + + + ); +} + /** Status chip with the job overlay under it: the running step and a thin progress bar, a wait, or what failed. */ export function StatusCell({ view, diff --git a/nextjs-frontend/components/library/filters-bar.tsx b/nextjs-frontend/components/library/filters-bar.tsx index 07423ea4f..fa7a756b4 100644 --- a/nextjs-frontend/components/library/filters-bar.tsx +++ b/nextjs-frontend/components/library/filters-bar.tsx @@ -3,7 +3,7 @@ import { Check, ChevronDown, FolderTree, Search, X } from "lucide-react"; import { forwardRef, useState, type ReactNode } from "react"; -import type { CollectionNode, FieldDef } from "@/app/openapi-client/types.gen"; +import type { CollectionNode, FieldDef, NsfwCounts } from "@/app/openapi-client/types.gen"; import { fieldFilterLabel } from "@/components/fields/fields-model"; import { collectionName } from "@/components/library/collections-model"; import { @@ -20,6 +20,7 @@ import { type SpeakerChoice, type StatusFilter, } from "@/components/library/model"; +import { SENSITIVE_CHIP, SENSITIVE_LABEL, type SensitiveFilter } from "@/components/nsfw/nsfw-model"; import { objectName } from "@/components/recording/objects-model"; import { Button } from "@/components/ui/button"; import { Input, Select } from "@/components/ui/field"; @@ -258,7 +259,7 @@ function FieldFilterForm({ /** * Library filters: namespace, collection, field, source, status, speaker, date, duration, language, media, tags and - * the kinds of object seen (once any have been). + * the kinds of object seen (once any have been), and NSFW (where it's on). */ export function FiltersBar({ filters, @@ -278,6 +279,7 @@ export function FiltersBar({ fields, fieldsLoading, objects = [], + nsfw, }: { filters: Filters; onChange: (f: Filters) => void; @@ -305,6 +307,8 @@ export function FiltersBar({ fieldsLoading?: boolean; /** The kinds of object seen in the recordings in scope (GET /recordings/objects), most first. */ objects?: { object: string; recordings: number }[]; + /** NSFW classification in scope (GET /recordings/nsfw): the chip shows where it's on. */ + nsfw?: NsfwCounts; }) { const { namespaces: full, partialNamespaces, namespace, setNamespace, isPartial } = useArchive(); const namespaces = [...full, ...partialNamespaces].sort((a, b) => a.name.localeCompare(b.name)); @@ -744,6 +748,27 @@ export function FiltersBar({ )} )} + {(nsfw?.on || filters.sensitive !== "any") && ( + set({ sensitive: "any" })} + width={230} + > + {(close) => ( + + value={filters.sensitive} + options={SENSITIVE_LABEL} + onChange={(sensitive) => set({ sensitive })} + close={close} + /> + )} + + )} ); diff --git a/nextjs-frontend/components/library/library-screen.tsx b/nextjs-frontend/components/library/library-screen.tsx index bfedf169d..d56508d65 100644 --- a/nextjs-frontend/components/library/library-screen.tsx +++ b/nextjs-frontend/components/library/library-screen.tsx @@ -41,6 +41,7 @@ import { useSpeakerChoices, useLanguages, useOrigins, + useNsfwCounts, useObjectCounts, useTagCounts, useWatchedSources, @@ -48,6 +49,7 @@ import { import { useCollectionTree } from "@/components/library/use-collections"; import { useIsNarrow } from "@/components/library/use-media"; import { fromView, viewState } from "@/components/library/views-model"; +import type { SensitiveFilter } from "@/components/nsfw/nsfw-model"; import { CreatePodcastDialog } from "@/components/podcasts/create-dialog"; import { Banner } from "@/components/ui/banner"; import { Button } from "@/components/ui/button"; @@ -93,7 +95,9 @@ function typing(t: EventTarget | null): boolean { /** The Library (L1–L6): every recording you can see, across namespaces, with live job progress in its row. `initial` * opens it on a namespace's collection (/library?namespace=…&collection=…, as a recording's breadcrumb links). */ -export function LibraryScreen({ initial }: { initial?: { namespace: string; collection: number | null } } = {}) { +export function LibraryScreen({ + initial, +}: { initial?: { namespace: string; collection: number | null; sensitive?: SensitiveFilter } } = {}) { const { namespace, namespaces, isPartial, roleIn, can, me, setNamespace } = useArchive(); const [layout, setLayout] = useState<"table" | "list">("table"); const [filters, setFilters] = useState(NO_FILTERS); @@ -127,6 +131,7 @@ export function LibraryScreen({ initial }: { initial?: { namespace: string; coll const tree = useCollectionTree(namespace); const tagCounts = useTagCounts(namespace); const objectCounts = useObjectCounts(namespace); + const nsfwCounts = useNsfwCounts(namespace); const origins = useOrigins(namespace); const languages = useLanguages(namespace); const [dragging, setDragging] = useState(false); @@ -245,7 +250,9 @@ export function LibraryScreen({ initial }: { initial?: { namespace: string; coll if (!initial?.namespace) return; setNamespace(initial.namespace); if (initial.collection != null) setPendingCollection({ id: initial.collection, ns: initial.namespace }); - }, [initial?.namespace, initial?.collection, setNamespace]); + const sensitive = initial.sensitive; + if (sensitive) setFilters((f) => ({ ...f, sensitive })); + }, [initial?.namespace, initial?.collection, initial?.sensitive, setNamespace]); useEffect(() => { if (!pendingCollection || pendingCollection.ns !== namespace || !tree.isSuccess) return; const name = collectionName(tree.data, pendingCollection.id); @@ -529,6 +536,7 @@ export function LibraryScreen({ initial }: { initial?: { namespace: string; coll speakersLoading={speakers.loading} tags={tagCounts.data ?? []} objects={objectCounts.data ?? []} + nsfw={nsfwCounts.data} tagsLoading={tagCounts.isPending} inputRef={filterRef} compact={narrow} diff --git a/nextjs-frontend/components/library/model.ts b/nextjs-frontend/components/library/model.ts index d1491a62b..d67da20f9 100644 --- a/nextjs-frontend/components/library/model.ts +++ b/nextjs-frontend/components/library/model.ts @@ -6,6 +6,7 @@ * endpoint says how many match in its X-Total-Count header. */ import type { Job, ListRecordingsData, RecordingSummary, Speaker } from "@/app/openapi-client/types.gen"; +import type { SensitiveFilter } from "@/components/nsfw/nsfw-model"; import type { Tone } from "@/components/ui/badge"; import { STEP_LABEL } from "@/components/ui/loop"; import { plural, tc } from "@/lib/format"; @@ -230,6 +231,8 @@ export type Filters = { collection: number | null; /** A custom field of the namespace shown: the recordings with a value for it, or with this value. */ field: FieldFilter | null; + /** Resources flagged NSFW (docs/nsfw.md): "any" leaves out those a namespace hides. Not kept in saved views. */ + sensitive: SensitiveFilter; }; /** A custom field filter: its id, and the value to match (empty: any value). */ @@ -248,6 +251,7 @@ export const NO_FILTERS: Filters = { objects: [], collection: null, field: null, + sensitive: "any", }; /** "en" → "English", "pt-BR" → "Brazilian Portuguese"; null (not known) → "Not known"; an odd code stays as it is. */ @@ -326,7 +330,8 @@ export function activeFilterCount(f: Filters): number { (f.languages.length ? 1 : 0) + (f.objects.length ? 1 : 0) + (f.collection != null ? 1 : 0) + - (f.field ? 1 : 0) + (f.field ? 1 : 0) + + (f.sensitive !== "any" ? 1 : 0) ); } @@ -391,6 +396,7 @@ export function libraryQuery( q.field = f.field.id; if (f.field.value.trim()) q.value = f.field.value.trim(); } + if (f.sensitive !== "any") q.sensitive = f.sensitive; if (view === "attention") q.attention = true; if (view === "processing") q.processing = true; if (view === "mine") q.edited_by = "me"; diff --git a/nextjs-frontend/components/library/recording-list.tsx b/nextjs-frontend/components/library/recording-list.tsx index 3f5853162..da53c437c 100644 --- a/nextjs-frontend/components/library/recording-list.tsx +++ b/nextjs-frontend/components/library/recording-list.tsx @@ -6,7 +6,7 @@ import { useRef } from "react"; import type { RecordingSummary } from "@/app/openapi-client/types.gen"; import { AccessBadge } from "@/components/access/access-fields"; -import { MediaIcon } from "@/components/library/cells"; +import { MediaIcon, NsfwBadge } from "@/components/library/cells"; import { lengthText, speakerList, statusView } from "@/components/library/model"; import { rowClick, type RowProps } from "@/components/library/recording-table"; import { Badge, speakerColor } from "@/components/ui/badge"; @@ -141,6 +141,7 @@ export function RecordingCards({ rows, jobs, reviews, selected, onOpen }: RowPro > {r.title || "Untitled"} + {view.label} diff --git a/nextjs-frontend/components/library/recording-table.tsx b/nextjs-frontend/components/library/recording-table.tsx index 7b019b960..a1ba4f93e 100644 --- a/nextjs-frontend/components/library/recording-table.tsx +++ b/nextjs-frontend/components/library/recording-table.tsx @@ -12,6 +12,7 @@ import { EmotionCell, ImportanceCell, MediaIcon, + NsfwBadge, SpeakersCell, StatusCell, TagsCell, @@ -284,6 +285,7 @@ export function RecordingTable({ {r.title || "Untitled"} {r.media_kind === "transcript" && } + diff --git a/nextjs-frontend/components/library/use-library.ts b/nextjs-frontend/components/library/use-library.ts index 1780aea11..6cf583295 100644 --- a/nextjs-frontend/components/library/use-library.ts +++ b/nextjs-frontend/components/library/use-library.ts @@ -226,6 +226,17 @@ export function useObjectCounts(ns: string | null) { }); } +/** NSFW classification in the recordings shown: whether it's on in any of their namespaces, how many are flagged and + * how many of those are hidden (docs/nsfw.md). */ +export function useNsfwCounts(ns: string | null) { + const client = useApiClient(); + return useQuery({ + queryKey: ["recording-nsfw", ns], + queryFn: () => data(Resources.nsfwCounts({ client, query: ns ? { ns } : {} })), + staleTime: 60_000, + }); +} + export function useTagCounts(ns: string | null) { const client = useApiClient(); return useQuery({ diff --git a/nextjs-frontend/components/nsfw/nsfw-model.ts b/nextjs-frontend/components/nsfw/nsfw-model.ts new file mode 100644 index 000000000..6532b081b --- /dev/null +++ b/nextjs-frontend/components/nsfw/nsfw-model.ts @@ -0,0 +1,54 @@ +/** + * NSFW classification as the web app shows it (docs/nsfw.md): what a resource's state says, the Library's filter, and + * a namespace's own choices. + */ +import type { NsfwChoices, NsfwState } from "@/app/openapi-client/types.gen"; + +/** The Library's NSFW filter: "any" leaves out what a namespace hides, the others are the API's `sensitive`. */ +export type SensitiveFilter = "any" | "include" | "only" | "exclude"; + +export const SENSITIVE_LABEL: Record = { + any: "As each namespace shows them", + include: "Include hidden ones", + only: "Only flagged NSFW", + exclude: "Leave flagged ones out", +}; + +export const SENSITIVE_CHIP: Record = { + any: "NSFW", + include: "NSFW: include hidden", + only: "NSFW only", + exclude: "No NSFW", +}; + +/** "0.93" */ +export const score = (v: number | null | undefined) => (v == null ? "—" : v.toFixed(2)); + +/** One line on what classification found: "Flagged NSFW: highest score 0.93 (threshold 0.80), 12 of 300 frames". */ +export function nsfwText(n: NsfwState, paged = false): string { + const what = paged ? "page" : "frame"; + const over = n.checked != null && n.of != null && n.of > 1 ? `, ${n.checked} of ${n.of} ${what}s scored` : ""; + const by = n.score != null ? `highest score ${score(n.score)} (threshold ${score(n.threshold)})${over}` : null; + if (n.manual === true) return `Marked NSFW by hand${by ? `; the model's ${by}` : ""}`; + if (n.manual === false) return `Marked not NSFW by hand${by ? `; the model's ${by}` : ""}`; + return `${n.flagged ? "Flagged NSFW" : "Not NSFW"}${by ? `: ${by}` : ""}`; +} + +/** A namespace's own on/off: "archive" follows Settings → NSFW. */ +export type OwnSwitch = "archive" | "on" | "off"; + +export function ownSwitch(own: NsfwChoices | null | undefined): OwnSwitch { + return own?.enabled == null ? "archive" : own.enabled ? "on" : "off"; +} + +export function switchValue(v: OwnSwitch): boolean | null { + return v === "archive" ? null : v === "on"; +} + +/** A threshold typed in a box: null when empty (follow the archive's), undefined when it isn't 0.5 to 0.99. */ +export function parseThreshold(text: string): number | null | undefined { + const t = text.trim(); + if (!t) return null; + const v = Number(t); + return Number.isFinite(v) && v >= 0.5 && v <= 0.99 ? Math.round(v * 100) / 100 : undefined; +} diff --git a/nextjs-frontend/components/nsfw/nsfw-section.tsx b/nextjs-frontend/components/nsfw/nsfw-section.tsx new file mode 100644 index 000000000..f7ca82d2b --- /dev/null +++ b/nextjs-frontend/components/nsfw/nsfw-section.tsx @@ -0,0 +1,226 @@ +"use client"; + +import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; +import { EyeOff } from "lucide-react"; +import Link from "next/link"; +import { useEffect, useState, type FormEvent } from "react"; + +import { Namespaces } from "@/app/openapi-client"; +import type { NamespaceNsfw, NamespaceNsfwUpdate, NsfwChoices } from "@/app/openapi-client/types.gen"; +import { isUnreachable } from "@/components/errors/error-states"; +import { libraryHref } from "@/components/library/collections-model"; +import { ownSwitch, parseThreshold, score, switchValue, type OwnSwitch } from "@/components/nsfw/nsfw-model"; +import { ChoiceCards } from "@/components/settings/controls"; +import { Button } from "@/components/ui/button"; +import { Field, Input, Select } from "@/components/ui/field"; +import { EmptyState, SkeletonRows } from "@/components/ui/states"; +import { useToast } from "@/components/ui/toast"; +import { data, useApiClient } from "@/lib/api/browser"; +import { count, plural } from "@/lib/format"; +import { needRole, useArchive } from "@/lib/hooks/session"; + +export const nsfwKey = (ns: string) => ["namespace", ns, "nsfw"] as const; + +const DISPLAY: Record = { + blur: "Blur their pictures", + hide: "Hide them from the Library, and blur", + show: "Show them, with a label", +}; + +/** What undoing a change sends: the namespace's own choices as they were. */ +const restore = (own: NsfwChoices | undefined): NamespaceNsfwUpdate => ({ + enabled: own?.enabled ?? null, + threshold: own?.threshold ?? null, + display: own?.display ?? null, +}); + +/** + * A namespace's NSFW classification (docs/nsfw.md): owners turn it on or off here or follow the archive's setting, and + * set the threshold and how flagged resources show; editors classify what was there before. + */ +export function NsfwSection({ ns, isOwner }: { ns: string; isOwner: boolean }) { + const client = useApiClient(); + const qc = useQueryClient(); + const toast = useToast(); + const { can } = useArchive(); + const editor = can("editor", ns); + const q = useQuery({ + queryKey: nsfwKey(ns), + queryFn: () => data(Namespaces.getNamespaceNsfw({ client, path: { name: ns } })), + }); + const changed = (got: NamespaceNsfw) => { + qc.setQueryData(nsfwKey(ns), got); + void qc.invalidateQueries({ queryKey: ["recordings"] }); + void qc.invalidateQueries({ queryKey: ["recording-nsfw"] }); + }; + const save = useMutation({ + mutationFn: (body: NamespaceNsfwUpdate) => + data(Namespaces.updateNamespaceNsfw({ client, path: { name: ns }, body })), + onMutate: () => ({ before: q.data?.own }), + onSuccess: (got, body, ctx) => { + changed(got); + if (body.classify) return; + toast({ + title: + body.enabled !== undefined + ? got.enabled + ? `NSFW classification is on in ${ns}` + : `NSFW classification is off in ${ns}` + : "NSFW settings saved", + body: + body.enabled !== undefined && got.enabled && got.unclassified + ? `New videos, documents and images are classified; ${plural(got.unclassified, "resource")} from before aren’t yet.` + : undefined, + tone: "green", + action: { label: "Undo", onClick: () => save.mutate(restore(ctx?.before)) }, + }); + }, + onError: (e) => toast({ title: "Couldn’t save", body: (e as Error).message, tone: "red" }), + }); + const classify = useMutation({ + mutationFn: () => data(Namespaces.classifyNamespaceNsfw({ client, path: { name: ns } })), + onSuccess: (got) => { + void qc.invalidateQueries({ queryKey: nsfwKey(ns) }); + toast({ + title: got.jobs.length ? `Classifying in ${plural(got.jobs.length, "run")}` : "Nothing left to classify", + body: got.jobs.length ? "Each resource is flagged as its run gets to the nsfw step." : undefined, + tone: got.jobs.length ? "green" : undefined, + }); + }, + onError: (e) => toast({ title: "Couldn’t classify them", body: (e as Error).message, tone: "red" }), + }); + + const s = q.data; + const [threshold, setThreshold] = useState(""); + useEffect(() => { + if (s) setThreshold(s.own.threshold != null ? String(s.own.threshold) : ""); + }, [s]); + const parsed = parseThreshold(threshold); + const saveThreshold = (e?: FormEvent) => { + e?.preventDefault(); + if (s && parsed !== undefined && parsed !== (s.own.threshold ?? null)) save.mutate({ threshold: parsed }); + }; + const ownerOnly = !isOwner ? needRole("owner", ns) : undefined; + const archiveOn = s?.archive.enabled ? "on" : "off"; + + return ( +
+
+

+ + NSFW +

+

+ Flag {ns}’s videos, documents and images that have NSFW pictures, with a model on this server: pictures aren’t + sent anywhere. Blurring and hiding are display preferences, not access control. +

+
+ {q.isPending ? ( + + ) : q.isError ? ( + } + title={isUnreachable(q.error) ? "Can’t reach the server" : "Couldn’t load the NSFW settings"} + actions={} + > + {q.error.message} + + ) : ( + s && ( + <> + save.mutate({ enabled: switchValue(v as OwnSwitch) })} + disabled={!isOwner || save.isPending} + disabledReason={ownerOnly} + options={[ + { value: "archive", label: "As the archive", hint: `${archiveOn} in Settings → NSFW` }, + { value: "on", label: "On", hint: "classify here" }, + { value: "off", label: "Off", hint: "never here" }, + ]} + /> +
+
+ + {(f) => ( + setThreshold(e.target.value)} + onBlur={() => saveThreshold()} + /> + )} + +
+ + {(f) => ( +