diff --git a/README.md b/README.md index 3ca5f90c..ea4ddd24 100644 --- a/README.md +++ b/README.md @@ -183,6 +183,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p you run), entities found by spaCy instead of the rules, and the dates people say read as days on the calendar, each off by default. Words transcription wasn't sure of are underlined, with how sure, to check and correct in place. Translate a transcript into another language when you ask, with the estimate shown first: it's kept beside the asset as a translation whose lines keep their times, found by search like an uploaded one. + A date said ("let's meet next Tuesday at 3") becomes a calendar event in one click, + prefilled from the line and added only once you approve it ([propose an event](docs/processing.md#propose-an-event)). ([Graph](docs/graph.md), [entity pages](docs/notes.md#entity-pages), [similar content](docs/processing.md#similar-content), [graph history](docs/graph-history.md), [topics](docs/topics.md), [video](docs/video.md), [NSFW classification](docs/nsfw.md), diff --git a/changelog.d/date-proposals.md b/changelog.d/date-proposals.md new file mode 100644 index 00000000..8868bff3 --- /dev/null +++ b/changelog.d/date-proposals.md @@ -0,0 +1,7 @@ +- **A date said becomes a calendar event, once you approve it.** Next to each date said in an asset (its Entities tab, + under Dates said), **Propose event** makes an event for a Google Calendar connection an admin let add events, + prefilled from the line: the day said at a time said in it ("at 3" is 15:00) or all day, the line as its title, and + who it names (for an email, its sender and recipients), never emailed unless you ask. It waits in Approvals, and + nothing is added or sent until you approve it there, signed in; declining it does nothing. Agents propose through + `POST /api/v1/dates/propose` and the MCP tool `propose_event`, and can decline but never approve + ([Propose an event](docs/processing.md#propose-an-event)). diff --git a/docs/assistant.md b/docs/assistant.md index eb09795e..48509ca7 100644 --- a/docs/assistant.md +++ b/docs/assistant.md @@ -422,7 +422,8 @@ tools for them, only for the kinds connected, with nothing to turn on: - **`add_event`** adds an event to a calendar of a Google Calendar connection an admin let add events: `title`, `start` and `end` (a date alone is all day; a time without an offset is in the calendar's own time zone; it lasts an hour, or a day, when `end` is left out), `where`, `description`, and `attendees`, who are emailed an invitation - unless `invite` is false. + unless `invite` is false. A date said in an asset can also be proposed as an event with one click, prefilled from + its line ([Propose an event](processing.md#propose-an-event)). - **`write_file`** writes a text file (Markdown, CSV, notes, code) to a storage connection: a new file, or one it replaces when `overwrite` is true. A file that turned up after it asked isn't replaced. Google Drive connected read only (its default) can't be written to. diff --git a/docs/automations.md b/docs/automations.md index 124f8381..77f78b6a 100644 --- a/docs/automations.md +++ b/docs/automations.md @@ -857,7 +857,8 @@ isn't the viewer's to see. ## Approvals Built: everything that waits for your pick is an approval in one table (`app/domain/approvals.py`): the assistant's -proposals in a conversation, as before, and an automation's actions that ask first. When an action node's intent asks +proposals in a conversation, as before, an automation's actions that ask first, and events proposed from dates said +in an asset ([Propose an event](processing.md#propose-an-event)), which only you, signed in, approve. When an action node's intent asks (its effect is `out`, or `lens` on anything but the event's own object), the engine hands it to `approvals.ask(db, cfg, intent, event=None)`, which keeps: diff --git a/docs/mcp.md b/docs/mcp.md index f7a677f7..8faec5dd 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -85,6 +85,7 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ | `list_entities` | people, organisations, products, places, events, works and terms mentioned, by name, type, namespace or asset | | `get_entity` | one entity: its names, its page's link, how often and when it is mentioned, what it is mentioned with, who mentions it most, and the lines that mention it | | `list_dates` | what was said about a day or a range: dates said in assets as days on the calendar ([Dates](processing.md#dates)), each with the words, the line and a link to that moment | +| `propose_event` | a date said, as a calendar event prefilled from its line (the day, a time said, the line as its title, who it names), waiting in Approvals for the person's yes; admins with a calendar connection that may add events ([Propose an event](processing.md#propose-an-event)) | | `explore_graph` | the knowledge graph around an entity (`e12`) or a person (`s3`), one or two steps out | | `find_path` | the shortest chain of links between two of them, with lines that show each link | | `sparql` | a read-only SPARQL query (SELECT, ASK, CONSTRUCT or DESCRIBE) over one namespace's linked data ([RDF](rdf.md)): JSON results, or Turtle | diff --git a/docs/processing.md b/docs/processing.md index 1a60710a..dbfc38e6 100644 --- a/docs/processing.md +++ b/docs/processing.md @@ -368,6 +368,34 @@ its line, linked to that moment. - The rules read English. Nothing is sent anywhere and no model is called. Turning it on reads dates as assets are analysed; Reprocess (Analyze) reads them for assets already there. +#### Propose an event + +Built (`app/domain/event_proposals.py`). A date said can become an event in a Google Calendar connection an admin let +add events ([Mail, calendars and files](assistant.md#mail-calendars-and-files)), proposed with one click: **Propose +event** next to it under Dates said in an asset's Entities tab, `POST /api/v1/dates/propose` (`recording_id` and the +line's `segment`, as `GET /api/v1/dates` gives them), or the MCP tool `propose_event`. It waits in +[Approvals](automations.md#approvals) as an `add_event`, with all of the event shown, and nothing is added or sent until +you approve it there, signed in to Lens; declining it does nothing. An API key, an app, MCP and the assistant can +propose and decline, never approve. Proposing the same date again while it waits gives the same approval. + +- **Prefilled from the line:** the day said, at a time of day said in the same line ("at 3", "3pm", "9:45 a.m.", + "15:30", "noon"; else all day), and lasting an hour. A time without am or pm from 1 to 6 is in the afternoon ("at 3" + is 15:00), from 7 to 11 in the morning, and it is in the calendar's own time zone. A weekend or a week is all of its + days; a month or a year is too long to be one event, so it needs a `start`. The title is the line itself, shortened; + who is invited are the addresses in the line and, for an email, its sender and recipients (not the calendar's own + account or yours), and they aren't emailed an invitation unless `invite` is true. The description says where it was + said, with a link to that moment. +- **Change any of it:** `title`, `start`, `end`, `where`, `description`, `attendees` (`[]` for nobody), `invite`, + `connection` and `calendar` replace what was guessed. +- **Who and when:** admins only (adding an event goes through an admin's connection), for an asset they can read, and + only when a calendar connection may add events: `GET /api/v1/dates` says so in `proposable`, and the button shows only + then. Nothing is proposed on its own: each proposal is asked for. No model is called; each proposal is in the audit + log (`calendar.propose`), and each event added (`calendar.add`). +- Refine later: a setting to propose dates on their own as assets are analysed (off by default); editing the event on + its approval card before approving (today: propose again with the fields changed, after declining); a time range or + a length said in the line ("from 3 to 5", "for half an hour"); speakers' own addresses as guests; calendars other than + Google Calendar (CalDAV); the assistant proposing from `find_dates` without a conversation (it uses `add_event`). + ### Entity setup Each namespace has an entity setup (Entities → Setup, `GET`/`PUT /api/v1/namespaces/{name}/entity-setup`): the types diff --git a/docs/requirements.md b/docs/requirements.md index 6c903c8a..527f7ee5 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -232,6 +232,7 @@ request, to another check that proves the same thing; loosening a requirement or | Entities found by spaCy, fetched the first time it's picked, with the defined entities first and the rules until it's there | [test_spacy_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_spacy_entities.py) `test_defined_entities_come_first_then_spacy_s_names`, `test_the_analyze_step_and_workflows_use_it`, `test_without_spacy_the_rules_find_them`, `test_spacy_is_fetched_only_when_it_is_picked` | | Dates said in an asset read as days ("next Tuesday", "last week"), off by default, with dates that read two ways left out | [test_dates.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_dates.py) `test_dates_are_read_as_days`, `test_what_could_be_two_dates_is_left_out`, `test_off_by_default_reads_none` | | A transcript translated into another language only when someone asks, the estimate first: in batches of lines that keep their times and speaker, the transcript marked as data, every call in the activity ledger and within budgets, kept as a translation file like an uploaded one; through the web app's Files tab, the API, MCP and the assistant (asking first) | [test_translations.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_translations.py) `test_translating_writes_a_translation_file`, `test_the_transcript_reaches_the_model_as_data`, `test_lines_go_in_batches_and_keep_their_place`, `test_a_spent_budget_refuses`, `test_the_estimate_comes_first`, `test_what_stops_a_translation`, `test_agents_translate`, `test_the_assistant_asks_first`; [recording-translate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/recording-translate.test.tsx) `is the first the browser reads that the transcript isn't in`, `translates into the browser's language with Enter, after showing the estimate` | +| A date said in an asset is proposed as a calendar event in one click, prefilled from its line (the day, a time said, the line as its title, who it names), and added only once the person approves it, signed in; declined, nothing is added; offered only to admins with a calendar connection that may add events, and agents propose over MCP but can't approve | [test_event_proposals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_event_proposals.py) `test_a_date_said_is_proposed_and_added_only_once_approved`, `test_a_declined_proposal_adds_nothing`, `test_offered_only_to_admins_with_a_calendar_that_adds_events`, `test_agents_propose_over_mcp_but_cannot_approve`, `test_the_time_of_day_is_read_from_the_line`; [said-dates.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/said-dates.test.tsx) `proposes an event from a date in one click, and says it waits in Approvals`, `offers no Propose event without a calendar that adds events, and shows nothing without dates` | ## Learn from it diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py index ab945a87..65f2cf32 100644 --- a/fastapi_backend/app/api/mcp_tools.py +++ b/fastapi_backend/app/api/mcp_tools.py @@ -57,6 +57,7 @@ descriptions, enhance, event_mutes, + event_proposals, event_reactions, events, extensions, @@ -1297,7 +1298,8 @@ def _scope(namespace: str | None) -> str: 'Tuesday after, "last week" the seven days before that Monday, "March 2024" all of March. Give start and end ' "(YYYY-MM-DD, or YYYY-MM or YYYY) for what falls between them, the earliest first, each with the words said, the " "line and a url to that moment. relative: worked out from the asset's own date. Only assets analysed while Settings " - "→ Analysis → Read dates was on have them (reading says whether it is).", + "→ Analysis → Read dates was on have them (reading says whether it is). proposable: propose_event can turn one " + "into a calendar event that waits for the person's yes.", Arg("start", "string", "from this day on: YYYY-MM-DD, YYYY-MM or YYYY", max_length=10), Arg("end", "string", "up to this day: YYYY-MM-DD, YYYY-MM or YYYY", max_length=10), Arg("namespace", "string", "only this namespace"), @@ -1331,6 +1333,7 @@ def list_dates( ) out: dict[str, Any] = { "reading": res.reading, + "proposable": res.proposable, "total": res.total, "dates": [ { @@ -1346,6 +1349,7 @@ def list_dates( "namespace": d.namespace, "recorded_at": d.recorded_at, "line": d.idx, + "segment": d.segment, "at": d.time, "speaker": d.speaker, "text": _cut(d.text), @@ -1361,6 +1365,69 @@ def list_dates( return out +@tool( + "propose_event", + "Propose a date said as an event", + "Propose a date said in an asset (from list_dates: its recording_id and line) as an event in a Google Calendar " + "connection that may add events, prefilled from the line: the day said at a time said in it (else all day), the " + "line as its title, and who it names (for an email, its sender and recipients). It waits in Approvals for the " + "person's yes; nothing is added or sent before, and you can't approve it. Give any field to use instead of what was " + "guessed. Admins only; needs a token with the write scope.", + Arg("recording_id", "integer", "the asset"), + Arg("segment", "integer", "the line's id, as list_dates gives it"), + Arg("line", "integer", "or the line's place in the asset's text, as list_dates gives it"), + Arg("index", "integer", "which of the line's dates, the earliest first", default=0, minimum=0, maximum=50), + Arg("title", "string", "else the line, shortened", max_length=300), + Arg("start", "string", "ISO 8601; else the day said, at the time said", max_length=40), + Arg("end", "string", "ISO 8601; else an hour after start, or a day", max_length=40), + Arg("where", "string", "where it is", max_length=500), + Arg("attendees", "string[]", "addresses ([] for nobody); else those it names"), + Arg("invite", "boolean", "email them an invitation when it's added (off by default)"), + Arg("connection", "string", "the calendar connection, by id or name", max_length=200), + Arg("calendar", "string", "one of its calendars, by id or name", max_length=300), + writes=True, + admin=True, +) +def propose_event( + ctx: Context, + recording_id: int, + segment: int | None, + line: int | None, + index: int, + title: str | None, + start: str | None, + end: str | None, + where: str | None, + attendees: list[str] | None, + invite: bool | None, + connection: str | None, + calendar: str | None, +) -> dict[str, Any]: + _writer(ctx, "propose events") + if not ctx.user.admin: + raise ToolError("only admins propose events: adding one goes through an admin's calendar connection") + try: + ctx.acl.recording(int(recording_id)) + except HTTPException: + raise ToolError(f"there's no asset {recording_id} you can read") from None + if segment is None and line is not None: + seg = ctx.db.one("SELECT record::id(id) AS id FROM segment WHERE recording = $r AND idx = $i", r=int(recording_id), i=int(line)) + segment = (seg or {}).get("id") + if segment is None: + raise ToolError("give the line: segment, or line (its place, as list_dates gives it)") + given = {"title": title, "start": start, "end": end, "where": where, "attendees": attendees, "invite": invite} + given.update(connection=connection, calendar=calendar) + try: + out = event_proposals.propose( + ctx.db, ctx.cfg, ctx.user.as_audit(), int(recording_id), int(segment), int(index or 0), web=ctx.web, + **{k: v for k, v in given.items() if v is not None}, + ) # fmt: skip + except (LookupError, ValueError) as e: + raise ToolError(str(e)) from None + out.pop("event", None) + return {**out, "url": f"{ctx.web}/approvals?id={out['approval']}"} + + @tool( "explore_graph", "Explore the knowledge graph", diff --git a/fastapi_backend/app/api/v1/routes/dates.py b/fastapi_backend/app/api/v1/routes/dates.py index ea391b6e..2a5de053 100644 --- a/fastapi_backend/app/api/v1/routes/dates.py +++ b/fastapi_backend/app/api/v1/routes/dates.py @@ -4,9 +4,10 @@ from fastapi import APIRouter, HTTPException, Query -from app.api.deps import Acl, Cfg, CurrentUser, Db -from app.domain import dates -from app.schemas.dates import DateList +from app.api.deps import Acl, AdminWriter, Cfg, CurrentUser, Db +from app.domain import approvals, dates, event_proposals +from app.email import app_url +from app.schemas.dates import DateList, EventProposal, EventProposed router = APIRouter(tags=["dates"]) @@ -34,4 +35,22 @@ def list_dates( raise HTTPException(400, str(e)) from None spaces, within = acl.scope(ns) out = dates.said(db, spaces, within, a, b, recording, q.strip() or None, max(1, min(limit, 200)), max(0, offset)) - return DateList.model_validate({**out, "reading": bool((cfg.get("analysis") or {}).get("dates"))}) + reading = bool((cfg.get("analysis") or {}).get("dates")) + return DateList.model_validate({**out, "reading": reading, "proposable": event_proposals.offered(db, user.admin)}) + + +@router.post("/dates/propose") +def propose_event(body: EventProposal, acl: Acl, user: AdminWriter, db: Db, cfg: Cfg) -> EventProposed: + """Propose a date said in an asset as a calendar event: an event for a Google Calendar connection that may add + events, prefilled from the line (the day said, a time said in it, the line as its title, who it names), waiting in + Approvals for your yes. Nothing is added or sent until you approve it there; declining it does nothing. Proposing + the same date again while it waits gives the same approval.""" + acl.recording(body.recording_id) + given = body.model_dump(exclude={"recording_id", "segment", "index"}, exclude_unset=True) + try: + out = event_proposals.propose(db, cfg, user.as_audit(), body.recording_id, body.segment, body.index, web=app_url(cfg, db), **given) + except LookupError as e: + raise HTTPException(404, str(e)) from None + except ValueError as e: + raise HTTPException(400, str(e)) from None + return EventProposed.model_validate({**out, "url": approvals.link(cfg, db, out["approval"])}) diff --git a/fastapi_backend/app/domain/event_proposals.py b/fastapi_backend/app/domain/event_proposals.py new file mode 100644 index 00000000..bb00b26c --- /dev/null +++ b/fastapi_backend/app/domain/event_proposals.py @@ -0,0 +1,221 @@ +"""Calendar proposals from dates said (docs/processing.md#dates, "Propose an event"): a date an asset's line gives +("let's meet next Tuesday at 3") becomes an event to add to a Google Calendar connection, prefilled from the line, that +waits in Approvals for the person's yes. Nothing is sent until they approve: the proposal is an approval of the +assistant's `add_event` (connection_tools.py), carried out by ai_tools.approve as one the assistant asked for would be, +and declining it does nothing. + +What is prefilled, all of it shown in the approval and each overridable by the caller: + +- the time: the day the date means (dates.py), at a time of day said in the same line ("at 3", "3pm", "15:30", + "noon"), else all day; a weekend or a week is all of its days. A time without am or pm from 1 to 6 is in the + afternoon, from 7 to 11 in the morning. A month or a year is too long to be one event, so it needs a start. +- the title: the line itself, shortened, so the person recognises what was said; +- who is invited: the addresses in the line and, for an email, its sender and recipients (not the calendar's own + account or the person's), never emailed an invitation unless `invite` is true; +- the description: where it was said, with a link to that moment. + +Only admins propose (adding an event through a connection is an admin's, as the assistant's tool is), only when a +calendar connection an admin let add events exists, and only for an asset they can read. Proposing the same date again +while one waits gives the one waiting. No model is called and nothing leaves Lens until the approval is approved. +""" + +from __future__ import annotations + +import datetime as dt +import re + +from . import connection_tools, dates, events_content, store + +R = store.R +TITLE_MAX = 120 +ATTENDEES_MAX = 20 +# a number that is a day of a month ("from 3 March"), or a count ("by 5 people", "at 2 weeks"), is no time +_NOT_TIME = rf"(?!\s*(?:st|nd|rd|th)\b|\s+(?:of\s+)?(?:{dates._MON})\b|\s+(?:%|percent|people|days?|weeks?|months?|years?|times?|minutes?|hours?)\b)" +TIME_RX = re.compile( + r"(?noon|midday)|(?:(?Pat|@|from|by|around)\s+)?(?P\d{1,2})(?::(?P[0-5]\d))?" + r"\s*(?Pa\.?m\.?|p\.?m\.?|o'?clock)?)(?![\w:/-])" + _NOT_TIME, + re.I, +) +EMAIL_RX = re.compile(r"[^@\s<>,;\"'()]+@[^@\s<>,;\"'()]+\.[A-Za-z]{2,}") + + +def calendars(db): + """The calendar connections that may add events (an admin let them, `write` yes).""" + return [s for s in connection_tools.listing(db, "calendar") if "add_event" in s["tools"]] + + +def offered(db, admin): + """Whether this person can propose an event: an admin, with a calendar connection that may add events.""" + return bool(admin) and bool(calendars(db)) + + +def time_of_day(line, near=None): + """The time of day a line says, as HH:MM, or None: the first one after the date's words (`near`), else the first + in the line. A bare number counts only after "at" (or with am, pm or o'clock), so "3 March" is no time.""" + text = line or "" + start = text.casefold().find(near.casefold()) if near else -1 + after = start + len(near) if start >= 0 else 0 + for chunk in (text[after:], text) if after else (text,): + for m in TIME_RX.finditer(chunk): + if m.group("noon"): + return "12:00" + h, mins, ap = int(m.group("h")), int(m.group("m") or 0), (m.group("ap") or "").casefold().replace(".", "") + if not (m.group("at") or ap or m.group("m")): + continue # a bare number: a day, a count, not a time + if ap in ("am", "pm"): + if not 1 <= h <= 12: + continue + h = (h % 12) + (12 if ap == "pm" else 0) + elif m.group("m") and h > 12: + pass # 15:30 is in the afternoon already + elif 1 <= h <= 6: + h += 12 # at 3: the afternoon + if h > 23: + continue + return f"{h:02d}:{mins:02d}" + return None + + +def _title(line): + text = " ".join(str(line or "").split()) + if len(text) > TITLE_MAX: + text = text[: TITLE_MAX - 1].rsplit(" ", 1)[0].rstrip(",;:") + "…" + return (text[:1].upper() + text[1:]) or None + + +def _said_at(db, rid, segment, index): + rows = db.rows( + "SELECT recording, segment, idx, t0, text, day, until, period FROM date_said WHERE recording = $r AND segment = $s", + r=int(rid), + s=int(segment), + ) + rows.sort(key=lambda x: (x.get("day") or "", x.get("text") or "")) + line = db.one("SELECT record::id(id) AS id, text, speaker FROM $s", s=R("segment", int(segment))) or {} + if not rows: + raise LookupError("no date was read in that line: dates are read while Settings › Analysis › Read dates is on") + said = [r for r in rows if r["text"].casefold() in (line.get("text") or "").casefold()] or rows + if not 0 <= int(index) < len(said): + raise LookupError(f"that line gives {len(said)} date(s): index is 0 to {len(said) - 1}") + return said[int(index)], line + + +def _guests(rec, line, skip): + found = EMAIL_RX.findall(line or "") + mail = events_content._mail(rec) if rec.get("mail") or rec.get("email") else {} + for k in ("from", "to", "cc"): + found += [a for a in mail.get(k) or [] if isinstance(a, str)] + out, seen = [], {s.casefold() for s in skip if s} + for a in found: + addr = (re.search(r"<([^>]+)>", a) or [None, a])[1].strip().casefold() + if addr and addr not in seen and EMAIL_RX.fullmatch(addr): + seen.add(addr) + out.append(addr) + return out[:ATTENDEES_MAX] + + +def draft(db, cfg, rid, segment, index=0, user=None, web=""): + """What would be proposed for one date said (the `index`-th in the line `segment` of asset `rid`): the add_event + fields, prefilled, and where it was said.""" + said, line = _said_at(db, rid, segment, index) + rec = db.one("SELECT record::id(id) AS id, title, space, mail, email FROM $r", r=R("recording", int(rid))) or {} + period, day, until = said.get("period"), said["day"], said.get("until") or said["day"] + at = time_of_day(line.get("text"), said["text"]) if period == "day" else None + if period in ("month", "year"): + start = end = None # too long to be one event: the caller gives start + elif at: + start, end = f"{day}T{at}", None + else: + start = day + end = (dt.date.fromisoformat(until) + dt.timedelta(days=1)).isoformat() if until != day else None + moment = f"{web}/resources/{rid}" + (f"?t={int(said['t0']) // 1000}" if (said.get("t0") or 0) >= 1000 else "") + where = f"“{rec.get('title') or 'an asset'}”" + (f" at {store.tc(said['t0'])}" if said.get("t0") else "") + skip = [(user or {}).get("email")] + [c.get("account") for c in calendars(db)] + return store.clean( + { + "recording_id": int(rid), + "segment": int(segment), + "index": int(index), + "said": said["text"], + "day": day, + "until": until, + "period": period, + "time": at, + "line": line.get("text"), + "title": _title(line.get("text")) or f"{said['text'][:1].upper()}{said['text'][1:]}", + "start": start, + "end": end, + "attendees": _guests(rec, line.get("text"), skip) or None, + "description": f"Said in {where}: “{' '.join(str(line.get('text') or '').split())[:2000]}”\n{moment}", + "space": rec.get("space"), + } + ) + + +def _waiting(db, account, rid, segment, index): + for a in db.rows( + "SELECT record::id(id) AS id, args, summary, preview, status FROM approval " + "WHERE account = $a AND tool = 'add_event' AND status = 'pending' AND args.said.recording = $r", + a=account, + r=int(rid), + ): + s = (a.get("args") or {}).get("said") or {} + if s.get("segment") == int(segment) and s.get("index") == int(index): + return a + return None + + +def propose(db, cfg, user, rid, segment, index=0, web="", **given): + """Propose one date said as an event: a pending approval of add_event, prefilled from the line, with `given` + (title, start, end, where, description, attendees, invite, connection, calendar) in place of what was guessed. + Gives {approval, status, summary, preview, event: the fields}, the one already waiting when it was proposed + before.""" + old = _waiting(db, user["id"], rid, segment, index) + if old: + return {"approval": old["id"], "status": "waiting", "summary": old.get("summary"), "preview": old.get("preview"), "again": True} + d = draft(db, cfg, rid, segment, index, user, web) + pick = {k: v for k, v in given.items() if v is not None} + start = pick.get("start") or d.get("start") + if not start: + raise ValueError(f"“{d['said']}” is a whole {d.get('period')}: give the event's start") + src, cal, args = connection_tools.new_event( + db, + cfg, + pick.get("connection"), + pick.get("calendar"), + pick.get("title") or d["title"], + start, + pick.get("end") if "start" in pick or "end" in pick else d.get("end"), + pick.get("where"), + pick.get("description") or d["description"], + pick.get("attendees") if "attendees" in pick else d.get("attendees"), + bool(pick.get("invite")), + ) + args["said"] = {"recording": int(rid), "segment": int(segment), "index": int(index), "day": d["day"]} + ev = args["event"] + when = ev["start"].get("date") or ev["start"]["dateTime"][:16].replace("T", " ") + summary = f"Add “{ev['summary'][:80]}” on {when} to {cal.get('name') or cal['id']}" + if args.get("invite"): + n = len(ev["attendees"]) + summary += f", inviting {n} {'person' if n == 1 else 'people'}" + preview = connection_tools.event_preview(cal, args) + aid = db.next_id("approval") + db.q( + "CREATE $r CONTENT $d", + r=R("approval", aid), + d=store.clean( + { + "account": user["id"], + "tool": "add_event", + "args": args, + "summary": summary, + "preview": preview, + "status": "pending", + "space": d.get("space"), + "created_at": store.now(), + } + ), + ) + from . import auth + + auth.audit(db, user, "calendar.propose", f"recording:{rid}", {"approval": aid, "segment": int(segment), "source": src["id"]}) + return {"approval": aid, "status": "waiting", "summary": summary, "preview": preview, "event": ev} diff --git a/fastapi_backend/app/schemas/dates.py b/fastapi_backend/app/schemas/dates.py index 8b0076dc..b8a214aa 100644 --- a/fastapi_backend/app/schemas/dates.py +++ b/fastapi_backend/app/schemas/dates.py @@ -2,7 +2,7 @@ from pydantic import Field -from app.schemas.common import ResponseModel +from app.schemas.common import RequestModel, ResponseModel class DateSaid(ResponseModel): @@ -29,3 +29,39 @@ class DateList(ResponseModel): total: int items: list[DateSaid] reading: bool = Field(description="whether analysis reads dates (Settings → Analysis → Read dates); off, nothing new is read") + proposable: bool = Field( + False, + description="whether you can propose a date as an event (POST /dates/propose): an admin, with a calendar connection that may add events", + ) + + +class EventProposal(RequestModel): + """A date said, to propose as an event: each field left out is prefilled from the line.""" + + recording_id: int + segment: int = Field(description="the line's id, as GET /dates gives it") + index: int = Field(0, ge=0, le=50, description="which of the line's dates, the earliest first, when it gives more than one") + title: str | None = Field(None, max_length=300, description="else the line itself, shortened") + start: str | None = Field( + None, + max_length=40, + description="ISO 8601: a date alone is all day; a time without an offset is in the calendar's time zone. Else the day said, at the time said in the line", + ) + end: str | None = Field(None, max_length=40, description="else an hour after start, or a day for a day") + where: str | None = Field(None, max_length=500) + description: str | None = Field(None, max_length=8000, description="else where it was said, with a link to that moment") + attendees: list[str] | None = Field( + None, max_length=50, description="addresses; else those in the line and, for an email, its sender and recipients ([] for nobody)" + ) + invite: bool = Field(False, description="email them an invitation when it's added") + connection: str | None = Field(None, max_length=200, description="the calendar connection, by id or name (else the only one)") + calendar: str | None = Field(None, max_length=300, description="one of its calendars, by id or name (else its first)") + + +class EventProposed(ResponseModel): + approval: int = Field(description="the approval waiting for your yes: nothing is added until you approve it") + status: str = Field(description="waiting") + summary: str | None = None + preview: str | None = Field(None, description="the event as the approval shows it") + again: bool = Field(False, description="this date was proposed before and still waits: the same approval") + url: str | None = Field(None, description="where you approve it in the web app") diff --git a/fastapi_backend/tests/api/test_event_proposals.py b/fastapi_backend/tests/api/test_event_proposals.py new file mode 100644 index 00000000..0310f526 --- /dev/null +++ b/fastapi_backend/tests/api/test_event_proposals.py @@ -0,0 +1,155 @@ +"""Dates said become calendar proposals (event_proposals.py; docs/processing.md#dates, Propose an event): a date in a +recording's line is proposed as an event, prefilled from the line, that waits in Approvals; approved, the calendar adds +it (a fake Google Calendar here), declined, nothing happens. Only admins propose, and only with a calendar connection +that may add events; agents propose over MCP but can't approve.""" + +from __future__ import annotations + +import json + +import pytest + +from app.domain import event_proposals, ingest, jobs, sources, store +from tests import fake_google +from tests.api.test_connection_tools import calendar # noqa: F401 - the fake Google Calendar +from tests.api.test_google_calendar_source import TEAM +from tests.api.test_mcp import tool +from tests.api.test_oauth import _bearer, _grant +from tests.helpers import drain, login, make_user + +R = store.R +CALL = """Ann|N|Thanks for calling about the budget. +Ann|N|Let's meet next Tuesday at 3 with tom@example.org to go over it. +Ben|N|And the whole of next weekend is the regatta.""" + + +def _connect(db, cfg, cal, write=True): + key, public = fake_google.service_account_key() + cal.service_accounts[key["client_email"]] = public + cal.calendars[TEAM]["readers"].add(key["client_email"]) + cal.calendars[TEAM]["writers"] = {key["client_email"]} + params = {"calendars": TEAM, **({"write": "yes"} if write else {})} + return sources.create(db, cfg, "Team calendar", "gcal", params, {"service_account_credentials": json.dumps(key)}) + + +def _call(db, cfg): + cfg["analysis"]["dates"] = True + rid = ingest.import_text(db, cfg, "pods", CALL, "Call with Tom", "auto") + db.q("UPDATE $r SET recorded_at = '2026-10-14T10:00:00'", r=R("recording", rid)) # a Wednesday + jobs.enqueue(db, rid, ["analyze"]) + drain(db, cfg) + return rid + + +@pytest.fixture +def root(client, db): + make_user(db, "root@x.io", "root password 1", admin=True) + return login(client, "root@x.io", "root password 1") + + +def _dates(client, h, rid): + r = client.get("/api/v1/dates", headers=h, params={"recording": rid}) + assert r.status_code == 200, r.text + return r.json() + + +def test_a_date_said_is_proposed_and_added_only_once_approved(client, db, cfg, calendar, root): # noqa: F811 + _connect(db, cfg, calendar) + rid = _call(db, cfg) + got = _dates(client, root, rid) + assert got["proposable"] is True + tue = next(d for d in got["items"] if d["said"] == "next Tuesday") + + r = client.post("/api/v1/dates/propose", headers=root, json={"recording_id": rid, "segment": tue["segment"]}) + assert r.status_code == 200, r.text + p = r.json() + assert p["status"] == "waiting" and p["url"].endswith(f"/approvals?id={p['approval']}") + assert p["summary"] == "Add “Let's meet next Tuesday at 3 with tom@example.org to go over it.” on 2026-10-20 15:00 to Harbour team" + assert "Starts: 2026-10-20T15:00:00 (Europe/Berlin)" in p["preview"] and "Ends: 2026-10-20T16:00:00" in p["preview"] + assert "Invites: tom@example.org (not emailed)" in p["preview"] and "Said in “Call with Tom” at" in p["preview"] + assert not calendar.added # nothing is sent before the yes + again = client.post("/api/v1/dates/propose", headers=root, json={"recording_id": rid, "segment": tue["segment"]}).json() + assert again["approval"] == p["approval"] and again["again"] is True # the same one, still waiting + + waiting = client.get("/api/v1/approvals", headers=root, params={"status": "pending"}).json() + assert [(a["id"], a["tool"], a["can"]["approve"]) for a in waiting] == [(p["approval"], "add_event", True)] + r = client.post(f"/api/v1/approvals/{p['approval']}", headers=root, json={"decision": "approve"}) + assert r.status_code == 200, r.text + assert r.json()["status"] == "added" + [(cal, event, q)] = calendar.added + assert cal == TEAM and q["sendUpdates"] == "none" + assert event["start"] == {"dateTime": "2026-10-20T15:00:00", "timeZone": "Europe/Berlin"} + assert event["summary"].startswith("Let's meet next Tuesday at 3") and event["attendees"] == [{"email": "tom@example.org"}] + + # a weekend is all of its days; what the caller gives replaces what was guessed + wk = next(d for d in got["items"] if d["said"] == "next weekend") + body = {"recording_id": rid, "segment": wk["segment"], "title": "Regatta", "attendees": []} + p2 = client.post("/api/v1/dates/propose", headers=root, json=body).json() + assert "Starts: 2026-10-24\nEnds: 2026-10-26" in p2["preview"] and "Invites" not in p2["preview"] + + +def test_a_declined_proposal_adds_nothing(client, db, cfg, calendar, root): # noqa: F811 + _connect(db, cfg, calendar) + rid = _call(db, cfg) + tue = next(d for d in _dates(client, root, rid)["items"] if d["said"] == "next Tuesday") + p = client.post("/api/v1/dates/propose", headers=root, json={"recording_id": rid, "segment": tue["segment"]}).json() + r = client.post(f"/api/v1/approvals/{p['approval']}", headers=root, json={"decision": "decline"}) + assert r.status_code == 200 and r.json()["status"] == "declined" + assert not calendar.added + assert db.one("SELECT status FROM $r", r=R("approval", p["approval"]))["status"] == "declined" + + +def test_offered_only_to_admins_with_a_calendar_that_adds_events(client, db, cfg, calendar, root): # noqa: F811 + _connect(db, cfg, calendar, write=False) # reads only + rid = _call(db, cfg) + got = _dates(client, root, rid) + assert got["proposable"] is False + seg = got["items"][0]["segment"] + r = client.post("/api/v1/dates/propose", headers=root, json={"recording_id": rid, "segment": seg}) + assert r.status_code == 400 and "no calendar connection may add events" in r.json()["detail"] + make_user(db, "ed@x.io", "editor password 1", roles={"pods": "editor"}) + ed = login(client, "ed@x.io", "editor password 1") + assert _dates(client, ed, rid)["proposable"] is False + assert client.post("/api/v1/dates/propose", headers=ed, json={"recording_id": rid, "segment": seg}).status_code == 403 + first = db.one("SELECT record::id(id) AS id FROM segment WHERE recording = $r AND idx = 0", r=rid)["id"] # no date there + assert client.post("/api/v1/dates/propose", headers=root, json={"recording_id": rid, "segment": first}).status_code == 404 + assert not db.rows("SELECT * FROM approval") + + +def test_agents_propose_over_mcp_but_cannot_approve(client, db, cfg, calendar, root): # noqa: F811 + _connect(db, cfg, calendar) + rid = _call(db, cfg) + _, tokens = _grant(client, root, scope="write") + listed = tool(client, _bearer(tokens), "list_dates", recording_id=rid) + assert listed["proposable"] is True + tue = next(d for d in listed["dates"] if d["said"] == "next Tuesday") + out = tool(client, _bearer(tokens), "propose_event", recording_id=rid, line=tue["line"], start="2026-10-20T09:30") + assert out["status"] == "waiting" and "on 2026-10-20 09:30" in out["summary"] + # an app's token can't approve it (only the person, signed in), but can decline it + shown = client.get(f"/api/v1/approvals/{out['approval']}", headers=_bearer(tokens)).json() + assert shown["can"] == {"approve": False, "decline": True, "edit": False, "always": False} + r = client.post(f"/api/v1/approvals/{out['approval']}", headers=_bearer(tokens), json={"decision": "approve"}) + assert r.status_code == 403 and "an app can only decline it" in r.json()["detail"] + assert not calendar.added + r = client.post(f"/api/v1/approvals/{out['approval']}", headers=_bearer(tokens), json={"decision": "decline"}) + assert r.status_code == 200 and r.json()["status"] == "declined" and not calendar.added + + +@pytest.mark.parametrize( + ("line", "near", "at"), + [ + ("Let's meet next Tuesday at 3", "next Tuesday", "15:00"), + ("next Tuesday at 10", "next Tuesday", "10:00"), + ("call me tomorrow, 3pm works", "tomorrow", "15:00"), + ("tomorrow at 9:45 a.m.", "tomorrow", "09:45"), + ("on 3 March 2027 from 15:30", "3 March 2027", "15:30"), + ("lunch at noon tomorrow", "tomorrow", "12:00"), + ("at 8 on Monday, and next Friday at 4 o'clock", "next Friday", "16:00"), + ("see you on 3 March 2027", "3 March 2027", None), + ("next Tuesday, 20 people", "next Tuesday", None), + ("next Tuesday, from 3 March on", "next Tuesday", None), + ("next Tuesday, by 5 people", "next Tuesday", None), + ], +) +def test_the_time_of_day_is_read_from_the_line(line, near, at): + assert event_proposals.time_of_day(line, near) == at diff --git a/fastapi_backend/tests/api/test_mcp.py b/fastapi_backend/tests/api/test_mcp.py index 3571ee1e..b247404e 100644 --- a/fastapi_backend/tests/api/test_mcp.py +++ b/fastapi_backend/tests/api/test_mcp.py @@ -71,8 +71,10 @@ def test_clients_are_sent_to_sign_in(client, app, env): tools = rpc(client, env["h"], "tools/list")["result"]["tools"] assert [t["name"] for t in tools] == list(mcp_tools.TOOLS) assert all(t["inputSchema"]["type"] == "object" for t in tools) - # every tool only reads, except suggesting a topic and asking for a graph change (which need the write scope) + # every tool only reads, except proposing an event, suggesting a topic and asking for a graph change (which need + # the write scope), and the rest below assert [t["name"] for t in tools if not t["annotations"]["readOnlyHint"]] == [ + "propose_event", "suggest_topic", "propose_graph_change", "undo_graph_change", diff --git a/nextjs-frontend/__tests__/recording-entities-edit.test.tsx b/nextjs-frontend/__tests__/recording-entities-edit.test.tsx index 02c6344d..202d0ecd 100644 --- a/nextjs-frontend/__tests__/recording-entities-edit.test.tsx +++ b/nextjs-frontend/__tests__/recording-entities-edit.test.tsx @@ -13,6 +13,7 @@ jest.mock("@/components/recording/hooks", () => ({ }), })); jest.mock("@/components/topics/recording-topics", () => ({ RecordingTopics: () => null })); +jest.mock("@/components/recording/said-dates", () => ({ SaidDates: () => null })); jest.mock("@/components/entities/entity-drawer", () => ({ EntityDrawer: ({ id, ns, onClose }: { id: number; ns: string; onClose: () => void }) => (
diff --git a/nextjs-frontend/__tests__/said-dates.test.tsx b/nextjs-frontend/__tests__/said-dates.test.tsx new file mode 100644 index 00000000..cf568cff --- /dev/null +++ b/nextjs-frontend/__tests__/said-dates.test.tsx @@ -0,0 +1,99 @@ +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import "@testing-library/jest-dom"; +import type { ReactNode } from "react"; + +import { Dates } from "@/app/openapi-client"; +import type { Approval, DateSaid } from "@/app/openapi-client/types.gen"; +import { groupApprovals } from "@/components/approvals/model"; +import { dayWords, SaidDates } from "@/components/recording/said-dates"; +import { ToastProvider } from "@/components/ui/toast"; + +jest.mock("@/app/openapi-client", () => ({ Dates: { listDates: jest.fn(), proposeEvent: jest.fn() } })); +jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) })); +const seek = jest.fn(); +jest.mock("@/components/player/media", () => ({ usePlayerApi: () => ({ seek }) })); + +const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } }); +const m = (f: unknown) => f as jest.Mock; + +function said(over: Partial): DateSaid { + return { + day: "2026-10-20", + until: "2026-10-20", + period: "day", + said: "next Tuesday", + relative: true, + recording_id: 4, + segment: 31, + idx: 1, + t0: 12_000, + text: "Let's meet next Tuesday at 3.", + ...over, + }; +} + +function wrap(children: ReactNode) { + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + return render( + + {children} + , + ); +} + +describe("dates said in a recording", () => { + beforeEach(() => jest.clearAllMocks()); + + it("proposes an event from a date in one click, and says it waits in Approvals", async () => { + const weekend = said({ + day: "2026-10-24", + until: "2026-10-25", + period: "weekend", + said: "next weekend", + segment: 32, + }); + m(Dates.listDates).mockReturnValue(ok({ total: 2, items: [said({}), weekend], reading: true, proposable: true })); + m(Dates.proposeEvent).mockReturnValue(ok({ approval: 77, status: "waiting", summary: "Add …", again: false })); + wrap(); + expect(await screen.findByText("Tue 20 Oct 2026")).toBeInTheDocument(); + expect(screen.getByText("24 Oct – 25 Oct 2026")).toBeInTheDocument(); + fireEvent.click(screen.getByText("“next Tuesday”")); + expect(seek).toHaveBeenCalledWith(12_000, { manual: true }); + + fireEvent.click(screen.getByRole("button", { name: "Propose an event on Tue 20 Oct 2026" })); + await waitFor(() => + expect(Dates.proposeEvent).toHaveBeenCalledWith( + expect.objectContaining({ body: { recording_id: 4, segment: 31, index: 0 } }), + ), + ); + expect(await screen.findByRole("link", { name: "Waiting in Approvals" })).toHaveAttribute( + "href", + "/approvals?id=77", + ); + expect(screen.getByText("Nothing is added to the calendar until you approve it.")).toBeInTheDocument(); + expect(screen.getAllByRole("button", { name: /^Propose an event/ })).toHaveLength(1); // the weekend's is still there + }); + + it("offers no Propose event without a calendar that adds events, and shows nothing without dates", async () => { + m(Dates.listDates).mockReturnValue(ok({ total: 1, items: [said({})], reading: true, proposable: false })); + const { unmount } = wrap(); + expect(await screen.findByText("Tue 20 Oct 2026")).toBeInTheDocument(); + expect(screen.queryByRole("button", { name: /Propose/ })).not.toBeInTheDocument(); + unmount(); + m(Dates.listDates).mockReturnValue(ok({ total: 0, items: [], reading: true, proposable: true })); + wrap(); + await waitFor(() => expect(Dates.listDates).toHaveBeenCalledTimes(2)); + expect(screen.queryByRole("region", { name: "Dates said" })).not.toBeInTheDocument(); + }); + + it("reads days in words and groups proposals apart from conversations", () => { + expect(dayWords({ day: "2026-11-01", until: "2026-11-30", period: "month" })).toBe("November 2026"); + const a = { id: 1, tool: "add_event", status: "pending", chat: null } as Approval; + const b = { id: 2, tool: "add_event", status: "pending", chat: 5 } as Approval; + expect(groupApprovals([a, b]).map((g) => g.title)).toEqual([ + "Events proposed from dates said", + "The assistant, in a conversation", + ]); + }); +}); diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index ae8379fc..79e63a3b 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -1176,6 +1176,8 @@ export type { EstimateBatchResponse, EstimateBatchResponses, EventInfo, + EventProposal, + EventProposed, EvidencePdfData, EvidencePdfError, EvidencePdfErrors, @@ -3006,6 +3008,11 @@ export type { ProjectSummary, ProjectThread, ProjectUpdate, + ProposeEventData, + ProposeEventError, + ProposeEventErrors, + ProposeEventResponse, + ProposeEventResponses, ProposeGraphChangeData, ProposeGraphChangeError, ProposeGraphChangeErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index af193e2d..b690d93e 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -1454,6 +1454,9 @@ import type { PreviewWatchData, PreviewWatchErrors, PreviewWatchResponses, + ProposeEventData, + ProposeEventErrors, + ProposeEventResponses, ProposeGraphChangeData, ProposeGraphChangeErrors, ProposeGraphChangeResponses, @@ -8798,6 +8801,27 @@ export class Dates { ...options, }); } + + /** + * Propose Event + * + * Propose a date said in an asset as a calendar event: an event for a Google Calendar connection that may add + * events, prefilled from the line (the day said, a time said in it, the line as its title, who it names), waiting in + * Approvals for your yes. Nothing is added or sent until you approve it there; declining it does nothing. Proposing + * the same date again while it waits gives the same approval. + */ + public static proposeEvent( + options: Options, + ): RequestResult { + return (options.client ?? client).post({ + url: "/api/v1/dates/propose", + ...options, + headers: { + "Content-Type": "application/json", + ...options.headers, + }, + }); + } } export class Topics { diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index bd1716bd..4ef541ef 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -6496,6 +6496,12 @@ export type DateList = { * whether analysis reads dates (Settings → Analysis → Read dates); off, nothing new is read */ reading: boolean; + /** + * Proposable + * + * whether you can propose a date as an event (POST /dates/propose): an admin, with a calendar connection that may add events + */ + proposable?: boolean; [key: string]: unknown; }; @@ -7909,6 +7915,123 @@ export type EventInfo = { [key: string]: unknown; }; +/** + * EventProposal + * + * A date said, to propose as an event: each field left out is prefilled from the line. + */ +export type EventProposal = { + /** + * Recording Id + */ + recording_id: number; + /** + * Segment + * + * the line's id, as GET /dates gives it + */ + segment: number; + /** + * Index + * + * which of the line's dates, the earliest first, when it gives more than one + */ + index?: number; + /** + * Title + * + * else the line itself, shortened + */ + title?: string | null; + /** + * Start + * + * ISO 8601: a date alone is all day; a time without an offset is in the calendar's time zone. Else the day said, at the time said in the line + */ + start?: string | null; + /** + * End + * + * else an hour after start, or a day for a day + */ + end?: string | null; + /** + * Where + */ + where?: string | null; + /** + * Description + * + * else where it was said, with a link to that moment + */ + description?: string | null; + /** + * Attendees + * + * addresses; else those in the line and, for an email, its sender and recipients ([] for nobody) + */ + attendees?: Array | null; + /** + * Invite + * + * email them an invitation when it's added + */ + invite?: boolean; + /** + * Connection + * + * the calendar connection, by id or name (else the only one) + */ + connection?: string | null; + /** + * Calendar + * + * one of its calendars, by id or name (else its first) + */ + calendar?: string | null; +}; + +/** + * EventProposed + */ +export type EventProposed = { + /** + * Approval + * + * the approval waiting for your yes: nothing is added until you approve it + */ + approval: number; + /** + * Status + * + * waiting + */ + status: string; + /** + * Summary + */ + summary?: string | null; + /** + * Preview + * + * the event as the approval shows it + */ + preview?: string | null; + /** + * Again + * + * this date was proposed before and still waits: the same approval + */ + again?: boolean; + /** + * Url + * + * where you approve it in the web app + */ + url?: string | null; + [key: string]: unknown; +}; + /** * Extension */ @@ -43185,6 +43308,31 @@ export type ListDatesResponses = { export type ListDatesResponse = ListDatesResponses[keyof ListDatesResponses]; +export type ProposeEventData = { + body: EventProposal; + path?: never; + query?: never; + url: "/api/v1/dates/propose"; +}; + +export type ProposeEventErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type ProposeEventError = ProposeEventErrors[keyof ProposeEventErrors]; + +export type ProposeEventResponses = { + /** + * Successful Response + */ + 200: EventProposed; +}; + +export type ProposeEventResponse = ProposeEventResponses[keyof ProposeEventResponses]; + export type ListTopicsData = { body?: never; path?: never; diff --git a/nextjs-frontend/components/approvals/model.ts b/nextjs-frontend/components/approvals/model.ts index 62e597a5..4e4a48e2 100644 --- a/nextjs-frontend/components/approvals/model.ts +++ b/nextjs-frontend/components/approvals/model.ts @@ -27,13 +27,17 @@ export function groupApprovals(items: Approval[]): ApprovalGroup[] { const groups = new Map(); for (const a of items) { const auto = a.origin?.automation ?? null; - const key = auto != null ? `automation-${auto}` : `chat-${a.chat ?? "none"}`; + // an event proposed from a date said in an asset (POST /dates/propose) belongs to no conversation + const said = auto == null && a.chat == null && a.tool === "add_event"; + const key = auto != null ? `automation-${auto}` : said ? "said" : `chat-${a.chat ?? "none"}`; let g = groups.get(key); if (!g) { g = auto != null ? { key, title: a.automation || `Automation ${auto}`, automation: auto, items: [] } - : { key, title: "The assistant, in a conversation", chat: a.chat ?? undefined, items: [] }; + : said + ? { key, title: "Events proposed from dates said", items: [] } + : { key, title: "The assistant, in a conversation", chat: a.chat ?? undefined, items: [] }; groups.set(key, g); } g.items.push(a); diff --git a/nextjs-frontend/components/recording/entities-tab.tsx b/nextjs-frontend/components/recording/entities-tab.tsx index ea8fb472..e25014d5 100644 --- a/nextjs-frontend/components/recording/entities-tab.tsx +++ b/nextjs-frontend/components/recording/entities-tab.tsx @@ -21,6 +21,7 @@ import { usePlayerApi } from "@/components/player/media"; import { useRec } from "@/components/recording/context"; import { useRecordingEntities } from "@/components/recording/hooks"; import { fold, type EntityRef } from "@/components/recording/model"; +import { SaidDates } from "@/components/recording/said-dates"; import { RecordingTopics } from "@/components/topics/recording-topics"; import { Label } from "@/components/ui/panel"; import { EmptyState, Skeleton } from "@/components/ui/states"; @@ -65,7 +66,7 @@ export function groupEntities( } /** - * Entities tab: the topics it is about, the keyword cloud, then entities by type with mentions in this recording. Choosing one marks its + * Entities tab: the topics it is about, the dates said (with Propose event), the keyword cloud, then entities by type with mentions in this recording. Choosing one marks its * mentions ● on the waveform and jumps to the first; the chevron opens it in the graph, and for editors the pencil * opens it to rename, retype, hide or merge without leaving the recording. */ @@ -96,7 +97,12 @@ export function EntitiesTab() { const max = Math.max(1, ...top.map((k) => k.weight)); const min = Math.min(...top.map((k) => k.weight), max); - const topics = ; + const topics = ( + <> + + + + ); if (!model.entities.length && !model.keywords.length) return ( <> diff --git a/nextjs-frontend/components/recording/said-dates.tsx b/nextjs-frontend/components/recording/said-dates.tsx new file mode 100644 index 00000000..e77d1843 --- /dev/null +++ b/nextjs-frontend/components/recording/said-dates.tsx @@ -0,0 +1,142 @@ +"use client"; + +import { useMutation, useQuery } from "@tanstack/react-query"; +import { CalendarClock, CalendarPlus } from "lucide-react"; +import Link from "next/link"; +import { useState } from "react"; + +import { Dates } from "@/app/openapi-client"; +import type { DateSaid } from "@/app/openapi-client/types.gen"; +import { useApprovalsChanged } from "@/components/approvals/use-approvals"; +import { usePlayerApi } from "@/components/player/media"; +import { Button } from "@/components/ui/button"; +import { Label } from "@/components/ui/panel"; +import { useToast } from "@/components/ui/toast"; +import { data, useApiClient } from "@/lib/api/browser"; + +const WEEKDAYS = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]; +const MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; +const MONTH_NAMES = [ + "January", + "February", + "March", + "April", + "May", + "June", + "July", + "August", + "September", + "October", + "November", + "December", +]; + +/** A date said, in words, the same in every locale: "Tue 20 Oct 2026", a span ("24 Oct – 25 Oct 2026"), a month + * ("November 2026") or a year. */ +export function dayWords(d: Pick): string { + const [y, mo, dd] = d.day.split("-").map(Number); + if (d.period === "year") return String(y); + if (d.period === "month") return `${MONTH_NAMES[mo - 1]} ${y}`; + const wd = WEEKDAYS[new Date(Date.UTC(y, mo - 1, dd)).getUTCDay()]; + if (d.until === d.day) return `${wd} ${dd} ${MONTHS[mo - 1]} ${y}`; + const [y2, mo2, dd2] = d.until.split("-").map(Number); + return `${dd} ${MONTHS[mo - 1]} – ${dd2} ${MONTHS[mo2 - 1]} ${y2}`; +} + +/** Dates said in a recording (docs/processing.md#dates), each going to its moment, with Propose event when the + * person can propose one (an admin, with a calendar connection that may add events): one click makes an event, + * prefilled from the line, that waits in Approvals; nothing is added until they approve it there. */ +export function SaidDates({ id }: { id: number }) { + const client = useApiClient(); + const api = usePlayerApi(); + const toast = useToast(); + const changed = useApprovalsChanged(); + const [asked, setAsked] = useState>({}); + const q = useQuery({ + queryKey: ["recording", id, "dates"], + queryFn: () => data(Dates.listDates({ client, query: { recording: id, limit: 200 } })), + staleTime: 60_000, + }); + const items = q.data?.items ?? []; + const propose = useMutation({ + mutationFn: (d: DateSaid) => + data(Dates.proposeEvent({ client, body: { recording_id: id, segment: d.segment, index: indexIn(items, d) } })), + onSuccess: (out, d) => { + setAsked((a) => ({ ...a, [key(d)]: out.approval })); + changed(); + toast({ + tone: "green", + title: out.again ? "Already waiting in Approvals" : "Proposed. It waits in Approvals", + body: "Nothing is added to the calendar until you approve it.", + }); + }, + onError: (e: Error) => toast({ tone: "red", title: "Couldn’t propose it", body: e.message }), + }); + if (!items.length) return null; + return ( +
+ +
    + {items.map((d) => { + const waiting = asked[key(d)]; + return ( +
  • + + {q.data?.proposable && + (waiting ? ( + + Waiting in Approvals + + ) : ( + + ))} +
  • + ); + })} +
+
+ ); +} + +function cmp(a: string, b: string): number { + return a < b ? -1 : a > b ? 1 : 0; +} + +function key(d: DateSaid): string { + return `${d.segment}:${d.day}:${d.said}`; +} + +/** Which of its line's dates this is, as the API counts them: the earliest first. */ +function indexIn(items: DateSaid[], d: DateSaid): number { + return items + .filter((x) => x.segment === d.segment) + .sort((a, b) => cmp(a.day, b.day) || cmp(a.said, b.said)) + .findIndex((x) => key(x) === key(d)); +} diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 5785d3ea..7c5dc52f 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -19304,6 +19304,48 @@ } } }, + "/api/v1/dates/propose": { + "post": { + "tags": [ + "dates" + ], + "summary": "Propose Event", + "description": "Propose a date said in an asset as a calendar event: an event for a Google Calendar connection that may add\nevents, prefilled from the line (the day said, a time said in it, the line as its title, who it names), waiting in\nApprovals for your yes. Nothing is added or sent until you approve it there; declining it does nothing. Proposing\nthe same date again while it waits gives the same approval.", + "operationId": "propose_event", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EventProposal" + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EventProposed" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/topics": { "get": { "tags": [ @@ -49206,6 +49248,12 @@ "type": "boolean", "title": "Reading", "description": "whether analysis reads dates (Settings \u2192 Analysis \u2192 Read dates); off, nothing new is read" + }, + "proposable": { + "type": "boolean", + "title": "Proposable", + "description": "whether you can propose a date as an event (POST /dates/propose): an admin, with a calendar connection that may add events", + "default": false } }, "additionalProperties": true, @@ -51503,6 +51551,209 @@ ], "title": "EventInfo" }, + "EventProposal": { + "properties": { + "recording_id": { + "type": "integer", + "title": "Recording Id" + }, + "segment": { + "type": "integer", + "title": "Segment", + "description": "the line's id, as GET /dates gives it" + }, + "index": { + "type": "integer", + "maximum": 50.0, + "minimum": 0.0, + "title": "Index", + "description": "which of the line's dates, the earliest first, when it gives more than one", + "default": 0 + }, + "title": { + "anyOf": [ + { + "type": "string", + "maxLength": 300 + }, + { + "type": "null" + } + ], + "title": "Title", + "description": "else the line itself, shortened" + }, + "start": { + "anyOf": [ + { + "type": "string", + "maxLength": 40 + }, + { + "type": "null" + } + ], + "title": "Start", + "description": "ISO 8601: a date alone is all day; a time without an offset is in the calendar's time zone. Else the day said, at the time said in the line" + }, + "end": { + "anyOf": [ + { + "type": "string", + "maxLength": 40 + }, + { + "type": "null" + } + ], + "title": "End", + "description": "else an hour after start, or a day for a day" + }, + "where": { + "anyOf": [ + { + "type": "string", + "maxLength": 500 + }, + { + "type": "null" + } + ], + "title": "Where" + }, + "description": { + "anyOf": [ + { + "type": "string", + "maxLength": 8000 + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "else where it was said, with a link to that moment" + }, + "attendees": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array", + "maxItems": 50 + }, + { + "type": "null" + } + ], + "title": "Attendees", + "description": "addresses; else those in the line and, for an email, its sender and recipients ([] for nobody)" + }, + "invite": { + "type": "boolean", + "title": "Invite", + "description": "email them an invitation when it's added", + "default": false + }, + "connection": { + "anyOf": [ + { + "type": "string", + "maxLength": 200 + }, + { + "type": "null" + } + ], + "title": "Connection", + "description": "the calendar connection, by id or name (else the only one)" + }, + "calendar": { + "anyOf": [ + { + "type": "string", + "maxLength": 300 + }, + { + "type": "null" + } + ], + "title": "Calendar", + "description": "one of its calendars, by id or name (else its first)" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "recording_id", + "segment" + ], + "title": "EventProposal", + "description": "A date said, to propose as an event: each field left out is prefilled from the line." + }, + "EventProposed": { + "properties": { + "approval": { + "type": "integer", + "title": "Approval", + "description": "the approval waiting for your yes: nothing is added until you approve it" + }, + "status": { + "type": "string", + "title": "Status", + "description": "waiting" + }, + "summary": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Summary" + }, + "preview": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Preview", + "description": "the event as the approval shows it" + }, + "again": { + "type": "boolean", + "title": "Again", + "description": "this date was proposed before and still waits: the same approval", + "default": false + }, + "url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Url", + "description": "where you approve it in the web app" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "approval", + "status" + ], + "title": "EventProposed" + }, "Extension": { "properties": { "id": { diff --git a/threat-model.json b/threat-model.json index 07de91e6..4f478c85 100644 --- a/threat-model.json +++ b/threat-model.json @@ -508,6 +508,15 @@ "provenance": "documented", "source": "docs/email-replies.md \"Lens never sends a reply on its own\"" }, + { + "entry_point": "event proposals from dates said", + "parameter": "a recording, a line and which of its dates; the fields to use instead of the prefill (title, start, end, where, description, attendees, invite, connection, calendar); the line's text and an email's addresses that fill the rest", + "attacker_controllable": "yes", + "component": "assistant", + "caller_must_enforce": "none; admins only, for an asset they can read, and only with a Google Calendar connection an admin let add events; the proposal is an approval of add_event with all of the event shown (the line, the time read from it, who is invited and that they aren't emailed unless invite is true), and nothing is added or sent until the person approves it signed in to Lens: an API key, an app, MCP and the assistant can only decline it; declining does nothing; the connection is checked again when approved; no model is called; at most 20 addresses are prefilled and 50 given; each proposal and each addition is in the audit log (calendar.propose, calendar.add)", + "provenance": "documented", + "source": "docs/processing.md \"Propose an event\"" + }, { "entry_point": "admin settings (model, embeddings, speech, decisions, Fedora, bridge, telemetry, IIIF import, rclone remotes, IMAP, Gmail sign-in, iCal, news feeds, MQTT bridge, OIDC issuer, web search, LanguageTool server)", "parameter": "URL, credentials, api_key_env", diff --git a/threat-model.md b/threat-model.md index 8a3786fb..db074ca8 100644 --- a/threat-model.md +++ b/threat-model.md @@ -303,6 +303,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `POST /imports/web`, extension `http` tools, workflow HTTP call nodes, notification targets | URL | yes: editors (web, tools), owners (targets), admins (workflow nodes, host written out) | resource-name, x-network-destination | none | *(maintainer, 2026-10)* | | the assistant's `web_search` and `web_read` (off until an admin picks a provider, `web.search`) | a search query and a page address, both chosen by the model; the results and pages that come back | yes: anyone who may use the assistant, and indirectly whatever it read, through the model; results and pages: whoever runs the site | resource-name, x-network-destination, x-prompt-injection, size | none; queries go only to the admin's provider; pages are fetched through netguard like web import, 10 MB at most (P-SSRF-GUARD); once the answer has read outside text, a query or address neither the person nor a tool gave it asks first (P-INJECTION-EGRESS); results and pages reach the model marked as data; the ledger keeps the provider or host, never the query or page | *(documented, docs/assistant.md "a search or a page read sends words out of Lens")* | | the assistant's connection tools (`connection_tools.py`: `connections`, `search_mail`, `read_mail`, `calendar_events`, `list_files`, `find_files`, `read_file`, `import_from_connection`, `send_mail`, `add_event`, `write_file`) | a connection's id or name, a Gmail search, a message id, a time span, a path; the mail, events and files that come back; the message, event or text file the model would send or write | yes: admins who use the assistant, and indirectly whatever it read, through the model; mail and events: anyone who can email the account or invite it; files: whoever writes to the storage | resource-name, x-prompt-injection, size | none — admins only, and only the tools for the connections there are; reading changes nothing in the account; a file is read up to 20 MB and a message's or file's text up to 11,000 characters; mail, events and files reach the model marked as data, and once read, tools that could carry it out of Lens ask first (P-INJECTION-EGRESS); bringing anything into a namespace waits for the person's yes; sending mail, saving a draft, adding an event and writing a file always wait for the person's yes, even in a setup conversation, with all of it shown (and a note when the answer read outside text), and the model is told never to act on what it read; sending and adding work only on a Gmail or Google Calendar connection an admin let do so (write, which asks Google for gmail.compose or calendar.events), checked again when approved; at most 50 addresses a field, 50,000 characters a message and 200,000 a file; a file is replaced only when asked and approved, and Google Drive connected read only is never written; each change is in the audit log; the ledger keeps what was called, never what came back | *(documented, docs/assistant.md "Mail, calendars and files")* | +| event proposals from dates said (`POST /dates/propose`, MCP `propose_event`; `event_proposals.py`) | a recording, a line and which of its dates; the fields to use instead of the prefill (title, start, end, where, description, attendees, invite, connection, calendar); the line's text and an email's addresses that fill the rest | yes: admins with a write token or session; the line's text and addresses: whoever wrote the asset (an email's sender, a speaker) | data, x-prompt-injection, size | none; admins only, for an asset they can read, and only with a Google Calendar connection an admin let add events; the proposal is an approval of add_event with all of the event shown (the line, the time read from it, who is invited and that they aren't emailed unless invite is true), and nothing is added or sent until the person approves it signed in to Lens: an API key, an app, MCP and the assistant can only decline it; declining does nothing; the connection is checked again when approved; no model is called; at most 20 addresses are prefilled and 50 given; each proposal and each addition is in the audit log (calendar.propose, calendar.add) | *(documented, docs/processing.md "Propose an event")* | | admin settings: model, embeddings, speech, decisions, Fedora, bridge, Telegram bot, Slack app tokens, assistant mailbox, telemetry, IIIF import, rclone remotes, IMAP, Gmail (a Google refresh token or a Workspace service account key), iCal, news feeds, MQTT bridge, OIDC issuer, web search (a SearXNG address, a Brave Search or Tavily key), a LanguageTool server for the clean step's grammar fixes | URL, credentials, `api_key_env` | admin only; documented guards still apply (P-ADMIN-GUARD) | resource-name, x-credential | admin is trusted | *(maintainer, 2026-10)* | | Secret references and the password manager's sign-in: `secrets.*`, a secret field set to `vaultwarden://…` (`PUT /settings/
`, MCP `set_secret_reference`, the assistant's `change_settings`), `GET /secrets`, `GET /secrets/items`, `POST /secrets/test`, MCP `list_secrets` and `list_vault_items`, the assistant's `list_secrets` and `vault_items` | a reference (provider, item, field); the vault's address, account, API key and master password; the items and values the vault sends back | admins only; the vault's answers: whoever can write to Lens's vault account or share items into its folder | resource-name, x-credential, x-network-destination | admin is trusted with the address; a reference is checked against the vault before it's saved, and the vault's own sign-in can't be one; only one folder is read; values go only where the field's own value would, never to the API, tools, approval cards or logs (P-SECRET-REFERENCES); the assistant's changes to the section always ask first | *(documented, docs/secrets.md "People and agents see the reference and whether it could be read, but never the value")* | | replies from remote servers | web page HTML and scripts, IIIF manifests, model replies, OIDC discovery, calendars and their redirects | yes, the remote | data, topology | treat as untrusted | *(inferred, Q17)* | diff --git a/threat-model.yaml b/threat-model.yaml index 19977178..bda62ae3 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -418,6 +418,15 @@ entry_points: control_kinds: [data, x-prompt-injection, size] caller_must_enforce: "drafting needs the right to process the email; the email reaches the model marked untrusted and the system prompt says not to follow it; only the email's own namespace is searched for excerpts; a web address in the draft that neither the email nor the excerpts hold is broken up, and a citation to an excerpt the model wasn't given is taken out; nothing is sent: a draft waits for the person, who sees its recipient (the address Reply-To names) before anything goes" provenance: {kind: documented, source: 'docs/email-replies.md "Lens never sends a reply on its own"'} + - id: "event proposals from dates said" + component: assistant + parameters: + - name: "a recording, a line and which of its dates; the fields to use instead of the prefill (title, start, end, where, description, attendees, invite, connection, calendar); the line's text and an email's addresses that fill the rest" + attacker_controllable: true + x-condition: "admins with a write token or session; the line's text and addresses: whoever wrote the asset (an email's sender, a speaker)" + control_kinds: [data, x-prompt-injection, size] + caller_must_enforce: "none; admins only, for an asset they can read, and only with a Google Calendar connection an admin let add events; the proposal is an approval of add_event with all of the event shown (the line, the time read from it, who is invited and that they aren't emailed unless invite is true), and nothing is added or sent until the person approves it signed in to Lens: an API key, an app, MCP and the assistant can only decline it; declining does nothing; the connection is checked again when approved; no model is called; at most 20 addresses are prefilled and 50 given; each proposal and each addition is in the audit log (calendar.propose, calendar.add)" + provenance: {kind: documented, source: "docs/processing.md \"Propose an event\""} - id: "admin settings (model, embeddings, speech, decisions, Fedora, bridge, telemetry, IIIF import, rclone remotes, IMAP, Gmail sign-in, iCal, news feeds, MQTT bridge, OIDC issuer, web search, LanguageTool server)" component: outbound parameters: