Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
- **Download notes as Markdown.** A note's ⋯ menu in the tree downloads it as a Markdown file, and the Notes page's ⋯
menu downloads all of a namespace's notes as a zip in folders that follow the tree, for Obsidian and other notes apps
([Downloading as Markdown](docs/notes.md#downloading-as-markdown)).
- **Templates.** Notes inside a namespace's Templates note are templates: **New note from a template…** on the Notes
page and in Jump to starts a note from one, with its text, drawings, files and filing. Any note's menu in the tree can
also **Make a copy** ([Templates](docs/notes.md#templates)).
- **Automations for agents.** Agents list, save, try and run automations through the MCP tools `automations`,
`automation_actions`, `save_automation`, `test_automation` and `run_automation`, with problems in plain words. The
assistant saves them switched off and asks before switching one on or running it
Expand Down
9 changes: 4 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,9 +125,10 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
[long recordings](docs/speech-providers.md#long-recordings); [checks](docs/requirements.md#understand-it))
- **Learn from it.** Make a two-host podcast episode about what you pick, every claim cited and checked against its
source, read aloud in two voices. Keep notes beside every asset, entity and topic, written by you or the assistant, in
a tree where notes are renamed, moved and pinned (pins are your own), with today's note one click away, a search by
their words, and Undo and Recently deleted instead of a confirm. Export an asset or a collection as an evidence PDF
with everything Lens found about it, or the parts you pick. ([Podcasts](docs/podcasts.md), [notes](docs/notes.md),
a tree where notes are renamed, moved, copied and pinned (pins are your own), with today's note one click away,
templates to start from, a search by their words, a download as Markdown, and Undo and Recently deleted instead of a
confirm. Export an asset or a collection as an evidence PDF with everything Lens found about it, or the parts you
pick. ([Podcasts](docs/podcasts.md), [notes](docs/notes.md),
[evidence PDFs](docs/api.md#evidence-pdfs); [checks](docs/requirements.md#learn-from-it))
- **Automate it.** Pipelines and workflows on a canvas, routines on a schedule, batch runs over a collection, and
notifications to webhooks, Slack, Discord and Matterbridge. Everything that happens in Lens (new assets, emails,
Expand Down Expand Up @@ -204,8 +205,6 @@ These are open pull requests, not yet part of Lens:
- **The assistant** answering questions about Lens itself, from its docs and this server
([#400](https://github.com/adeelahmad/lense/pull/400)).
- **Chat:** a conversation exported as Markdown, PDF or JSON ([#474](https://github.com/adeelahmad/lense/pull/474)).
- **Notes:** download a note, or all of a namespace's notes, as Markdown
([#419](https://github.com/adeelahmad/lense/pull/419)).
- **Sources:** RSS and Atom news feeds ([#453](https://github.com/adeelahmad/lense/pull/453)).
- **Pictures:** described on this machine with Florence-2 ([#477](https://github.com/adeelahmad/lense/pull/477)), and
low-quality images enhanced before they're read ([#479](https://github.com/adeelahmad/lense/pull/479)).
Expand Down
7 changes: 6 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,8 @@ deleted.
GET /api/v1/notes
POST /api/v1/notes
POST /api/v1/notes/today
GET /api/v1/notes/templates
POST /api/v1/notes/templates
GET /api/v1/notes/search
GET /api/v1/notes/export
GET /api/v1/notes/deleted
Expand All @@ -826,6 +828,7 @@ GET /api/v1/notes/{pid}
PATCH /api/v1/notes/{pid}
DELETE /api/v1/notes/{pid}
POST /api/v1/notes/{pid}/move
POST /api/v1/notes/{pid}/copy
PUT /api/v1/notes/{pid}/pin
DELETE /api/v1/notes/{pid}/pin
GET /api/v1/notes/{pid}/markdown
Expand All @@ -840,7 +843,9 @@ DELETE /api/v1/notes/{pid}/files

Note pages ([Notes](notes.md)), apart from the notes on an asset above. `GET /notes?ns=` is a namespace's page tree
(`all=true` adds the pages of things). `today` gives the namespace's note for `date`, made the first time it's asked
for ([Today's note](notes.md#todays-note)). `search?q=` finds pages by the words in their title, summary or text, title
for ([Today's note](notes.md#todays-note)). `templates` lists the notes inside the namespace's Templates note, and
`POST /notes/templates` makes that note the first time; `copy` makes a new note from a page, with its text, drawings,
files and filing ([Templates](notes.md#templates)). `search?q=` finds pages by the words in their title, summary or text, title
matches first, each with the words around the match ([Finding notes](notes.md#finding-notes)). `DELETE /notes/{pid}`
moves a page to recently deleted for 30 days: `deleted` lists them, `restore` puts one back where it was and
`DELETE /notes/deleted/{pid}` deletes one for good ([Deleting](notes.md#deleting-and-putting-back)). `move` puts a
Expand Down
14 changes: 13 additions & 1 deletion docs/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,8 @@ Free notes sit in a tree in the left navigation, under the main links. The links
long tree has the whole height. A row opens its note, its arrow shows the notes inside it (the tree remembers which are
open) and **+** adds a note inside. Each row's **⋯** menu, which always shows on a touch screen, adds a note inside,
renames the note in place (Enter saves, Escape leaves it as it was), moves it (**Move to…** finds the note to put it in
by name, Enter picks the first; **Move to the top**), pins it ([below](#pinned-notes)), downloads it as Markdown and
by name, Enter picks the first; **Move to the top**), makes a copy beside it ("Copy of …", with its text, drawings
and files), pins it ([below](#pinned-notes)), downloads it as Markdown and
deletes it with Undo. Dragging a note onto another puts it inside, and onto the Notes heading moves it to the top. The
pages of recordings and entities aren't in the tree: they open from those things.

Expand All @@ -143,6 +144,17 @@ day in the namespace in view, made the first time it's asked for: titled with th
everyone there shares; rename or move it like any note. `POST /api/v1/notes/today` with `{ns, date}` (editors) gives
the same note to agents.

## Templates

Each note inside a namespace's **Templates** note is a template. **New note from a template…** in the Notes page's
**⋯** menu, or the same action in Jump to (⌘K), lists them: type to find one, Enter takes the first. The new note has the
template's title (selected, to type over), text, drawings, files, filing and view, and goes at the top of the tree.
With no templates yet, the same place makes the Templates note.

`GET /api/v1/notes/templates?ns=` lists a namespace's templates; `POST /api/v1/notes/templates` with `{ns}` gives its
Templates note, made the first time; `POST /api/v1/notes/{pid}/copy` with optional `{title, parent}` makes a new note
from any page (editors).

## Pinned notes

**Pin** in a note's ⋯ menu in the tree pins it for you: it shows at the top of the tree, under **Pinned**, and first
Expand Down
2 changes: 2 additions & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@ request, to another check that proves the same thing; loosening a requirement or
| A two-host podcast episode, every claim cited and checked against its source, in two voices | [test_podcasts.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_podcasts.py) `test_episode_is_made_by_a_job`, `test_fact_check_rewrites_or_drops`, `test_episode_gets_two_voices`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `podcast-create`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `podcast-create` ≤ 5 clicks |
| Notes beside every asset, entity and topic, written by you or the assistant | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_pages_of_things`, `test_links_and_backlinks`, `test_assistant_keeps_notes`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `note-create` |
| A tree of notes renamed, moved and pinned (pins are your own), today's note one click away, and a search | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_tree`, `test_pins`, `test_search`; [notes-tree.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-tree.test.tsx); [notes-today.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-today.test.tsx) `is one click from the Notes page` |
| A note, or all of a namespace's notes, downloaded as Markdown in folders that follow the tree | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_markdown_export`; [notes-tree.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-tree.test.tsx) `downloads a note as Markdown`; [notes-find.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-find.test.tsx) `downloads every note as Markdown from its menu` |
| Templates a note starts from, and a copy of any note | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_templates_and_copies`; [notes-templates.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-templates.test.tsx); [notes-tree.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-tree.test.tsx) `makes a copy beside the note and opens it` |
| Undo and Recently deleted instead of a confirm | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_recently_deleted`; [notes-delete.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-delete.test.tsx) |
| An asset or a collection exported as an evidence PDF, whole or the parts you pick | [test_evidence.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_evidence.py) `test_a_collection_as_one_evidence_pdf`, `test_everything_lens_found_goes_in`, `test_picking_the_parts`; [evidence-parts.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/evidence-parts.test.tsx) |

Expand Down
34 changes: 34 additions & 0 deletions fastapi_backend/app/api/v1/routes/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from app.domain.store import DB
from app.schemas.common import Ok
from app.schemas.notebook import (
NoteCopy,
NoteDeleted,
NoteFile,
NoteHistory,
Expand All @@ -37,6 +38,9 @@
NotePageMove,
NotePageUpdate,
NoteSearch,
NoteTemplate,
NoteTemplates,
NoteTemplatesCreate,
NoteToday,
NoteTree,
NoteVersion,
Expand Down Expand Up @@ -128,6 +132,26 @@ def today_page(body: NoteToday, user: Writer, acl: Acl, db: Db) -> NotePage:
return _out(acl, db, notebook.get(db, pid), user)


@router.get("/templates")
def list_templates(ns: str, user: CurrentUser, acl: Acl, db: Db) -> NoteTemplates:
"""The namespace's templates: the notes inside its "Templates" note, in the tree's order. Start a note from one
with `POST /notes/{pid}/copy`."""
sid = acl.namespace(ns)
return NoteTemplates(
namespace=ns, root=notebook.templates_root(db, sid), templates=[NoteTemplate(**t) for t in notebook.templates(db, sid)]
)


@router.post("/templates")
def templates_note(body: NoteTemplatesCreate, user: Writer, acl: Acl, db: Db) -> NotePage:
"""The namespace's "Templates" note, made the first time it's asked for: each note put inside it is a template.
Needs editor access."""
sid = acl.namespace(body.ns, "editor")
with domain_errors():
pid = notebook.templates_root(db, sid, user.id)
return _out(acl, db, notebook.get(db, pid), user)


@router.get("/targets")
def link_targets(
ns: str, user: CurrentUser, acl: Acl, db: Db, q: str = "", sign: str = "@", limit: int = Query(20, ge=1, le=50)
Expand Down Expand Up @@ -363,6 +387,16 @@ def move_page(pid: int, body: NotePageMove, user: Writer, acl: Acl, db: Db) -> N
return _out(acl, db, notebook.get(db, pid), user)


@router.post("/{pid}/copy")
def copy_page(pid: int, body: NoteCopy, user: Writer, acl: Acl, db: Db, cfg: Cfg) -> NotePage:
"""A new free note with this page's text, drawings, files, filing and view: a note started from a template, or a
copy. Titled `title` (default the page's), inside `parent` (default the top). Needs editor access."""
_page(acl, db, pid, "editor")
with domain_errors(), _vault():
new = notebook.copy(db, cfg, pid, user.id, body.title, body.parent)
return _out(acl, db, notebook.get(db, new), user)


@router.put("/{pid}/pin")
def pin_page(pid: int, user: CurrentUser, acl: Acl, db: Db) -> NotePage:
"""Pin the page for you: it shows first on your Notes page and in your tree. Pins are each person's own; anyone
Expand Down
59 changes: 59 additions & 0 deletions fastapi_backend/app/domain/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
import json
import os
import re
import shutil
import tempfile
import zipfile

from . import activity, store
Expand Down Expand Up @@ -435,6 +437,63 @@ def today(db, sid, account, date=None):
return pid


TEMPLATES_ROOT = "Templates"


def templates_root(db, sid, account=None):
"""The id of the namespace's "Templates" note, whose notes are its templates; made when `account` is given and
there's none yet. None otherwise."""
root = db.one("SELECT VALUE record::id(id) FROM note_page WHERE space = $s AND templates_root = true LIMIT 1", s=sid)
if root or account is None:
return int(root) if root else None
root = create(
db, sid, account, TEMPLATES_ROOT, "Each note inside this one is a template for new notes.", summary="Notes to start new notes from"
)
db.q("UPDATE $r SET templates_root = true", r=R("note_page", root))
return root


def templates(db, sid):
"""The namespace's templates, the notes inside its "Templates" note, in the tree's order: [{id, title, summary}]."""
root = templates_root(db, sid)
if root is None:
return []
rows = db.rows(
"SELECT record::id(id) AS id, title, summary, position FROM note_page WHERE space = $s AND parent = $p AND about = NONE",
s=sid,
p=root,
)
rows.sort(key=lambda r: (r.get("position") or 0, r["id"]))
return [{"id": r["id"], "title": r["title"], "summary": r.get("summary")} for r in rows]


def copy(db, cfg, pid, account, title=None, parent=None):
"""A new free note with page pid's text, drawings, files, filing and view: to start a note from a template, or
to make a copy. Titled `title` (default the page's), inside `parent` (default the top), dated today. Its id."""
p = get(db, pid)
new = create(
db,
p["space"],
account,
title or p["title"],
p.get("body") or "",
place=p.get("place"),
parent=parent,
doc=p.get("doc"),
view=p.get("view"),
)
# the editor's images and attachments, under the same keys, so the copied drawings and pictures find them
to = get(db, new)
for f in files(db, pid):
_, opened = open_file(db, cfg, pid, f["key"])
with opened as src, tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "file")
with open(path, "wb") as out:
shutil.copyfileobj(src, out)
add_file(db, cfg, to, f["key"], path, f.get("name"), f.get("type"), account)
return new


PINS_MAX = 50 # a person's pinned notes, per namespace


Expand Down
21 changes: 21 additions & 0 deletions fastapi_backend/app/schemas/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,27 @@ class NoteToday(RequestModel):
date: str | None = Field(None, description="YYYY-MM-DD, your today (default: the server's)")


class NoteTemplate(ResponseModel):
id: int
title: str
summary: str | None = None


class NoteTemplates(ResponseModel):
namespace: str
root: int | None = Field(None, description='the "Templates" note: each note inside it is a template; null until made')
templates: list[NoteTemplate]


class NoteTemplatesCreate(RequestModel):
ns: str = Field(description="the namespace whose Templates note to make")


class NoteCopy(RequestModel):
title: str | None = Field(None, min_length=1, max_length=200, description="its title (default: the page's)")
parent: int | None = Field(None, description="put it inside this page (default: the top)")


class NotePageMove(RequestModel):
parent: int | None = Field(None, description="the page to put it inside; null for the top")
before: int | None = Field(None, description="put it before this sibling; null for the end")
Expand Down
40 changes: 40 additions & 0 deletions fastapi_backend/tests/api/test_notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -593,6 +593,46 @@ def test_daily_notes(client, env, db):
assert again["parent"] not in (None, root["id"])


def test_templates_and_copies(client, env, db, cfg, folder):
"""Notes inside the Templates note are templates; a copy of one, or of any note, has its text, files and filing."""
he, hv = env["he"], env["hv"]
assert client.get("/api/v1/notes/templates?ns=pods", headers=hv).json() == {"namespace": "pods", "root": None, "templates": []}
assert client.post("/api/v1/notes/templates", headers=hv, json={"ns": "pods"}).status_code == 403
root = client.post("/api/v1/notes/templates", headers=he, json={"ns": "pods"}).json()
assert (root["title"], root["parent"]) == ("Templates", None)
assert client.post("/api/v1/notes/templates", headers=he, json={"ns": "pods"}).json()["id"] == root["id"] # one
body = f"## Agenda\n\n- [ ] \n\nSee @[the call](recording:{env['a']})."
meeting = _new(client, he, title="Meeting", body=body, summary="For weekly calls", place="area", parent=root["id"])
_new(client, he, title="Trip", parent=root["id"])
got = client.get("/api/v1/notes/templates?ns=pods", headers=hv).json()
assert got["root"] == root["id"]
assert [(t["title"], t["summary"]) for t in got["templates"]] == [("Meeting", "For weekly calls"), ("Trip", None)]
# an image in the template comes with it, under the same key
png = b"\x89PNG\r\n\x1a\n" + b"pixels" * 50
key = base64.urlsafe_b64encode(hashlib.sha256(png).digest()).decode()
r = client.put(
f"/api/v1/notes/{meeting['id']}/blobs",
params={"key": key, "name": "logo.png"},
content=png,
headers={**he, "Content-Type": "image/png"},
)
assert r.status_code == 200, r.text
r = client.post(f"/api/v1/notes/{meeting['id']}/copy", headers=he, json={})
assert r.status_code == 200, r.text
new = r.json()
assert new["id"] != meeting["id"] and (new["title"], new["body"], new["place"], new["parent"]) == ("Meeting", body, "area", None)
assert new["summary"] is None and new["author"] == "person"
assert [x["target"] for x in new["links"]] == [f"recording:{env['a']}"]
assert client.get(f"/api/v1/notes/{new['id']}/blobs", params={"key": key}, headers=hv).content == png
# a copy of any note, titled and placed
inside = client.post(f"/api/v1/notes/{new['id']}/copy", headers=he, json={"title": "Copy", "parent": new["id"]}).json()
assert (inside["title"], inside["parent"]) == ("Copy", new["id"])
# editors only; other namespaces' notes can't be a parent
assert client.post(f"/api/v1/notes/{meeting['id']}/copy", headers=hv, json={}).status_code == 403
other = notebook.create(db, store.ns_id(db, "calls"), None, "Elsewhere")
assert client.post(f"/api/v1/notes/{meeting['id']}/copy", headers=he, json={"parent": other}).status_code == 400


def test_markdown_export(client, env, db):
import io
import zipfile
Expand Down
Loading
Loading