From b3616d43e29a6095cf0b1431fc35a1a0f0c3fb68 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 10:53:41 +0000 Subject: [PATCH 1/3] fix(entities): give names their real types instead of Term The default rules extractor typed a capitalised name as a person only after a title, an organisation only before Inc/Corp-like words and a product only with a digit, so people, places and most organisations came out as Term. name_types.py types each name from the name and the words around it: titles and roles, given names, verbs of saying, the words a name starts or ends with, "a company called X", built-in lists of places, well-known organisations and products, holidays and dates, and the namespace's named speakers. A name gets one type per recording, "and" no longer joins two names, and a role before a name is not part of it. Upgrade step entity-types types the Terms analysis made on existing installs, keeping types people or agents set; it is recorded in the graph history as entity.typed so it can be rolled back. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT --- CHANGELOG.md | 8 + DEVLOG.md | 15 + docs/graph-history.md | 2 +- docs/processing.md | 28 + fastapi_backend/app/domain/analyze.py | 71 +- fastapi_backend/app/domain/entities.py | 77 + .../app/domain/lexicon/given_names.txt | 3223 +++++++++++++++++ .../app/domain/lexicon/organisations.txt | 172 + fastapi_backend/app/domain/lexicon/places.txt | 692 ++++ .../app/domain/lexicon/products.txt | 108 + fastapi_backend/app/domain/migrations.py | 4 +- fastapi_backend/app/domain/name_types.py | 300 ++ fastapi_backend/tests/api/test_enrich.py | 4 +- .../tests/api/test_entity_setup.py | 4 +- fastapi_backend/tests/api/test_evidence.py | 4 +- fastapi_backend/tests/api/test_topics.py | 6 +- .../tests/domain/test_migrations.py | 8 +- .../tests/domain/test_name_types.py | 133 + .../components/routines/history-model.ts | 1 + 19 files changed, 4823 insertions(+), 37 deletions(-) create mode 100644 fastapi_backend/app/domain/lexicon/given_names.txt create mode 100644 fastapi_backend/app/domain/lexicon/organisations.txt create mode 100644 fastapi_backend/app/domain/lexicon/places.txt create mode 100644 fastapi_backend/app/domain/lexicon/products.txt create mode 100644 fastapi_backend/app/domain/name_types.py create mode 100644 fastapi_backend/tests/domain/test_name_types.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 681e608f4..b04a6a5b9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **Names get their real types, not Term.** The built-in extractor now tells people, places, organisations, events, + works and dates apart without a model: by a title or role before a name, a given name, the word a name ends or + starts with (Corp, University of, Summit, Act, River), lists of countries, regions and large cities and of + well-known companies and products, what a name is called ("a company called Lumen") and the namespace's named + speakers. Only names nothing tells stay Terms, and "and" + no longer joins two names into one. Existing entities analysis left as Terms are typed once on upgrade, keeping + every type people set; the change is in the graph's history to roll back + ([How a name gets its type](docs/processing.md#entities-and-the-graph-explorer)). - **Backups hold Lens's files too.** Each backup now also keeps the files under `data_dir` (uploads, objects, media, frames, renditions, web pages, reports, exports, podcasts, certificates), and restoring one puts them back. Files that haven't changed since the previous backup are hard links, so a backup takes only the room of what changed; the diff --git a/DEVLOG.md b/DEVLOG.md index 1b28864f9..28eebc51d 100644 --- a/DEVLOG.md +++ b/DEVLOG.md @@ -2,6 +2,21 @@ Plans and progress for work in flight. Newest first. +## 2026-10-15 · Entity types beyond Term + +Adeel: "every entity comes as term? why!!". The default rules extractor typed a capitalised name PERSON only after a +title, ORG only before Inc/Corp-like words and PRODUCT with a digit; everything else (people, places, most +organisations) was TERM. `name_types.py` now types names from built-in lists (given names, places, well-known +organisations and products, holidays), the words a name starts or ends with, roles and "called" phrases before it, +verbs of saying after it and the namespace's named speakers; a recording gives each name one type. Upgrade step +`entity-types` retypes old analysis-made Terms, skipping types people or agents set (graph history and audit log). + +Refine later: the decision model (Jev or Laya) for the names the rules leave as Terms, in analysis and as an action +on existing Terms (next PR); place, name and brand lists for more languages and regions (Custom vocabulary in +Settings covers one install's own names); type existing entities again when a later recording gives better evidence +(needs a marker for types people set, kept out of the versioned rows); "Google Cloud"-like names (a known company, +then a word) as products. + ## 2026-10-14 · Long recordings in pieces a speech provider takes Adeel: large audio files are segmented automatically to each transcription provider's limits. `speech.Engine` cuts a diff --git a/docs/graph-history.md b/docs/graph-history.md index cd1980ae9..19b5e887d 100644 --- a/docs/graph-history.md +++ b/docs/graph-history.md @@ -31,7 +31,7 @@ it shows in both entities' histories. | --- | --- | | `version` | The graph's version after this change. | | `at` | When. | -| `op` | What happened: `entity.rename`, `entity.describe`, `entity.retype`, `entity.hide`, `entity.show`, `entity.define`, `entity.undefine`, `entity.aliases`, `entity.delete`, `entity.merge`, `entity.unmerge`, `link.add`, `link.remove`, `distinct.add`, `mention.move`, `mention.remove`, `merge.apply`, `link.apply`, `merge.accept`, `link.accept`, `merge.dismiss`, `merge.undo`, `link.undo`, `analysis`. | +| `op` | What happened: `entity.rename`, `entity.describe`, `entity.retype`, `entity.typed` (an upgrade gave old Terms a type), `entity.hide`, `entity.show`, `entity.define`, `entity.undefine`, `entity.aliases`, `entity.delete`, `entity.merge`, `entity.unmerge`, `link.add`, `link.remove`, `distinct.add`, `mention.move`, `mention.remove`, `merge.apply`, `link.apply`, `merge.accept`, `link.accept`, `merge.dismiss`, `merge.undo`, `link.undo`, `analysis`. | | `actor` | Who: the person's email, `routine:`, `analysis`, or `system`. | | `via` | Through what: `web`, `token` (an API token), `oauth` (an app), `assistant`, `mcp`, `routine`, `workflow`, `analysis`, `cli`, `system`. | | `why` | The reason given: a graph change's reason, the model's verdict, why an entity was hidden. | diff --git a/docs/processing.md b/docs/processing.md index adbfa7260..f7610ea3f 100644 --- a/docs/processing.md +++ b/docs/processing.md @@ -110,6 +110,34 @@ Nodes are speakers and named things. Edges are: People, organisations, products, places, events, works and topics are extracted from every transcript. Dates and numbers are extracted too, but hidden unless asked for. +**How a name gets its type.** The built-in extractor (`analysis.entities: rules`, the default) needs no model. It +reads each name and the words around it (`app/domain/name_types.py`): + +- **Person:** a title (Dr Lee, President Obama) or a role before it (our CEO Maria Garcia, my colleague Tom); a + given name first (Sarah Johnson, Maria); a full name followed by a verb of saying (Sarah Connor said); or one of the + namespace's named speakers. +- **Organisation, event, work, place:** the word it ends or starts with (Acme Corp, University of Oxford; Web Summit, + World Cup; Clean Air Act; Hudson River, Mount Fuji), or what it's called (a company called Lumen, an app called + Notion, a town called Hope). +- **Built-in lists:** countries, regions, states and large cities (Place); well-known companies and institutions + (Organisation: Google, Anthropic, NASA) and products and tools (Product: ChatGPT, Kubernetes, Excel); holidays + (Event); days and months on their own (Date). A name with a number in it is a Product (iPhone 16). +- **Term:** anything none of these tells, such as a single unknown word (a brand or a project). A Term can be made a + topic, or retyped on the Entities page (one at a time or ticked together). Names listed in Settings › Analysis › + Custom vocabulary (`Name | TYPE`) always get the type given there. + +A name gets one type per recording, the one most of its lines give it, so "Lee" after "Dr Lee" is a person too. "And" +no longer joins two names into one ("Kenya and Nigeria" are two places). The given names are mostly the US Census +Bureau's frequent first names of 1990 (public domain); the other lists are written for Lens. With +`analysis.entities: spacy`, spaCy's types are used instead, and with +[Find places and dates](configuration.md#anytopdf) anytopdf's places join them. + +Older installs get the same types once, as an upgrade step (`entity-types`, [Upgrades](database.md#upgrades)) when +they first start on this version: entities analysis left as Terms get the type these rules give them from their names +and the lines that mention them. Types people or agents set, entities on a fixed list, hidden ones, Unknown and +Unlabeled stay as they are. The changes are recorded in the graph's history as **Gave a type to** events, so they can +be rolled back there. + The **Entities** page in the web app lists a namespace's entities (search, type and collection filters, a Hidden tab) and opens each in a side panel where editors rename, retype, describe, merge and hide it. diff --git a/fastapi_backend/app/domain/analyze.py b/fastapi_backend/app/domain/analyze.py index 266c70ba3..f4998a80a 100644 --- a/fastapi_backend/app/domain/analyze.py +++ b/fastapi_backend/app/domain/analyze.py @@ -9,7 +9,7 @@ import urllib.request from collections import Counter, defaultdict -from . import enrich, entity_map, entity_setup, graph_history, store, telemetry +from . import enrich, entity_map, entity_setup, graph_history, name_types, store, telemetry STOP = set( """a about above after again against all almost also am an and any are aren as at be because been before being @@ -47,7 +47,7 @@ def words(text): return out -# ---------- named things (rules; spaCy optional) ---------- +# ---------- named things (rules, typed by name_types.py; spaCy optional) ---------- STOPCAP = set( ( "I A The This That These Those It Its So And But Or If When While What Why How Who Where Which Yes No Okay Ok Right " @@ -65,12 +65,7 @@ def words(text): ACR_STOP = {"I", "A", "OK", "AI", "TV", "PM", "AM", "US", "UK", "EU", "OMG"} TITLES = {"Dr", "Mr", "Mrs", "Ms", "Prof", "Professor", "Sir", "Dame", "Senator", "President", "Minister"} CONTRACTION = re.compile(r"['’](?:m|re|ve|ll|d|t)$", re.I) -CONNECT = {"of", "for", "and", "de", "la", "van", "von", "the", "&"} -ORG_END = re.compile( - r"^(Inc|Ltd|Corp|Corporation|Company|Co|LLC|Group|Bank|Therapeutics|Labs?|Laboratories|University|Institute|" - r"Foundation|Agency|Department|Committee|Council|Association|Society|Commission|Ministry|Office|Board|" - r"Partners|Capital|Systems|Technologies|Holdings|Hospital|School|College)$" -) +CONNECT = {"of", "for", "de", "la", "van", "von", "the", "&"} # not "and": "Kenya and Nigeria" are two names TOK = re.compile(r"[^\W_](?:[\w’'&.-]*[^\W_])?") MONTHS = "January|February|March|April|May|June|July|August|September|October|November|December" DATE_RX = re.compile( @@ -98,6 +93,11 @@ def _acr(t): return re.fullmatch(r"[A-Z0-9&]{2,8}", t) is not None and any(c.isupper() for c in t) +def _role(t, nxt): + """A role said before a name, as in "CEO Maria" or "Director Maria", rather than the start of a name.""" + return t.casefold() in name_types.ROLES and _base(t) not in TITLES and (t.isupper() or nxt.casefold() in name_types.given_names()) + + def ent_key(name): return re.sub(r"\s+", " ", _base(name).lower()).strip() @@ -112,7 +112,10 @@ def parse_gazetteer(items): return sorted(out, key=lambda g: -len(g[0])) -def extract_entities(text, gaz=()): +def extract_entities(text, gaz=(), people=frozenset()): + """[(name, type)] in the order said: the gazetteer's names, dates, runs of capitalised words (typed by + name_types.kind from the name and the words around it; TERM when nothing tells) and numbers. `people`: keys of names + known to be people.""" out, taken = [], [] def add(s, e, t, typ): @@ -161,6 +164,8 @@ def initial(k): or CONTRACTION.search(toks[a][0]) ): a += 1 + while a < b and _role(toks[a][0], toks[a + 1][0]): + a += 1 # "CEO Maria Garcia" is Maria Garcia, said after her role while b >= a and toks[b][0] in CONNECT: b -= 1 span = toks[a : b + 1] @@ -175,16 +180,7 @@ def initial(k): continue s0 = span[0][1] name = _base(text[s0 : span[-1][2]]) - last = _base(span[-1][0]) - typ = ( - "PERSON" - if first.rstrip(".") in TITLES and len(span) > 1 - else "ORG" - if ORG_END.match(last) - else "PRODUCT" - if any(ch.isdigit() for x in span for ch in x[0]) and not _acr(name) - else "TERM" - ) + typ = name_types.kind([x[0] for x in span], text[:s0], text[s0 + len(name) :], people) or "TERM" add(s0, s0 + len(name), name, typ) for m in NUM_RX.finditer(text): s = m.group(0).strip() @@ -383,6 +379,29 @@ def _place(db, cfg, nid, rec, setup, listed, first, segs, ch): return known +def people_of(db, nid): + """Keys of the names of the namespace's named speakers, whole and first names: they're people where they're said.""" + out = set() + for n in db.values("SELECT VALUE name FROM speaker WHERE space = $s AND name != NONE AND name != ''", s=nid): + k = ent_key(str(n)) + if k and not k.startswith("speaker"): + out.add(k) + if " " in k and len(k.split()[0]) >= 3: + out.add(k.split()[0]) + return frozenset(out) + + +def one_type(seg_ents): + """Each name gets one type in a recording: the one most of its lines give it, a term only where none gives another + (a Dr Lee in one line is a person in every line).""" + votes = defaultdict(Counter) + for es in seg_ents: + for n, t in es: + votes[ent_key(n)][t] += 1 + best = {k: max(c, key=lambda t: (t != "TERM", c[t])) for k, c in votes.items()} + return [[(n, best.get(ent_key(n), t)) for n, t in es] for es in seg_ents] + + def analyze_recording(db, cfg, rid, seg_ents=None): """Entities, keywords, sections and talk statistics. `seg_ents` (per line, [(name, type)]) replaces the extractor: a workflow's save entities node passes what its nodes found.""" @@ -397,12 +416,15 @@ def analyze_recording(db, cfg, rid, seg_ents=None): extra = entity_map.terms(db, nid, entity_map.chain_of(db, rec.get("collection"))) if listed else [] gaz = parse_gazetteer(list(cfg["analysis"].get("gazetteer") or []) + extra) # the defined entities, however written use_spacy = cfg["analysis"]["entities"] == "spacy" + people = people_of(db, nid) if seg_ents is None and not use_spacy else frozenset() given, seg_ents, toks, wc, tn, ts = seg_ents, [], [], [], Counter(), defaultdict(Counter) for k, s in enumerate(segs): if given is not None: seg_ents.append(list(given[k]) if k < len(given) else []) else: - seg_ents.append(spacy_entities(s["text"], cfg["analysis"]["spacy_model"]) if use_spacy else extract_entities(s["text"], gaz)) + seg_ents.append( + spacy_entities(s["text"], cfg["analysis"]["spacy_model"]) if use_spacy else extract_entities(s["text"], gaz, people) + ) ws = words(s["text"]) toks.append([w for w, _ in ws]) wc.append(len(s["text"].split())) @@ -418,6 +440,7 @@ def analyze_recording(db, cfg, rid, seg_ents=None): es[:] = [(n, typed.get(ent_key(n), t) if t == "TERM" else t) for n, t in es] # a bare term anytopdf knows have = {ent_key(n) for n, _ in es} es.extend(h for h in hints if ent_key(h[0]) not in have) + seg_ents = one_type(seg_ents) if setup["types"] and not listed: # the namespace (or collection) keeps only some types seg_ents = [[(n, t) for n, t in es if entity_setup.keeps(setup, t)] for es in seg_ents] surface = {t: c.most_common(1)[0][0] for t, c in ts.items()} @@ -425,10 +448,14 @@ def analyze_recording(db, cfg, rid, seg_ents=None): titles = section_titles(starts, toks, surface, seg_ents, len(segs)) if segs else [] keys = {ent_key(n) for es in seg_ents for n, _ in es} - {""} alias = {k: max((L for L in keys if len(L) > len(k) and L.endswith(" " + k)), key=len, default=k) for k in keys} - first = {} + first, votes = {}, defaultdict(Counter) for es in seg_ents: for name, typ in es: - first.setdefault(alias.get(ent_key(name), ent_key(name)), (name, typ)) + k = ent_key(name) + first.setdefault(alias.get(k, k), (name, typ)) + votes[alias.get(k, k)][typ] += 1 + # a short form takes its full name's type ("Johnson" after "Dr Sarah Johnson"), a term only when nothing else + first = {k: (n, max(votes[k], key=lambda t: (t != "TERM", votes[k][t]))) for k, (n, _) in first.items()} with graph_history.acting(via="analysis", recording=rid), graph_history.change(db, "analysis", recording=rid) as ch: known = _place(db, cfg, nid, rec, setup, listed, first, segs, ch) over = { diff --git a/fastapi_backend/app/domain/entities.py b/fastapi_backend/app/domain/entities.py index 62848b478..9b5a56b00 100644 --- a/fastapi_backend/app/domain/entities.py +++ b/fastapi_backend/app/domain/entities.py @@ -988,3 +988,80 @@ def unlink(db, a, b): lo, hi = sorted((int(a), int(b))) with graph_history.change(db, "link.remove", entities=[lo, hi]): db.q("DELETE $r", r=R("entity_link", f"{lo}-{hi}")) + + +# ---------- types the old rules left as terms ---------- +MACHINE_OPS = ("analysis", "entity.found", "entity.builtin", "entity.learned", "entity.typed") +TYPED_CHUNK = 500 # entities given types in one graph change +TYPED_LINES = 12 # lines read per entity + + +def typed_by_people(db): + """Ids of entities whose type a person or an agent set or chose: from the graph's history, and from the audit log + for changes made before there was one.""" + out = set() + for ev in db.rows("SELECT ops FROM graph_event WHERE op NOT IN $m", m=list(MACHINE_OPS)): + for o in ev.get("ops") or []: + if o.get("t") == "entity" and o.get("a") and (not o.get("b") or o["b"].get("type") != o["a"].get("type")): + out.add(int(o["k"])) + for r in db.rows("SELECT detail FROM audit_log WHERE action = 'entity.retype'"): + out.update(int(i) for i in (r.get("detail") or {}).get("ids") or [] if str(i).isdigit()) + return out + + +def _lines_of(db, eids): + """{entity id: [the text of up to TYPED_LINES lines that mention it]}.""" + segs = defaultdict(list) + for m in db.rows("SELECT entity, record::id(in) AS seg FROM mentions WHERE entity IN $e", e=list(eids)): + if len(segs[m["entity"]]) < TYPED_LINES: + segs[m["entity"]].append(m["seg"]) + ids = sorted({s for ss in segs.values() for s in ss}) + text = ( + { + r["id"]: r.get("text") or "" + for r in db.rows("SELECT record::id(id) AS id, text FROM segment WHERE id IN $i", i=[R("segment", i) for i in ids]) + } + if ids + else {} + ) + return {e: [text[s] for s in ss if s in text] for e, ss in segs.items()} + + +def migrate_types(db): + """Upgrade step: entities analysis made when its rules called most names terms get the type the rules give them + now (name_types.py), from their name and the lines that mention them. Types people or agents set, entities on a fixed + list, hidden ones and Unknown and Unlabeled are left as they are. Each namespace's changes are recorded in the graph's + history (entity.typed), so they can be rolled back. Safe to run twice: an entity given a type is no longer a term, + and one the rules can't type stays a term.""" + from . import name_types + + held = typed_by_people(db) + for sid in sorted(store.space_names(db)): + setup = entity_setup.effective(db, sid) + codes = entity_setup.type_codes(db, sid) + ents = [ + e + for e in db.rows( + "SELECT record::id(id) AS id, name, defined, builtin, hidden FROM entity WHERE space = $s AND type = 'TERM'", s=sid + ) + if not (e.get("defined") or e.get("builtin") or e.get("hidden")) and e["id"] not in held + ] + if not ents: + continue + people = analyze.people_of(db, sid) + for k in range(0, len(ents), TYPED_CHUNK): + part = ents[k : k + TYPED_CHUNK] + lines = _lines_of(db, [e["id"] for e in part]) + found = defaultdict(list) + for e in part: + t = name_types.guess(e["name"], lines.get(e["id"]) or [], people) + if t and t != "TERM" and t in codes and entity_setup.keeps(setup, t): + found[t].append(e["id"]) + if not found: + continue + with ( + graph_history.acting(actor="system", via="system", why="upgrade: the type analysis's rules now give names it called terms"), + graph_history.change(db, "entity.typed", entities=[i for ids in found.values() for i in ids]), + ): + for t, ids in sorted(found.items()): + db.q("UPDATE $ids SET type = $t", ids=[R("entity", i) for i in ids], t=t) diff --git a/fastapi_backend/app/domain/lexicon/given_names.txt b/fastapi_backend/app/domain/lexicon/given_names.txt new file mode 100644 index 000000000..1e1c5a3b5 --- /dev/null +++ b/fastapi_backend/app/domain/lexicon/given_names.txt @@ -0,0 +1,3223 @@ +# Given names, one per line: a name that starts with one is a person (name_types.py). +# Most are the US Census Bureau's 1990 frequent first names (public domain, as a work of the US government), less +# names that are also common words or places; the others were added by hand for Lens (MIT). +Aaliyah +Aamir +Aarav +Aaron +Abbas +Abbie +Abby +Abdul +Abdullah +Abe +Abebe +Abel +Abena +Abhishek +Abigail +Abraham +Abram +Ada +Adaeze +Adalberto +Adam +Adan +Addie +Addison +Adeel +Adela +Adelaida +Adelaide +Adele +Adelina +Adeline +Adell +Adil +Aditi +Aditya +Adolfo +Adolph +Adrian +Adriana +Adrianne +Adrienne +Advait +Agatha +Agnes +Agnieszka +Agustin +Agustín +Ahmad +Ahmed +Ahmet +Ahsan +Aida +Aiden +Aileen +Aimee +Aisha +Aishwarya +Ajay +Akash +Akbar +Akira +Akosua +Akshay +Al +Alaina +Alan +Alana +Alba +Albert +Alberta +Alberto +Albie +Alda +Alden +Aldo +Alec +Alecia +Alejandra +Alejandro +Alene +Alessandro +Alessia +Aletha +Alex +Alexa +Alexander +Alexandra +Alexandria +Alexei +Alexey +Alexis +Alfie +Alfonso +Alfonzo +Alfred +Alfreda +Alfredo +Ali +Alice +Alicia +Alina +Aline +Alisa +Alisha +Alison +Alissa +Aliya +Aliyah +Allan +Allen +Allene +Allie +Allison +Allyson +Alma +Alok +Alonso +Alonzo +Alpha +Alphonse +Alphonso +Alta +Althea +Alton +Alva +Alvaro +Alvin +Alvina +Alyce +Alyson +Alyssa +Ama +Amado +Amalia +Amanda +Amani +Amara +Amber +Ambrose +Amelia +Amelie +Amie +Amina +Amir +Amit +Ammar +Amna +Amos +Amparo +Amrit +Amy +Amélie +Ana +Anand +Ananya +Anas +Anastasia +Anders +Anderson +Andra +Andre +Andrea +Andreas +Andrei +Andrej +Andres +Andrew +Andria +Andy +Angel +Angela +Angelia +Angelica +Angelina +Angeline +Angelique +Angelita +Angelo +Angie +Anibal +Aniket +Anil +Anissa +Anita +Anjali +Ankit +Ann +Anna +Annabelle +Annamarie +Anne +Annemarie +Annetta +Annette +Annie +Annika +Annmarie +Anthony +Antione +Antionette +Antoine +Antoinette +Anton +Antone +Antonette +Antonia +Antonio +Antony +Antwan +Anum +Anupam +Anushka +Anwar +Aoi +Aoife +April +Araceli +Archie +Arden +Aretha +Aria +Ariana +Ariel +Arif +Arjun +Arleen +Arlen +Arlene +Arlie +Arline +Armand +Armando +Arnold +Arnoldo +Arnulfo +Aron +Arron +Art +Arthur +Arturo +Arun +Arvind +Asa +Asad +Asher +Ashish +Ashlee +Ashleigh +Ashley +Ashok +Asia +Asif +Asma +Astrid +Athena +Atif +Aubrey +Audra +Audrey +August +Augusta +Augustine +Augustus +Aura +Aurelia +Aurelie +Aurelio +Aurora +Aurélie +Austin +Autumn +Ava +Avery +Avi +Avis +Axel +Ayan +Ayesha +Ayo +Ayse +Ayşe +Babar +Baljit +Barack +Baraka +Barb +Barbara +Barbra +Barney +Barrett +Barry +Bart +Barton +Basil +Beatrice +Beatriz +Beau +Becky +Belen +Belinda +Bella +Belle +Belva +Ben +Benedict +Benita +Benito +Benjamin +Benjamín +Bennett +Bennie +Benny +Benoit +Benoît +Benton +Bernadette +Bernadine +Bernard +Bernardo +Bernice +Bernie +Berniece +Berry +Bert +Berta +Bertha +Bertie +Bertram +Beryl +Bess +Bessie +Beth +Bethany +Betsy +Bette +Bettie +Bettina +Betty +Bettye +Beulah +Beverley +Beverly +Bhavna +Bianca +Bilal +Bill +Billie +Billy +Birdie +Birgit +Bjorn +Björn +Blaine +Blair +Blake +Blanca +Blanche +Bo +Bob +Bobbi +Bobbie +Bobby +Bola +Bonita +Bonnie +Booker +Boris +Boyce +Boyd +Brad +Bradford +Bradley +Bradly +Brady +Bram +Branden +Brandi +Brandie +Brandon +Brandy +Brant +Breanna +Brenda +Brendan +Brendon +Brenna +Brent +Brenton +Bret +Brett +Brian +Briana +Brianna +Brianne +Brice +Bridget +Bridgett +Bridgette +Brigitte +Britney +Britt +Brittany +Brittney +Brock +Broderick +Brook +Brooke +Brooklyn +Brooks +Bruce +Bruno +Bryan +Bryant +Bryce +Bryon +Buck +Bud +Buddy +Buford +Burak +Burl +Burt +Burton +Bushra +Buster +Byron +Caitlin +Caleb +Callie +Callum +Calvin +Cameron +Camila +Camilla +Camille +Candace +Candi +Candice +Candida +Candy +Cara +Caren +Carey +Cari +Caridad +Carissa +Carl +Carla +Carlene +Carlo +Carlos +Carlotta +Carlton +Carly +Carmel +Carmela +Carmelita +Carmella +Carmelo +Carmen +Carmine +Carol +Carole +Carolina +Caroline +Carolyn +Carrie +Carrol +Carroll +Carson +Carter +Cary +Caryn +Casandra +Casey +Cassandra +Cassie +Catalina +Catharine +Catherine +Cathie +Cathleen +Cathrine +Cathryn +Cathy +Catina +Catrina +Cecelia +Cecil +Cecile +Cecilia +Cedric +Cedrick +Celeste +Celestine +Celia +Celina +Celine +Cem +Cesar +Chad +Chadwick +Chance +Chanda +Chandra +Chanel +Chang +Chantal +Chantel +Charity +Charla +Charleen +Charlene +Charles +Charley +Charlie +Charline +Charlotte +Charmaine +Chase +Chasity +Chastity +Chauncey +Chelsea +Chelsey +Cheri +Cherie +Cherry +Cheryl +Chester +Chet +Chi +Chiamaka +Chiara +Chidi +Chinedu +Chiquita +Chitra +Chloe +Chloé +Chong +Chris +Christa +Christal +Christen +Christi +Christian +Christie +Christin +Christina +Christine +Christophe +Christopher +Christy +Chrystal +Chuck +Chung +Cian +Ciara +Cindi +Cindy +Clair +Claire +Clara +Clare +Clarence +Clarice +Clarissa +Clark +Claud +Claude +Claudette +Claudia +Claudine +Claudio +Clay +Clayton +Clement +Clemente +Clementine +Cleo +Cletus +Cleveland +Cliff +Clifford +Clifton +Clint +Clinton +Clyde +Cody +Colby +Cole +Coleen +Coleman +Colette +Colin +Colleen +Collette +Collin +Colton +Columbus +Concepcion +Concetta +Connie +Conrad +Constance +Consuelo +Cora +Cordelia +Cordell +Corey +Cori +Corina +Corine +Corinne +Cornelia +Cornelius +Cornell +Corrie +Corrine +Cortez +Cortney +Cory +Courtney +Coy +Craig +Cristal +Cristina +Cristobal +Cristopher +Cristy +Cruz +Crystal +Curt +Curtis +Cynthia +Cyril +Cyrus +Céline +Daan +Daisuke +Daisy +Dale +Dalia +Dallas +Dalton +Damaris +Damian +Damien +Damion +Damon +Dan +Dana +Dane +Danelle +Danette +Daniel +Daniela +Danielle +Danilo +Danish +Danita +Danna +Dannie +Danny +Dante +Daphne +Dara +Darcy +Darell +Daren +Darin +Dario +Darius +Darla +Darleen +Darlene +Darnell +Daron +Darrel +Darrell +Darren +Darrick +Darrin +Darron +Darryl +Darwin +Daryl +Dave +David +Davide +Davis +Dawn +Dawna +Dayna +Dean +Deana +Deandre +Deangelo +Deann +Deanna +Deanne +Debbie +Debby +Debi +Debora +Deborah +Debra +Debrah +Declan +Dee +Deena +Deepak +Deepika +Deidra +Deidre +Deirdre +Del +Delbert +Delfina +Delia +Delilah +Della +Delma +Delmar +Delmer +Delois +Delores +Deloris +Delphine +Demarcus +Demetria +Demetrius +Demis +Dena +Denice +Denis +Denise +Deniz +Dennis +Denny +Denver +Deon +Derek +Derick +Derrick +Deshawn +Desiree +Desmond +Dessie +Destiny +Devin +Devon +Dewayne +Dewey +Dewitt +Dexter +Dian +Diana +Diane +Diann +Dianna +Dianne +Dick +Didier +Diego +Dieter +Dillon +Dina +Dinah +Dinesh +Dino +Dion +Dionne +Dirk +Divya +Dixie +Dmitri +Dmitry +Dollie +Dolly +Dolores +Domenic +Dominga +Domingo +Dominic +Dominick +Dominique +Don +Dona +Donald +Dong +Donn +Donna +Donnell +Donnie +Donny +Donovan +Donte +Dora +Dorcas +Doreen +Dorene +Doretha +Dorian +Doris +Dorothea +Dorothy +Dorris +Dorsey +Dortha +Dorthy +Dottie +Doug +Douglas +Douglass +Dovie +Doyle +Drew +Duane +Duarte +Dudley +Dulce +Duncan +Dustin +Dusty +Dwain +Dwayne +Dwight +Dylan +Earl +Earle +Earlene +Earline +Earnest +Earnestine +Easton +Ebony +Ed +Eddie +Eddy +Edgar +Edgardo +Edison +Edith +Edmond +Edmund +Edmundo +Edna +Eduardo +Edward +Edwardo +Edwin +Edwina +Edythe +Effie +Efrain +Efren +Eileen +Eitan +Ekaterina +Elaine +Elba +Elbert +Elda +Elden +Eldon +Eldridge +Eleanor +Eleanore +Elena +Eli +Elia +Elias +Elida +Elif +Elijah +Elinor +Elisa +Elisabeth +Elise +Eliseo +Elisha +Elissa +Eliza +Elizabeth +Elizebeth +Ella +Ellen +Ellie +Elliot +Elliott +Ellis +Ellsworth +Elma +Elmer +Elmo +Elnora +Elodie +Eloisa +Eloise +Elon +Elouise +Eloy +Elroy +Elsa +Elsie +Elton +Elva +Elvia +Elvin +Elvira +Elvis +Elwood +Elyse +Emanuel +Emeka +Emerson +Emery +Emiko +Emil +Emile +Emilia +Emilie +Emilio +Emily +Emine +Emma +Emmanuel +Emmett +Emmitt +Emory +Emre +Enid +Enoch +Enrique +Eoin +Erasmo +Eric +Erica +Erich +Erick +Ericka +Erik +Erika +Erin +Erlinda +Erma +Erna +Ernest +Ernestina +Ernestine +Ernesto +Ernie +Errol +Ervin +Erwin +Esme +Esmeralda +Esperanza +Essie +Esteban +Estela +Estella +Estelle +Ester +Esther +Ethan +Ethel +Etta +Eugene +Eugenia +Eugenio +Eula +Eulalia +Eunice +Eunji +Eusebio +Eva +Evan +Evangelina +Evangeline +Eve +Evelyn +Everett +Everette +Everly +Evette +Evie +Ewan +Ezekiel +Ezequiel +Ezra +Fabian +Fabio +Fabiola +Fahad +Faisal +Faith +Faiza +Fang +Fannie +Fanny +Farah +Farhan +Fariha +Fatima +Fatimah +Fatma +Faustino +Fausto +Fawad +Fawn +Fay +Faye +Federica +Federico +Fela +Felecia +Felicia +Felicita +Felipe +Felix +Felton +Femi +Femke +Feng +Ferdinand +Fermin +Fern +Fernanda +Fernando +Fidel +Filiberto +Filomena +Finn +Fletcher +Flor +Flora +Florence +Florencia +Florencio +Florentino +Florian +Florine +Flossie +Floy +Floyd +Forest +Forrest +Foster +Fran +Frances +Francesca +Francesco +Francine +Francis +Francisca +Francisco +Francois +Frank +Frankie +Franklin +Franklyn +François +Fraser +Fred +Freda +Freddie +Freddy +Frederic +Frederick +Fredric +Fredrick +Freeman +Freida +Freya +Frieda +Fritz +Funmi +Gabriel +Gabriela +Gabriella +Gabrielle +Gail +Gale +Galen +Ganesh +Garfield +Garland +Garret +Garrett +Garry +Garth +Gary +Gaston +Gaurav +Gavin +Gay +Gaye +Gayla +Gayle +Gaylord +Geeta +Gena +Genaro +Gene +Genesis +Geneva +Genevieve +Genoveva +Geoffrey +George +Georgette +Georgia +Georgiana +Georgianna +Georgie +Georgina +Gerald +Geraldine +Geraldo +Gerard +Gerardo +Geri +Germaine +German +Gerri +Gerry +Gertie +Gertrude +Ghazal +Giacomo +Gianluca +Gianna +Gil +Gilbert +Gilberto +Gilda +Gillian +Gina +Ginger +Ginny +Gino +Giorgio +Giovanni +Gisela +Gita +Giulia +Giuseppe +Gladys +Glen +Glenda +Glenn +Glenna +Gloria +Goldie +Goncalo +Gonzalo +Gonçalo +Gopal +Gordon +Grace +Gracie +Graciela +Grady +Graham +Grant +Granville +Grayson +Greg +Gregg +Gregoria +Gregorio +Gregory +Greta +Gretchen +Griselda +Grover +Guadalupe +Guillaume +Guillermina +Guillermo +Gunther +Gurpreet +Gus +Gussie +Gustavo +Guy +Gwen +Gwendolyn +Günther +Hafsa +Hai +Hailey +Hal +Haley +Hallie +Hamad +Hamid +Hammad +Hamza +Hana +Hank +Hanna +Hannah +Hans +Hao +Hardeep +Haris +Harish +Harlan +Harland +Harley +Harold +Harper +Harpreet +Harriet +Harriett +Harriette +Harris +Harrison +Harry +Haruka +Haruki +Haruto +Harvey +Hasan +Hassan +Hattie +Haydee +Hayden +Hayley +Haywood +Hazel +Heath +Heather +Hector +Heidi +Heike +Heinz +Helen +Helena +Helene +Helga +Hellen +Helmut +Hemant +Henk +Henrietta +Henry +Herb +Herbert +Heriberto +Herlinda +Herman +Herminia +Herschel +Hershel +Hester +Hilario +Hilary +Hilda +Hildegard +Hillary +Hilton +Hina +Hipolito +Hira +Hiram +Hiroshi +Hobert +Holli +Hollie +Hollis +Holly +Homer +Hong +Hope +Horace +Horacio +Horst +Hortencia +Hosea +Houston +Howard +Hoyt +Hua +Hubert +Hudson +Huey +Hugh +Hugo +Hui +Huma +Humberto +Hung +Hunter +Husnain +Hussain +Hussein +Hyman +Hyun +Hélène +Ian +Ibrahim +Ida +Idella +Ido +Ifeanyi +Ignacio +Igor +Ike +Ila +Ilene +Ilya +Ima +Iman +Imelda +Imogen +Imogene +Imran +Ina +Inderjit +India +Indira +Ines +Inez +Ingrid +Inês +Iola +Iona +Ione +Iqbal +Iqra +Ira +Irene +Irfan +Irina +Iris +Irma +Irvin +Irving +Irwin +Isaac +Isabel +Isabell +Isabella +Isabelle +Isaiah +Isaias +Isha +Isiah +Isidro +Isla +Ismael +Ismail +Israel +Issac +Itai +Iva +Ivan +Ivette +Ivonne +Ivory +Ivy +Iván +Jacinto +Jack +Jackie +Jacklyn +Jackson +Jaclyn +Jacob +Jacque +Jacquelin +Jacqueline +Jacquelyn +Jacques +Jacquline +Jade +Jae +Jagdeep +Jaime +Jake +Jalal +Jamaal +Jamal +Jamar +Jameel +Jamel +James +Jamey +Jami +Jamie +Jamison +Jan +Jana +Jane +Janeen +Janel +Janell +Janelle +Janet +Janette +Janice +Janie +Janine +Janis +Janna +Jannette +Jannie +Jaqueline +Jared +Jarod +Jarred +Jarrett +Jarrod +Jarvis +Jasmin +Jasmine +Jason +Jasper +Jaspreet +Javaid +Javed +Javeria +Javier +Jaxon +Jay +Jaya +Jayant +Jayden +Jayme +Jayne +Jayson +Jean +Jeana +Jeanette +Jeanie +Jeanine +Jeanna +Jeanne +Jeannette +Jeannie +Jeannine +Jed +Jeff +Jefferey +Jefferson +Jeffery +Jeffrey +Jeffry +Jena +Jenifer +Jenna +Jennie +Jennifer +Jenny +Jens +Jensen +Jerald +Jeramy +Jere +Jeremiah +Jeremy +Jeri +Jermaine +Jerold +Jerome +Jeromy +Jerrell +Jerri +Jerrod +Jerrold +Jerry +Jesica +Jess +Jesse +Jessica +Jessie +Jesus +Jewel +Jewell +Jian +Jie +Jill +Jillian +Jim +Jimmie +Jimmy +Jing +Jisoo +Jiwoo +Jo +Joan +Joann +Joanna +Joanne +Joao +Joaquin +Joaquín +Jocelyn +Jodi +Jodie +Jody +Joe +Joel +Joelle +Joerg +Joey +Johan +Johanna +John +Johnathan +Johnathon +Johnie +Johnna +Johnnie +Johnny +Johnson +Jolene +Jon +Jonah +Jonas +Jonathan +Jonathon +Joni +Jonnie +Joost +Jordan +Jordon +Jorge +Jose +Josef +Josefa +Josefina +Joseph +Josephine +Josette +Josh +Joshua +Josiah +Josie +Josue +Jovita +Joy +Joyce +João +Juan +Juana +Juanita +Jude +Judi +Judith +Judson +Judy +Jules +Juli +Julia +Julian +Juliana +Julianna +Julianne +Julie +Julien +Juliet +Juliette +Julio +Julius +Juma +Jun +Junaid +June +Jung +Junior +Jurgen +Justin +Justina +Justine +Jyoti +Jörg +Jürgen +Kagiso +Kai +Kaitlin +Kaitlyn +Kajal +Kamala +Kami +Kamran +Kara +Karan +Kareem +Karen +Kari +Karim +Karin +Karina +Karl +Karla +Karol +Karon +Karrie +Karyn +Kasey +Kashif +Katarzyna +Kate +Katelyn +Katharina +Katharine +Katherine +Katheryn +Kathi +Kathie +Kathleen +Kathrine +Kathryn +Kathy +Katie +Katina +Katrina +Katy +Kavya +Kay +Kaye +Kayla +Kazuo +Keenan +Keiko +Keir +Keisha +Keith +Kelechi +Kelley +Kelli +Kellie +Kelly +Kelsey +Kelvin +Kemal +Kemi +Ken +Kendall +Kendra +Kendrick +Kenji +Kenneth +Kenny +Kent +Kenta +Kenton +Kenya +Keri +Kermit +Kerri +Kerrie +Kerry +Kesha +Keshia +Keven +Kevin +Khadija +Khalid +Kia +Kieran +Kim +Kimberlee +Kimberley +Kimberly +King +Kinsley +Kip +Kira +Kiran +Kirby +Kirk +Kirsten +Kisha +Kitty +Klaus +Kofi +Kojo +Korey +Kory +Kraig +Kris +Krishna +Krista +Kristal +Kristen +Kristi +Kristie +Kristin +Kristina +Kristine +Kristofer +Kristopher +Kristy +Kriti +Krystal +Krystle +Krzysztof +Kuldeep +Kunal +Kurt +Kurtis +Kwame +Kyla +Kyle +Kylie +Lacey +Lacy +Ladonna +Lakeisha +Lakesha +Lakisha +Lakshmi +Lamar +Lamont +Lan +Lana +Lance +Landon +Lane +Lanny +Laquita +Lara +Larissa +Larry +Lars +Lashawn +Lashonda +Latanya +Latasha +Latisha +Latonia +Latonya +Latosha +Latoya +Latrice +Latricia +Laura +Laureen +Laurel +Lauren +Laurence +Lauretta +Lauri +Laurie +Lavern +Laverne +Lavonne +Lawanda +Lawrence +Layla +Lazaro +Lea +Leah +Leandro +Leann +Leanna +Leanne +Lee +Leeann +Lei +Leif +Leigh +Leila +Leilani +Leisa +Lela +Leland +Lelia +Lemuel +Len +Lena +Lenard +Lenny +Lenora +Lenore +Leo +Leola +Leon +Leona +Leonard +Leonardo +Leone +Leonel +Leonor +Leonora +Leopoldo +Leora +Leota +Lerato +Leroy +Les +Lesa +Lesley +Leslie +Lessie +Lester +Leta +Letha +Leticia +Letitia +Lettie +Levi +Lewis +Lia +Liam +Libby +Lidia +Lila +Lilia +Lilian +Liliana +Lillian +Lillie +Lilly +Lily +Lin +Lina +Lincoln +Linda +Lindsay +Lindsey +Lindy +Ling +Linnie +Lino +Linwood +Lionel +Liora +Lisa +Lisette +Lissette +Liz +Liza +Lizette +Lizzie +Lloyd +Logan +Lois +Lola +Lolita +Lon +Lona +Lonnie +Lonny +Lora +Loraine +Loren +Lorena +Lorene +Lorenza +Lorenzo +Loretta +Lori +Lorie +Lorine +Lorna +Lorraine +Lorri +Lorrie +Lotte +Lottie +Lou +Louann +Louella +Louie +Louis +Louisa +Louise +Lourdes +Lowell +Loyd +Lu +Luana +Luann +Luanne +Luca +Lucas +Lucia +Luciano +Lucien +Lucila +Lucile +Lucille +Lucinda +Lucio +Lucius +Lucretia +Lucy +Lucía +Luella +Luigi +Luis +Luisa +Lukas +Luke +Lula +Lulu +Luna +Lupe +Lura +Luther +Luz +Lydia +Lyle +Lyman +Lyn +Lynda +Lyndon +Lynette +Lynn +Lynne +Lynnette +Lynwood +Léa +Mabel +Mable +Mac +Mack +Madeleine +Madeline +Madelyn +Madge +Madhu +Madiha +Madonna +Mae +Magda +Magdalena +Magdalene +Maggie +Magnus +Mahesh +Mahmoud +Mahnoor +Mai +Majid +Major +Mala +Malcolm +Malcom +Malgorzata +Malik +Malinda +Malissa +Mallory +Mamie +Mandi +Mandy +Manfred +Manisha +Manoj +Manon +Manpreet +Mansoor +Manuel +Manuela +Mara +Marc +Marcel +Marcela +Marcelina +Marcelino +Marcella +Marcelle +Marcellus +Marcelo +Marci +Marcia +Marcie +Marco +Marcos +Marcus +Marcy +Marek +Margaret +Margarita +Margarito +Margaux +Marge +Margery +Margie +Margo +Margot +Margret +Marguerite +Mari +Maria +Mariah +Mariam +Marian +Mariana +Mariann +Marianna +Marianne +Mariano +Maribel +Maricela +Marie +Marietta +Marilyn +Marilynn +Marina +Mario +Marion +Marisa +Marisela +Marisol +Marissa +Maritza +Marjorie +Marjory +Mark +Markus +Marla +Marlena +Marlene +Marlin +Marlon +Marlyn +Marlys +Marnie +Marquis +Marquita +Marsha +Marshall +Marta +Martha +Martin +Martina +Marty +Martín +Marva +Marvin +Mary +Maryam +Maryann +Maryanne +Marybeth +Maryellen +Maryjane +Marylou +Masato +Mason +Massimo +Mateo +Mathew +Mathieu +Matias +Matilda +Matilde +Matt +Matteo +Matthew +Matthias +Mattie +Matías +Maud +Maude +Maudie +Maura +Maureen +Maurice +Mauricio +Maurine +Mauro +Mavis +Max +Maximilian +Maximo +Maxine +Maxwell +May +Maya +Mayme +Maynard +Mayra +Małgorzata +Mckinley +Meagan +Meaghan +Meera +Megan +Meghan +Mehmet +Mehwish +Mei +Mel +Melanie +Melba +Melinda +Melisa +Melissa +Melodie +Melody +Melva +Melvin +Melvina +Mercedes +Meredith +Merle +Merlin +Merrill +Merry +Mert +Mervin +Mia +Micaela +Micah +Michael +Michaela +Michal +Micheal +Michel +Michele +Michell +Michelle +Mickey +Migdalia +Miguel +Mike +Mikel +Mikhail +Milagros +Milan +Mildred +Miles +Milford +Millard +Millicent +Millie +Milo +Milton +Mimi +Mina +Mindy +Minerva +Ming +Minh +Minho +Minji +Minnie +Miquel +Miranda +Miriam +Mirna +Misbah +Missy +Misti +Misty +Mitch +Mitchel +Mitchell +Mitzi +Modesto +Mohamed +Mohammad +Mohammed +Mohan +Moin +Moises +Moiz +Mollie +Molly +Mona +Monica +Monika +Monique +Monroe +Monte +Monty +Morgan +Moritz +Morris +Morton +Moses +Moshe +Mozelle +Mubashir +Mudassar +Muhammad +Muhammed +Mukesh +Murat +Muriel +Murray +Musa +Mustafa +Myles +Myra +Myrna +Myron +Myrtis +Myrtle +Na +Nabila +Nadeem +Nadia +Nadine +Naeem +Naila +Nakia +Nan +Nancy +Nandini +Nanette +Nannie +Naoki +Naomi +Napoleon +Narendra +Nasser +Natalia +Natalie +Natasha +Nathalie +Nathan +Nathanael +Nathanial +Nathaniel +Natividad +Nauman +Naveed +Naveen +Navjot +Nawaz +Neal +Ned +Nedra +Neeraj +Neha +Neil +Nelda +Nell +Nellie +Nelly +Nelson +Nereida +Nestor +Nettie +Neva +Neville +Newton +Ngozi +Niamh +Nicholas +Nichole +Nick +Nicki +Nickolas +Nicky +Nicolas +Nicole +Nicolás +Nigel +Nikhil +Niki +Nikita +Nikki +Nikolai +Nilda +Nils +Nimra +Nina +Nisha +Nita +Nnamdi +Noa +Noah +Noam +Noble +Noe +Noel +Noelle +Noemi +Nola +Nolan +Noman +Nomvula +Nona +Noor +Nora +Norbert +Norberto +Noreen +Norma +Norman +Normand +Norris +Nova +Obaid +Obinna +Octavia +Octavio +Odell +Odessa +Ofelia +Oisin +Ola +Olaf +Oleg +Oleksandr +Olen +Olena +Oleta +Olga +Olin +Olive +Oliver +Olivia +Olivier +Ollie +Oluwaseun +Oma +Omar +Omer +Ona +Opal +Ophelia +Ora +Oralia +Oren +Orla +Orlando +Orval +Orville +Osama +Oscar +Osman +Osvaldo +Oswaldo +Otis +Otto +Owais +Owen +Ozan +Pablo +Padma +Padraig +Paige +Paisley +Palmer +Pam +Pamala +Pamela +Pankaj +Pansy +Paolo +Paris +Parker +Parvati +Parvez +Pasquale +Pat +Patrice +Patricia +Patrick +Patsy +Patti +Pattie +Patty +Paul +Paula +Paulette +Paulina +Pauline +Pawel +Paweł +Pearl +Pearlie +Pearline +Pedro +Peggy +Penelope +Penny +Percy +Perla +Perry +Pervez +Pete +Peter +Petra +Phil +Philip +Philippe +Phillip +Phillis +Philomena +Phoebe +Phyllis +Pierre +Pieter +Pilar +Piotr +Polly +Pooja +Poppy +Porfirio +Porter +Portia +Prakash +Pranav +Prateek +Precious +Preeti +Preston +Prince +Princess +Priscilla +Priya +Qasim +Qiang +Queen +Quentin +Quincy +Quinn +Quintin +Quinton +Rabia +Rachael +Racheal +Rachel +Rachelle +Radha +Rae +Rafael +Rafaela +Rafiq +Rahul +Rainer +Raj +Rajesh +Rajiv +Rakesh +Raleigh +Ralph +Ramesh +Ramiro +Ramon +Ramona +Randal +Randall +Randell +Randi +Randolph +Randy +Rani +Ranjit +Raphael +Raquel +Rashad +Rashid +Rashmi +Ratan +Rauf +Raul +Raven +Ravi +Ray +Rayford +Raymon +Raymond +Raymundo +Raza +Raúl +Reba +Rebeca +Rebecca +Rebekah +Recep +Reed +Refugio +Reggie +Regina +Reginald +Rehan +Reid +Reina +Reinaldo +Rekha +Ren +Rena +Renae +Renaldo +Renata +Renate +Renato +Rene +Renee +Renita +Reta +Retha +Reuben +Reva +Rex +Rey +Reyes +Reyna +Reynaldo +Rhea +Rhett +Rhoda +Rhonda +Rhys +Ricardo +Riccardo +Rich +Richard +Richelle +Richie +Rick +Rickey +Rickie +Ricky +Rico +Rida +Rigoberto +Riko +Riley +Rishi +Rita +Ritu +Riya +Rizwan +Rob +Robbie +Robbin +Robby +Robert +Roberta +Roberto +Robin +Robyn +Rocco +Rochelle +Rocio +Rocky +Rod +Roderick +Rodger +Rodney +Rodolfo +Rodrick +Rodrigo +Rogelio +Roger +Rohan +Rohit +Roland +Rolando +Rolf +Rolland +Roma +Roman +Romeo +Romona +Ron +Ronald +Ronda +Ronnie +Ronny +Roosevelt +Rory +Rosa +Rosalba +Rosalee +Rosalia +Rosalie +Rosalina +Rosalind +Rosalinda +Rosalyn +Rosanna +Rosanne +Rosario +Roscoe +Rose +Roseann +Roseanne +Rosella +Roselyn +Rosemarie +Rosemary +Rosendo +Rosetta +Rosie +Rosina +Rosita +Roslyn +Ross +Rowan +Rowena +Roxana +Roxann +Roxanna +Roxanne +Roxie +Roy +Royal +Royce +Ruairi +Ruben +Rubin +Rubina +Ruby +Rubye +Rudolf +Rudolph +Rudy +Rueben +Rufus +Rui +Rupert +Russ +Russel +Russell +Rusty +Ruth +Ruthann +Ruthie +Ryan +Saad +Saba +Sabina +Sabine +Sabrina +Sachin +Sadia +Sadie +Sadiq +Saeed +Said +Saif +Saima +Sajid +Sakshi +Sakura +Sal +Sallie +Sally +Salman +Salvador +Salvatore +Sam +Samantha +Sameen +Sameer +Samir +Samira +Sammie +Sammy +Samual +Samuel +Sana +Sandeep +Sander +Sandi +Sandra +Sandy +Sanford +Sang +Sanjay +Sanne +Santa +Santiago +Santo +Santos +Santosh +Saoirse +Saqib +Sara +Sarah +Sarfraz +Sasha +Satoshi +Satya +Saul +Saundra +Saurabh +Savannah +Savita +Scarlett +Scot +Scott +Scottie +Scotty +Seamus +Sean +Sebastian +Sebastien +Sebastián +Sejin +Selena +Selin +Selina +Selma +Seojun +Serena +Sergei +Sergey +Sergio +Seth +Seymour +Shad +Shafiq +Shahid +Shahzad +Shakeel +Shalini +Shameka +Shana +Shanda +Shane +Shankar +Shanna +Shannon +Shanon +Shanta +Shante +Shantel +Shanti +Shari +Sharla +Sharlene +Sharon +Sharonda +Sharron +Sharyn +Shaun +Shauna +Shawn +Shawna +Shayla +Shayne +Shazia +Sheena +Shehzad +Sheila +Shelby +Sheldon +Shelia +Shelley +Shelli +Shellie +Shelly +Shelton +Sheraz +Sheree +Sheri +Sherman +Sherri +Sherrie +Sherry +Sherwood +Sheryl +Shilpa +Shinzo +Shira +Shiraz +Shirlene +Shirley +Shiv +Shoaib +Shon +Shonda +Shota +Shreya +Shweta +Shyam +Sid +Siddharth +Sidney +Sidra +Sienna +Sierra +Sigrid +Silas +Silvia +Simon +Simone +Simran +Sinead +Siobhan +Sipho +Sita +Skylar +Smita +Sneha +Sobia +Socorro +Sofia +Sohail +Sol +Soledad +Solomon +Sonal +Sondra +Sonia +Sonja +Sonny +Sonya +Soon +Sophia +Sophie +Sota +Spencer +Stacey +Staci +Stacia +Stacie +Stacy +Stan +Stanford +Stanley +Stanton +Stefan +Stefanie +Stefano +Stella +Stephan +Stephanie +Stephany +Stephen +Stephenie +Sterling +Steve +Steven +Stevie +Stewart +Stuart +Sudhir +Sue +Sultan +Sumit +Summer +Sundar +Sung +Sunil +Sunita +Sunny +Suresh +Susan +Susana +Susanna +Susanne +Susie +Suzan +Suzanna +Suzanne +Suzette +Sven +Svetlana +Swati +Sybil +Sydney +Sylvester +Sylvia +Sébastien +Tabatha +Tabitha +Tad +Taeyang +Tahir +Tahira +Taimoor +Taimur +Takashi +Takeshi +Talha +Talia +Tamar +Tamara +Tameka +Tamela +Tamera +Tami +Tamika +Tammi +Tammie +Tammy +Tamra +Tana +Tania +Tanisha +Tanner +Tanveer +Tanvi +Tanya +Tao +Tara +Taras +Tariq +Taro +Tarun +Taryn +Tasha +Tatiana +Tawana +Taylor +Ted +Teddy +Tena +Tendai +Teodoro +Tera +Terence +Teresa +Terese +Teresita +Teressa +Teri +Terra +Terrance +Terrell +Terrence +Terri +Terrie +Terry +Tessa +Tessie +Thabo +Thad +Thaddeus +Thanh +Thea +Theda +Thelma +Themba +Theo +Theodora +Theodore +Theresa +Therese +Theron +Thiago +Thibault +Thierry +Thomas +Thorsten +Thurman +Tia +Tiago +Tiana +Tiara +Tiffani +Tiffany +Tillie +Tim +Timmy +Timothy +Tina +Tisha +Titus +Tobias +Toby +Tod +Todd +Tom +Tomas +Tomasa +Tomasz +Tommie +Tommy +Tomoko +Toni +Tonia +Tonja +Tony +Tonya +Tori +Torsten +Tory +Tosha +Tracey +Traci +Tracie +Tracy +Travis +Trena +Trent +Trenton +Treva +Trevor +Trey +Tricia +Trina +Trinidad +Trisha +Trista +Tristan +Troy +Trudy +Truman +Tuan +Tunde +Twila +Twyla +Ty +Tyler +Tyra +Tyree +Tyrell +Tyron +Tyrone +Tyson +Uchenna +Uday +Ulrike +Ulysses +Umar +Una +Urooj +Ursula +Usama +Usha +Usman +Uwe +Uzair +Uzma +Vada +Val +Valarie +Valencia +Valentin +Valentina +Valentine +Valeria +Valerie +Valorie +Van +Vance +Vandana +Vanessa +Varun +Vaughn +Veda +Velda +Velma +Venita +Venus +Vera +Verda +Verla +Vern +Verna +Vernell +Vernice +Vernon +Veronica +Vesta +Vicente +Vickey +Vicki +Vickie +Vicky +Victor +Victoria +Vida +Vidya +Vijay +Vikas +Vikki +Vikram +Vilma +Vinay +Vince +Vincent +Vincenzo +Vinod +Viola +Violet +Violeta +Virat +Virgie +Virgil +Virgilio +Virginia +Vito +Vivek +Vivian +Viviana +Vladimir +Volker +Volodymyr +Von +Vonda +Wade +Wajid +Waldo +Waleed +Walid +Walker +Wallace +Wally +Walter +Walton +Wanda +Wanjiru +Waqas +Ward +Warner +Warren +Waseem +Waylon +Wayne +Wei +Weldon +Wendell +Wendi +Wendy +Werner +Wes +Wesley +Weston +Whitney +Wilber +Wilbert +Wilbur +Wilburn +Wilda +Wiley +Wilford +Wilfred +Wilfredo +Wilhelmina +Will +Willa +Willard +William +Williams +Willie +Willis +Willow +Willy +Wilma +Wilmer +Wilson +Wilton +Windy +Winford +Winfred +Winifred +Winnie +Winnifred +Winona +Winston +Wolfgang +Woodrow +Wyatt +Xavier +Xiao +Xiaoming +Ximena +Xin +Yadira +Yael +Yan +Yann +Yaser +Yash +Yasir +Yasmeen +Yasmin +Yaw +Yesenia +Yi +Ying +Yogesh +Yolanda +Yonatan +Yong +Yosef +Young +Yousef +Youssef +Yu +Yui +Yuki +Yuko +Yulia +Yuri +Yusuf +Yuto +Yvette +Yvonne +Zachariah +Zachary +Zachery +Zack +Zackary +Zafar +Zahid +Zahra +Zaid +Zain +Zainab +Zane +Zanele +Zara +Zeeshan +Zelda +Zella +Zelma +Zenaida +Zeynep +Zhen +Zoe +Zoey +Zoila +Zoya +Zubair +Zunaira +Álvaro +Élodie diff --git a/fastapi_backend/app/domain/lexicon/organisations.txt b/fastapi_backend/app/domain/lexicon/organisations.txt new file mode 100644 index 000000000..a1a47d1e2 --- /dev/null +++ b/fastapi_backend/app/domain/lexicon/organisations.txt @@ -0,0 +1,172 @@ +# Well-known companies and institutions, one per line: names analysis types as ORG wherever they're said +# (name_types.py). Written by hand for Lens (MIT). +Google +Alphabet +Microsoft +Apple +Amazon +Meta +Facebook +Instagram +WhatsApp +Netflix +Tesla +SpaceX +OpenAI +Anthropic +DeepMind +Nvidia +NVIDIA +Intel +AMD +Qualcomm +IBM +Oracle +Salesforce +Adobe +SAP +Cisco +Dell +HP +Lenovo +Samsung +Sony +LG +Huawei +Xiaomi +Alibaba +Tencent +Baidu +ByteDance +TikTok +Uber +Lyft +Airbnb +Spotify +Stripe +PayPal +Shopify +Twitter +LinkedIn +Reddit +Pinterest +Snapchat +Dropbox +Atlassian +GitHub +GitLab +Hugging Face +Mistral +Cohere +Perplexity +xAI +Databricks +Snowflake +Palantir +Cloudflare +Akamai +Vercel +Netlify +Twilio +Zoom +Slack +Notion +Figma +Canva +Discord +Telegram +Yahoo +eBay +Walmart +Costco +IKEA +Nike +Adidas +Puma +Coca-Cola +Pepsi +PepsiCo +Nestlé +Nestle +Unilever +McDonald's +Starbucks +KFC +Toyota +Honda +Nissan +Hyundai +Kia +Volkswagen +BMW +Mercedes-Benz +Audi +Porsche +Ferrari +Ford +Chevrolet +Volvo +Boeing +Airbus +Emirates +Lufthansa +Ryanair +Mastercard +JPMorgan +Goldman Sachs +Morgan Stanley +Citibank +Citigroup +HSBC +Barclays +Deutsche Bank +UBS +BlackRock +Berkshire Hathaway +Deloitte +PwC +KPMG +EY +Accenture +McKinsey +BCG +Pfizer +Moderna +AstraZeneca +Novartis +Roche +Merck +BP +ExxonMobil +Chevron +Aramco +Disney +Warner Bros +BBC +CNN +Reuters +Bloomberg +NASA +ESA +FBI +CIA +NSA +NATO +UNESCO +UNICEF +WHO +IMF +OPEC +FIFA +UEFA +Interpol +Wikipedia +Mozilla +Red Hat +Canonical +Docker +HashiCorp +MongoDB +Confluent +Grafana Labs +Typesafe diff --git a/fastapi_backend/app/domain/lexicon/places.txt b/fastapi_backend/app/domain/lexicon/places.txt new file mode 100644 index 000000000..366a7015d --- /dev/null +++ b/fastapi_backend/app/domain/lexicon/places.txt @@ -0,0 +1,692 @@ +# Countries, regions, states and large cities, one per line: names analysis types as PLACE (name_types.py). +# Written by hand for Lens (MIT). A name here is a place wherever it's said on its own. +Afghanistan +Albania +Algeria +Andorra +Angola +Antigua +Argentina +Armenia +Australia +Austria +Azerbaijan +Bahamas +Bahrain +Bangladesh +Barbados +Belarus +Belgium +Belize +Benin +Bhutan +Bolivia +Bosnia +Herzegovina +Botswana +Brazil +Brunei +Bulgaria +Burkina Faso +Burundi +Cambodia +Cameroon +Canada +Cape Verde +Chile +China +Colombia +Comoros +Congo +Costa Rica +Croatia +Cuba +Cyprus +Czechia +Czech Republic +Denmark +Djibouti +Dominica +Dominican Republic +Ecuador +Egypt +El Salvador +Eritrea +Estonia +Eswatini +Ethiopia +Fiji +Finland +France +Gabon +Gambia +Georgia +Germany +Ghana +Greece +Grenada +Guatemala +Guinea +Guyana +Haiti +Honduras +Hungary +Iceland +India +Indonesia +Iran +Iraq +Ireland +Israel +Italy +Ivory Coast +Jamaica +Japan +Jordan +Kazakhstan +Kenya +Kiribati +Kosovo +Kuwait +Kyrgyzstan +Laos +Latvia +Lebanon +Lesotho +Liberia +Libya +Liechtenstein +Lithuania +Luxembourg +Madagascar +Malawi +Malaysia +Maldives +Mali +Malta +Mauritania +Mauritius +Mexico +Moldova +Monaco +Mongolia +Montenegro +Morocco +Mozambique +Myanmar +Burma +Namibia +Nauru +Nepal +Netherlands +Holland +New Zealand +Nicaragua +Niger +Nigeria +North Korea +North Macedonia +Macedonia +Norway +Oman +Pakistan +Palau +Palestine +Panama +Papua New Guinea +Paraguay +Peru +Philippines +Poland +Portugal +Qatar +Romania +Russia +Rwanda +Samoa +San Marino +Saudi Arabia +Senegal +Serbia +Seychelles +Sierra Leone +Singapore +Slovakia +Slovenia +Somalia +South Africa +South Korea +Korea +South Sudan +Spain +Sri Lanka +Sudan +Suriname +Sweden +Switzerland +Syria +Taiwan +Tajikistan +Tanzania +Thailand +Togo +Tonga +Trinidad +Tobago +Tunisia +Turkey +Türkiye +Turkmenistan +Tuvalu +Uganda +Ukraine +United Arab Emirates +UAE +United Kingdom +Britain +Great Britain +England +Scotland +Wales +Northern Ireland +United States +United States of America +USA +America +Uruguay +Uzbekistan +Vanuatu +Vatican +Venezuela +Vietnam +Yemen +Zambia +Zimbabwe +Greenland +Puerto Rico +Hong Kong +Macau +Tibet +Kurdistan +Catalonia +Bavaria +Siberia +Patagonia +Kashmir +Punjab +Sindh +Balochistan +Khyber Pakhtunkhwa +Gilgit +Baltistan +Bengal +Gujarat +Maharashtra +Kerala +Tamil Nadu +Karnataka +Rajasthan +Goa +Africa +Antarctica +Asia +Europe +North America +South America +Latin America +Central America +Oceania +Australasia +Eurasia +Middle East +Near East +Far East +Southeast Asia +South Asia +East Asia +Central Asia +Western Europe +Eastern Europe +Scandinavia +Balkans +Baltics +Caribbean +Mediterranean +Arctic +Pacific +Atlantic +Persian Gulf +Silicon Valley +Bay Area +Midwest +New England +West Coast +East Coast +Levant +Maghreb +Gaza +West Bank +Crimea +Donbas +Sinai +Himalayas +Alps +Andes +Rockies +Sahara +Nile +Danube +Rhine +Thames +Ganges +Indus +Yangtze +Everest +Kilimanjaro +Alabama +Alaska +Arizona +Arkansas +California +Colorado +Connecticut +Delaware +Florida +Hawaii +Idaho +Illinois +Indiana +Iowa +Kansas +Kentucky +Louisiana +Maine +Maryland +Massachusetts +Michigan +Minnesota +Mississippi +Missouri +Montana +Nebraska +Nevada +New Hampshire +New Jersey +New Mexico +New York +North Carolina +North Dakota +Ohio +Oklahoma +Oregon +Pennsylvania +Rhode Island +South Carolina +South Dakota +Tennessee +Texas +Utah +Vermont +Virginia +Washington +West Virginia +Wisconsin +Wyoming +Ontario +Quebec +British Columbia +Alberta +Manitoba +Saskatchewan +Nova Scotia +New Brunswick +Newfoundland +Labrador +Prince Edward Island +Yukon +Nunavut +New South Wales +Victoria +Queensland +Tasmania +Western Australia +South Australia +Northern Territory +Yorkshire +Lancashire +Cornwall +Devon +Kent +Surrey +Essex +Sussex +Cumbria +Midlands +Tokyo +Delhi +New Delhi +Shanghai +Beijing +Mumbai +Bombay +Kolkata +Calcutta +Chennai +Madras +Bangalore +Bengaluru +Hyderabad +Ahmedabad +Pune +Jaipur +Lucknow +Karachi +Lahore +Islamabad +Rawalpindi +Peshawar +Faisalabad +Multan +Quetta +Sialkot +Gujranwala +Dhaka +Kathmandu +Colombo +Kabul +Tehran +Baghdad +Riyadh +Jeddah +Mecca +Makkah +Medina +Dubai +Abu Dhabi +Sharjah +Doha +Muscat +Kuwait City +Manama +Amman +Beirut +Damascus +Jerusalem +Tel Aviv +Ankara +Istanbul +Cairo +Alexandria +Casablanca +Marrakesh +Rabat +Tunis +Algiers +Tripoli +Khartoum +Addis Ababa +Nairobi +Mombasa +Kampala +Kigali +Dar es Salaam +Zanzibar +Lagos +Abuja +Accra +Dakar +Kinshasa +Luanda +Johannesburg +Cape Town +Durban +Pretoria +Harare +Lusaka +Maputo +Moscow +Saint Petersburg +St Petersburg +Kyiv +Kiev +Kharkiv +Odesa +Odessa +Minsk +Warsaw +Krakow +Kraków +Prague +Vienna +Budapest +Bucharest +Belgrade +Zagreb +Athens +Rome +Milan +Naples +Turin +Florence +Venice +Madrid +Barcelona +Valencia +Seville +Malaga +Lisbon +Porto +Paris +Lyon +Marseille +Toulouse +Bordeaux +Strasbourg +Brussels +Antwerp +Amsterdam +Rotterdam +The Hague +Utrecht +Eindhoven +Berlin +Hamburg +Munich +Frankfurt +Cologne +Stuttgart +Düsseldorf +Dusseldorf +Leipzig +Dresden +Zurich +Zürich +Geneva +Basel +Bern +Copenhagen +Stockholm +Gothenburg +Oslo +Bergen +Helsinki +Reykjavik +Dublin +Cork +Belfast +London +Manchester +Birmingham +Liverpool +Leeds +Glasgow +Edinburgh +Bristol +Oxford +Cambridge +Cardiff +Newcastle +Sheffield +Nottingham +Brighton +Tallinn +Riga +Vilnius +Ljubljana +Bratislava +Sarajevo +Skopje +Tirana +Tbilisi +Yerevan +Baku +Almaty +Astana +Tashkent +Bishkek +Ulaanbaatar +Seoul +Busan +Pyongyang +Osaka +Kyoto +Yokohama +Nagoya +Sapporo +Hiroshima +Fukuoka +Taipei +Shenzhen +Guangzhou +Chengdu +Wuhan +Hangzhou +Nanjing +Tianjin +Chongqing +Xi'an +Kuala Lumpur +Penang +Jakarta +Bali +Bangkok +Phuket +Hanoi +Ho Chi Minh City +Saigon +Manila +Phnom Penh +Yangon +Sydney +Melbourne +Brisbane +Perth +Adelaide +Canberra +Auckland +Wellington +Christchurch +Toronto +Montreal +Vancouver +Ottawa +Calgary +Edmonton +Winnipeg +Quebec City +Halifax +New York City +NYC +Manhattan +Brooklyn +Queens +Bronx +Los Angeles +Chicago +Houston +Phoenix +Philadelphia +San Antonio +San Diego +Dallas +San Jose +Austin +Jacksonville +Fort Worth +Columbus +Charlotte +San Francisco +Indianapolis +Seattle +Denver +Washington DC +Boston +El Paso +Nashville +Detroit +Oklahoma City +Portland +Las Vegas +Memphis +Louisville +Baltimore +Milwaukee +Albuquerque +Tucson +Fresno +Sacramento +Kansas City +Atlanta +Miami +Orlando +Tampa +Raleigh +Minneapolis +Cleveland +Pittsburgh +Cincinnati +St Louis +Saint Louis +New Orleans +Honolulu +Anchorage +Salt Lake City +Palo Alto +Mountain View +Cupertino +Menlo Park +Redmond +Oakland +Berkeley +Santa Clara +Santa Monica +Sunnyvale +Hollywood +Mexico City +Guadalajara +Monterrey +Cancun +Cancún +Havana +Bogota +Bogotá +Medellin +Medellín +Lima +Quito +Caracas +Santiago +Buenos Aires +Montevideo +Asuncion +Asunción +La Paz +Sao Paulo +São Paulo +Rio de Janeiro +Brasilia +Brasília +Panama City +San Juan +Kingston +Nassau diff --git a/fastapi_backend/app/domain/lexicon/products.txt b/fastapi_backend/app/domain/lexicon/products.txt new file mode 100644 index 000000000..87ac040f2 --- /dev/null +++ b/fastapi_backend/app/domain/lexicon/products.txt @@ -0,0 +1,108 @@ +# Well-known products, tools and models, one per line: names analysis types as PRODUCT wherever they're said +# (name_types.py). Written by hand for Lens (MIT). +ChatGPT +GPT +Claude +Gemini +Copilot +Llama +Grok +Siri +Alexa +Cortana +Bard +Midjourney +DALL-E +Stable Diffusion +Whisper +iPhone +iPad +iPod +MacBook +iMac +Mac +macOS +iOS +iPadOS +watchOS +Android +Windows +Linux +Ubuntu +Debian +Fedora +Chrome +Chromium +Firefox +Safari +Gmail +Outlook +Excel +PowerPoint +Teams +OneDrive +SharePoint +Google Drive +Google Docs +Google Sheets +YouTube +Kubernetes +Terraform +Ansible +Jenkins +Postgres +PostgreSQL +MySQL +SQLite +Redis +Kafka +Elasticsearch +OpenSearch +SurrealDB +Python +JavaScript +TypeScript +Rust +Golang +Java +Kotlin +Swift +React +Next.js +Vue +Angular +Node.js +Django +FastAPI +Flask +PyTorch +TensorFlow +Jupyter +VS Code +Xcode +Jira +Confluence +Trello +Asana +Salesforce CRM +Photoshop +Illustrator +Premiere +Lightroom +Kindle +PlayStation +Xbox +Nintendo Switch +Raspberry Pi +Arduino +Tesla Model 3 +Model S +Model Y +Starlink +Bitcoin +Ethereum +Matterbridge +Tailscale +WireGuard +Nginx +Caddy diff --git a/fastapi_backend/app/domain/migrations.py b/fastapi_backend/app/domain/migrations.py index 78e7de9e3..2d6eed65e 100644 --- a/fastapi_backend/app/domain/migrations.py +++ b/fastapi_backend/app/domain/migrations.py @@ -49,7 +49,7 @@ def steps(): """[(name, fn(db))] in the order they run. Append only.""" - from . import access, auth, hierarchy, jobs + from . import access, auth, entities, hierarchy, jobs return [ # IIIF-only access levels (public, transcript, signed-in, private) become the access setting @@ -60,6 +60,8 @@ def steps(): ("tour-seen", auth.migrate_tours), # documents and images queued to be transcribed are read instead, and those that failed there run again ("document-read-step", jobs.migrate_document_steps), + # entities the old rules called terms get the type the rules give them now (people, places, organisations...) + ("entity-types", entities.migrate_types), ] diff --git a/fastapi_backend/app/domain/name_types.py b/fastapi_backend/app/domain/name_types.py new file mode 100644 index 000000000..a31c77fe5 --- /dev/null +++ b/fastapi_backend/app/domain/name_types.py @@ -0,0 +1,300 @@ +"""What kind of named thing a name is, by rules: the type analysis gives each name its extractor finds (analyze.py). + +The rules read the name and the few words around it, and need no model: +- PERSON: a title first (Dr, Ms, Professor, King) or a role before it (CEO, my colleague); a given name first ("Sarah + Johnson", "Maria"); a name of two or three words followed by a verb of saying ("Sarah Connor said"); a named + speaker of the namespace. +- ORG, EVENT, WORK, PLACE: the word a name ends or starts with (Acme Inc, University of Oxford; Web Summit, World Cup; + Clean Air Act; Hudson River, Mount Fuji), or what it's called ("a company called Acme", "a town called Hope"). +- PLACE: countries, regions, states and large cities (lexicon/places.txt); EVENT: holidays; DATE: days and months. +- ORG, PRODUCT: well-known organisations and products (lexicon/organisations.txt, products.txt), and for PRODUCT a + name with a number in it (iPhone 16). +A name none of these place is a TERM: a single unknown word (a brand, a project) needs a model to tell. + +The given names are mostly the US Census Bureau's frequent first names of 1990 (public domain), with others added by +hand; the places, organisations and products are written by hand (lexicon/). Words that are names but mostly something else on their own (Will, +Mark, Rose, Claude) count only as the first of several words. +""" + +from __future__ import annotations + +import functools +import pathlib +import re + +HERE = pathlib.Path(__file__).parent / "lexicon" +TITLES = { + "Dr", + "Mr", + "Mrs", + "Ms", + "Miss", + "Prof", + "Professor", + "Sir", + "Dame", + "Lord", + "Lady", + "Senator", + "President", + "Minister", + "Governor", + "Mayor", + "Judge", + "Justice", + "King", + "Queen", + "Prince", + "Princess", + "Pope", + "Saint", + "St", + "Sheikh", + "Imam", + "Rabbi", + "Father", + "Rev", + "Reverend", + "Captain", + "General", + "Colonel", + "Uncle", + "Aunt", + "Auntie", +} +# a role or relation said just before a name: "our CEO Maria", "my colleague Tom Hughes" +ROLES = set( + """ceo cto cfo coo cmo cpo cio ciso vp founder cofounder co-founder president chairman chairwoman chairperson director +manager engineer developer designer colleague coworker co-worker friend boss wife husband son daughter brother sister +mother father mom mum dad uncle aunt auntie cousin grandma grandpa grandmother grandfather nephew niece teacher doctor +nurse lawyer attorney senator governor mayor judge coach author writer novelist poet artist singer rapper musician +actor actress journalist reporter host anchor analyst investor researcher scientist student intern assistant secretary +neighbour neighbor girlfriend boyfriend fiancé fiance fiancee spouse mentor tutor therapist dentist sergeant officer +detective pastor imam rabbi priest sheikh mr mrs ms dr prof""".split() +) +# verbs of saying after a name: "Sarah Connor said" +SAYS = set( + """said says say told tells asked asks thinks thought wrote writes mentioned mentions explained explains replied +replies added adds noted notes argued argues suggested suggests agreed agrees believes believed claimed claims recalled +recalls insisted insists laughed smiled emailed texted messaged answered answers""".split() +) +# "a company called Acme", "a guy named Jim": the word before called or named says what the name is +CALLED = { + **dict.fromkeys( + "company firm startup business agency bank organisation organization charity nonprofit brand band team club " + "school university college lab".split(), + "ORG", + ), + **dict.fromkeys( + "app tool product service platform model library framework plugin extension device software program website site gadget".split(), + "PRODUCT", + ), + **dict.fromkeys( + "town city village place country island street restaurant cafe café bar hotel neighbourhood neighborhood area region".split(), + "PLACE", + ), + **dict.fromkeys("guy man woman girl boy person lady kid someone somebody gentleman chap dude fellow".split(), "PERSON"), + **dict.fromkeys( + "book film movie song album paper article podcast show series play novel essay poem documentary episode report".split(), + "WORK", + ), + **dict.fromkeys("conference event festival meetup summit workshop hackathon".split(), "EVENT"), +} +ORG_END = re.compile( + r"^(Inc|Ltd|Corp|Corporation|Company|Co|LLC|LLP|LP|PLC|Plc|AG|GmbH|SA|SAS|SpA|BV|NV|Pty|Group|Bank|Bancorp|" + r"Therapeutics|Pharmaceuticals|Pharma|Labs?|Laboratories|University|Institute|Foundation|Agency|Department|" + r"Committee|Council|Association|Society|Commission|Ministry|Office|Board|Partners|Capital|Ventures|Systems|" + r"Technologies|Technology|Tech|Holdings|Hospital|Clinic|School|College|Academy|Airlines|Airways|Motors|Studios?|" + r"Media|Entertainment|Pictures|Records|Press|Publishing|Times|Post|News|Journal|Networks?|Software|Solutions|" + r"Services|Industries|Enterprises|Consulting|Energy|Insurance|Trust|Fund|Party|Union|Club|FC|Church|Museum|Army|" + r"Navy|Corps|Police|Parliament|Senate|Congress|Government|Administration|Authority|Bureau|Federation|Alliance|" + r"Coalition|League|Organi[sz]ation|Initiative|Cooperative|Robotics|Dynamics|Analytics|Electronics|Brands|Foods|" + r"Center|Centre|Court|Electric|Mills|AI)$" +) +ORG_START = { + "University", + "Bank", + "Department", + "Ministry", + "Institute", + "Museum", + "Church", + "College", + "School", + "Office", + "Board", + "Council", + "Committee", + "Federation", + "Association", + "Society", + "Commission", + "Agency", + "Academy", + "Embassy", + "Bureau", + "Court", +} +EVENT_END = re.compile( + r"^(Conference|Summit|Festival|Fest|Olympics|Olympiad|Games|Cup|Championships?|Expo|Fair|Week|Awards|Forum|" + r"Symposium|Convention|War|Revolution|Elections?|Marathon|Hackathon|Meetup|Workshop|Webinar|Tournament|Series|" + r"Prix|Bowl|Gala|Rally|Crisis|Pandemic|Massacre|Day|Eve|Uprising|Riots?|Keynote|Ceremony|Parade|Jubilee|Carnival|" + r"Exhibition|Concert|Tour)$" +) +EVENT_START = {"Battle", "Siege", "Operation"} +WORK_END = re.compile( + r"^(Act|Treaty|Accords?|Constitution|Manifesto|Declaration|Charter|Directive|Regulation|Bible|Testament|Gospel|" + r"Report|Symphony|Agreement|Protocol|Doctrine|Deal|Law)$" +) +WORK_START = {"Treaty"} +PLACE_END = re.compile( + r"^(Street|Road|Avenue|Boulevard|Lane|Square|Bridge|River|Lake|Mountains?|Island|Islands|Isles|Valley|County|City|" + r"Province|Prefecture|Bay|Ocean|Sea|Beach|Airport|Station|Hills|Heights|Desert|Canyon|Peninsula|Coast|Harbou?r|" + r"Falls|Park|Gardens?|Forest|Region|District|Village|Borough|Township|Plaza|Highway|Strait|Glacier|Volcano|Reef|" + r"Gulf|Delta|Basin|Plateau|Plains|Creek|Tower|Palace|Castle|Cathedral|Mosque|Temple|Stadium|Arena|Hall|Mall|" + r"Pier|Wharf|Quay)$" +) +PLACE_START = {"Mount", "Mt", "Lake", "Port", "Fort", "Cape", "Isle", "Gulf"} +HOLIDAYS = { + h.strip() + for h in ( + "christmas, easter, thanksgiving, halloween, ramadan, eid, eid al-fitr, eid ul-fitr, eid al-adha, eid ul-adha, " + "diwali, hanukkah, chanukah, passover, hajj, holi, navratri, lent, advent, pentecost, kwanzaa, oktoberfest, " + "coachella, olympics, brexit, hogmanay, nowruz, yom kippur, rosh hashanah, black friday, cyber monday, wwdc, ces, " + "sxsw" + ).split(",") +} +DAYS = {h.strip() for line in HOLIDAYS for h in re.split(r"\s{2,}|(?<=[a-z]) (?=[a-z])", line)} - {""} +DAYS = {"monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday"} +MONTHS = { + "january", + "february", + "march", + "april", + "june", + "july", + "august", + "september", + "october", + "november", + "december", +} +# names that, on their own, are mostly something else: a person only as the first of several words ("Bill Gates") +AMBIGUOUS = set( + """will mark bill grant hope faith joy art guy pat rich major king queen prince princess royal noble young summer +autumn dawn crystal amber ruby pearl ivy lily rose iris jade holly cherry ginger precious destiny charity sunny misty +windy rusty dusty penny sandy candy bud buddy buck chance chase hunter christian page porter walker ward reed lane clay +cole drew dean miles gay fern flora olive violet hazel heather sterling forest forrest brook brooks april may june +august ebony ivory jewel merry rocky sol wiley trinity unique santa santos cruz reyes mac del van von easter mercedes +chanel alexa claude tiffany lincoln harley mack sierra raven angel august china india asia kenya paris georgia +jordan""".split() +) +WORD = re.compile(r"[^\W\d_][\w'’-]*") + + +def _lines(name): + return [s.strip() for s in (HERE / name).read_text(encoding="utf-8").splitlines() if s.strip() and not s.startswith("#")] + + +def key(name): + """A name as the lists hold it: lower case, single spaces, no possessive, no leading "the".""" + k = re.sub(r"\s+", " ", re.sub(r"[’']s$", "", str(name).strip()).replace("’", "'").casefold()).strip() + return k[4:] if k.startswith("the ") else k + + +@functools.cache +def given_names(): + return frozenset(n.casefold() for n in _lines("given_names.txt")) + + +@functools.cache +def places(): + return frozenset(key(n) for n in _lines("places.txt")) + + +@functools.cache +def known(): + """{key: ORG or PRODUCT} for well-known organisations and products (lexicon/).""" + return {**{key(n): "PRODUCT" for n in _lines("products.txt")}, **{key(n): "ORG" for n in _lines("organisations.txt")}} + + +def _base(t): + return re.sub(r"[’']s$", "", t) + + +def _initial(t): + return re.fullmatch(r"[A-Z]\.?", t) is not None + + +def kind(words, before="", after="", people=frozenset()): + """The type of a name made of `words` (its tokens, as said), between `before` and `after` (the text around it); + None when nothing tells. `people`: keys of names known to be people (a namespace's named speakers, and their first + names).""" + words = [_base(w) for w in words if w] + if not words: + return None + n, first, last = len(words), words[0].rstrip("."), words[-1] + k = key(" ".join(words)) + prev = [w.casefold() for w in WORD.findall(before[-60:])][-3:] + nxt = next(iter(WORD.findall(after[:40])), "").casefold() + named = first.casefold() in given_names() + given = named and (n > 1 or first.casefold() not in AMBIGUOUS) + if n > 1 and k in places(): + return "PLACE" + if len(prev) >= 2 and prev[-1] in ("called", "named") and prev[-2] in CALLED: + return CALLED[prev[-2]] + if k in known(): + return known()[k] + if k in people: + return "PERSON" + if n > 1: # what a name of several words ends or starts with outranks a title in it (General Motors, King's College) + if any(ORG_END.match(w) for w in words[1:]): + return "ORG" + if EVENT_END.match(last) or first in EVENT_START: + return "EVENT" + if WORK_END.match(last) or first in WORK_START: + return "WORK" + if PLACE_END.match(last) or first in PLACE_START: + return "PLACE" + if first in TITLES: + return "PERSON" + if prev and prev[-1] in ROLES: + return "PERSON" + if any(ch.isdigit() for w in words for ch in w) and not re.fullmatch(r"[A-Z0-9&]{2,8}", "".join(words)): + return "PRODUCT" + if named and nxt in SAYS: + return "PERSON" + if k in places(): + return "PLACE" + if n > 1 and first in ORG_START: + return "ORG" + if k in HOLIDAYS: + return "EVENT" + if n == 1 and (k in MONTHS or (k.endswith("s") and k[:-1] in DAYS) or k in DAYS): + return "DATE" + if given or (n > 1 and _initial(words[0]) and not _initial(last)): + return "PERSON" + if 1 < n <= 3 and nxt in SAYS: + return "PERSON" + return None + + +def guess(name, lines=(), people=frozenset()): + """The type rules give `name` from what it is and the lines it's said in: the type most of them agree on, else + None.""" + words = WORD.findall(name) or [name] + votes = {} + found = False + for line in lines: + text = line or "" + m = re.search(rf"(? = { "entity.rename": "Renamed", "entity.describe": "Described", "entity.retype": "Changed the type of", + "entity.typed": "Gave a type to", "entity.hide": "Hid", "entity.show": "Showed", "entity.define": "Put on the list", From 9196139b6c5a2a388302f8131f93a16f2c795714 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 11:51:13 +0000 Subject: [PATCH 2/3] test(clicks): wait for the imports before merging in the graph The graph shows Raspberry Pi only once both imports are analysed (one recording's mentions are below graph.min_edge_weight). Converting the uploaded file took 42-46 s on some runners, so graph-merge opened the graph before the second import was done and timed out. It now waits, uncounted, until the top bar's activity says nothing is running or queued. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT --- ci/clicks/journeys.mjs | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/ci/clicks/journeys.mjs b/ci/clicks/journeys.mjs index e42d83845..35076d1c9 100644 --- a/ci/clicks/journeys.mjs +++ b/ci/clicks/journeys.mjs @@ -49,6 +49,19 @@ async function madeInBackground({ page, base }, where, locator, from, what) { await page.waitForTimeout(2000); } +/** Waits, not counted, until nothing runs in the background (the imports above), as a person coming back later would + * find it: the top bar's activity button says what is running or queued. */ +async function settled({ page }) { + const activity = page.getByRole("banner").getByRole("button", { name: /^Activity: / }); + // idle twice in a row, a poll of the jobs apart: a job just sent may not be listed yet + for (let i = 0, idle = 0; idle < 2; i++) { + const now = (await activity.getAttribute("aria-label").catch(() => null)) || ""; + idle = now && !/running|queued/.test(now) ? idle + 1 : 0; + if (i === 90) throw new Error(`the imports were still running after 3 minutes (${now || "no activity button"})`); + await page.waitForTimeout(2000); + } +} + /** Opens the namespace switcher: in the top bar, or at the top of the navigation drawer on a phone. */ async function openSwitcher(ctx) { const { page, meter, phone } = ctx; @@ -670,6 +683,9 @@ export const JOURNEYS = [ from: "/", async run(ctx) { const { page, meter } = ctx; + // Raspberry Pi is in the graph once both imports are analysed: a name one recording mentions once is below the + // graph's minimum edge weight (2). Converting the uploaded file can take most of a minute on a slow runner. + await settled(ctx); await go(ctx, "Graph"); const find = page.getByRole("combobox", { name: "Find in graph" }); await page.getByRole("region", { name: "Graph" }).waitFor(); From 10ff5641d6e66d1e47bf971d049a9f261ab075e9 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 17 Oct 2026 04:05:45 +0000 Subject: [PATCH 3/3] docs(readme): list typed names as built, with their checks Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT --- README.md | 4 ++-- docs/requirements.md | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 83d7342d1..6ad49d127 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p server (off by default, per namespace), entities and topics in a knowledge graph you can explore, query with Cypher or SPARQL, and see as it was at any version. Every entity has its own page at a wiki address (`/wiki//`), and entities, topics and people are edited and merged from their lists, the graph or an - asset, with Undo instead of a confirm. Find what is most like a passage, an asset, a topic or an entity, in a Similar + asset, with Undo instead of a confirm. Names get their real types (people, places, organisations, events, works) by + rules, without a model. Find what is most like a passage, an asset, a topic or an entity, in a Similar tab on every asset or by selecting text anywhere, and, with anytopdf, photos by what they show. ([Graph](docs/graph.md), [entity pages](docs/notes.md#entity-pages), [similar content](docs/processing.md#similar-content), [graph history](docs/graph-history.md), @@ -206,7 +207,6 @@ These are open pull requests, not yet part of Lens: - **Sources:** RSS and Atom news feeds ([#453](https://github.com/adeelahmad/lense/pull/453)). - **Pictures:** described on this machine with Florence-2 ([#477](https://github.com/adeelahmad/lense/pull/477)), and low-quality images enhanced before they're read ([#479](https://github.com/adeelahmad/lense/pull/479)). -- **Entities:** names given their real types instead of Term ([#395](https://github.com/adeelahmad/lense/pull/395)). - **Web clipper:** signing in, or using your API token, not only an import webhook's ([#406](https://github.com/adeelahmad/lense/pull/406)). - **Automations:** more operators for conditions ([#434](https://github.com/adeelahmad/lense/pull/434)). diff --git a/docs/requirements.md b/docs/requirements.md index d3670a383..ddb27c37e 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -137,6 +137,7 @@ request, to another check that proves the same thing; loosening a requirement or | The graph as it was at any version | [test_graph_history.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_graph_history.py) `test_the_explorer_and_cypher_as_of_a_version`, `test_the_canvas_as_of_a_version`; [test_graph_history.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_graph_history.py) `test_every_curation_is_an_event_and_every_version_replays` | | Every entity's own page at `/wiki//` | [test_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_entities.py) `test_wiki_addresses`; [wiki-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/wiki-page.test.tsx); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `entity-find` | | Entities, topics and people edited and merged from their lists, the graph or an asset, with Undo instead of a confirm | [test_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_entities.py) `test_suggestions_and_merge_undo`; [entities-curate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/entities-curate.test.tsx); [topics-curate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/topics-curate.test.tsx); [recording-entities-edit.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/recording-entities-edit.test.tsx); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `entity-merge`, `graph-merge`, `speaker-rename` | +| Names get their real types (people, places, organisations, events, works) by rules, without a model; older Terms are typed on upgrade | [test_name_types.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_name_types.py) `test_names_get_the_type_their_words_give`, `test_analysis_types_people_places_and_events`, `test_the_upgrade_types_the_old_terms_and_keeps_what_people_chose` | | What is most like a passage, an asset, a topic or an entity | [test_similar.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_similar.py) `test_like_a_text_by_its_key_words`, `test_like_a_resource_by_words_topics_and_entities`, `test_like_a_topic_and_an_entity`; [similar.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/similar.test.tsx) | | With anytopdf, photos found by what they show | [test_photos.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_photos.py) `test_photos_found_by_what_they_show`, `test_off_by_default` |