diff --git a/CHANGELOG.md b/CHANGELOG.md index 139a3063..721ca87b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,7 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`. ## [Unreleased] ### Added +- Mission goals and the Aladdin handoff (ROADMAP §4.12, M9.11): `aloop mission goal derive` turns the Evaluator's gaps (in its own priority order) into a goal (autonomous: the top N; spoon-fed: the gaps the human picks with `--pick … --by`, which is the goal's approval) that names its targets and is bound to the mission's content, the eval, the principles block for the mission's mode and the Aladdin autonomy it maps to; `goal check` holds every intent until the goal is the content `derive` recorded, the mission is unchanged and approved and the mission's and the goal's ethics pass; `goal intents` writes one Intent Contract per gap (`intent new --source mission --goal --gap`, filled without a model, and refused for a goal that fails its check); `goal next` normalizes them one sprint at a time; `goal close` marks an active goal met once an evaluation of it says so; goals come only from the latest recorded evaluation, whose state `mission assess` measured, and `events emit` can no longer forge the state, eval or intent records; a gap a hard line fixed is held out of every later goal; `events trace` walks a commit back through intent → goal → eval → ethics → mission → its approver. Test: `mission-goal.test.py`. - Mission layer hardening (ROADMAP §4.12, a second adversarial review of M9.0/M9.2/M9.3/M9.10, 31 unique findings, each now a test): gates decide from ledger records, not files. Frozen journeys, replacements and retirements, states (`state.assessed`) and the active goal (`goal.derived`) are read from the ledger, so a hidden or edited file changes nothing. Nothing decides from a broken ledger or an edited current baseline, and a restore cannot be undone by a re-apply. The runner pins every project file it runs and refuses links, inline code and scripts outside the project. A judged metric pins its rubric by sha256. A veto binds the content in any order. An unknown flag exits 64. Text is UTF-8 in any locale. GA4's reserved names are one list, and `ethics.*` and `goal.*` decisions are ledger rows. Tests: `mission-{core,journeys,baseline,evaluate}.test.py`. - Mission ethics (ROADMAP §4.12, M9.4): `aloop mission ethics new --subject mission|` writes a review bound to the subject's sha256 with the topics a fixed screen matched; `record` checks and seals it; `check` passes a clear review, asks (exit 3) on a gray one until `mission ethics consent --by NAME --reason TEXT` is recorded for exactly that content and review (stored inside the goal, never carried over), and blocks a hard line (illegal, deceptive, harmful) for good: no consent, later or edited review, deleted file, edited goal, the same words under a new goal id or a goal derived again from the same gap unlocks it, and the goal is retired (`blocked`). The screen reads normalized text and the mechanics a planner would write ("hide the unsubscribe link", "hack into"), and `check --json` names the state the loop acts on. `events emit` refuses human approvals and ethics verdicts, so only their own commands write them. Reason: the spec's three ethics tiers. Test: `mission-ethics.test.py`. - Mission principles (ROADMAP §4.12, M9.7): `plugin/mission/principles.md` holds the standing principles in two variants, autonomous and spoon-fed, with the IBM autonomic loop as the engine and four Amazon-style tension pairs; `aloop mission principles show --mode M` renders the block a goal carries, `hash` its sha256, and `check` refuses an edit that drops a principle, a variant or a pair. Reason: the spec's principles in both modes. Test: `mission-principles.test.py`. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 89c2eca7..4aa345ec 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -425,7 +425,7 @@ Each task flips only its own row here and in TRACKING.md. | M9.8 | Loop: `tick` (the state machine above), `due` (zero-token cadence check), `cron` (host routine or crontab line), budget caps from the token ledger | M9.0 | `plugin/mission/{tick,due,cron}.py`, `scripts/test/mission-tick.test.py` | yes | done: `mission-tick.test.py` (20 checks, on baselines proposed, measured and confirmed through `aloop mission`; a fresh checkout decides the same) | | M9.9 | Roles and wiring: the six personas, hook matchers, model routes, host install, eval suites, role counts in docs; `gate-mission-scope` (role path guards) and `gate-mission-artifact` (a role's artifact passes its check at stop) | M9.0 | `plugin/agents/` (6 new files), `plugin/hooks/hooks.json`, `plugin/hosts/`, `plugin/evals/`, `plugin/bin/gate-mission-*`, role counts in README / OVERVIEW / plugin docs | yes (the only track touching shared lists) | open | | M9.10 | Assess and evaluate: the State Assessor's report (numbers only, from pinned scripts) and the Evaluator's gap list against the baseline and rivals, with `goal_met` | M9.2, M9.3, M9.5 | `plugin/mission/{assess,evaluate}.py`, `schemas/{state,eval}.json`, `scripts/test/mission-evaluate.test.py` | after its needs | done: `mission-evaluate.test.py` (8 checks); judged metrics come from `judge.verdict(cwd, rubric, journey_id, rubric_sha256=…)` (M9.5) for the rubric content the journey pinned and stay pending until a pass or fail verdict; every input is the content its record covers (state `state.assessed`, baseline confirmation, frozen journeys, goal `goal.derived`/`goal.approved`). Open: rival values keyed per benchmark in the baseline, so the gap list compares against rivals (review finding) | -| M9.11 | Goals and the Aladdin handoff: top gaps → a goal → intents (`intent` source `mission`), ethics and principles applied, mode → autonomy; `events trace` reaches goal → mission | M9.4, M9.7, M9.10 | `plugin/mission/goal.py`, `schemas/goal.json`, `plugin/bin/intent` (source list), `plugin/bin/events` (trace) | after its needs | blocked | +| M9.11 | Goals and the Aladdin handoff: top gaps → a goal → intents (`intent` source `mission`), ethics and principles applied, mode → autonomy; `events trace` reaches goal → mission | M9.4, M9.7, M9.10 | `plugin/mission/goal.py`, `schemas/goal.json`, `plugin/bin/intent` (source list), `plugin/bin/events` (trace) | after its needs | done: `mission goal options / derive / check / approve / intents / next / close / show`, `intent new --source mission --goal --gap`, `events trace` adds goal → eval → ethics → mission; `mission-goal.test.py` (13 checks, on the real journeys → baseline → assess → evaluate pipeline; every new gate mutation-tested: deleting it fails a check). Calls: a goal takes M9.10's gaps in its own order (regressions, then closest to target) and names them as `targets` [{journey_id, criterion_id}], which `mission evaluate` narrows to while the goal is active; autonomous takes the top 3 by default, spoon-fed the human's `--pick` (the pick is `goal.approved`); a goal comes only from the latest eval of the whole mission (not an older one, whose gaps may be closed, nor one narrowed to an active goal) whose gap list `evaluate check` confirms, whose content its `eval.completed` recorded, and whose state a `state.assessed` recorded (a hand-written state that `mission evaluate` accepted is still refused); `goal check`, `intents`, `next` and `close` hold a goal to the content its `goal.derived` recorded, so a crafted or edited-and-resealed goal file is never handed off or closed; a goal is stale once the mission's content changes; intents are filled from the gap and the journey without a model (zero tokens), one per gap, normalized one at a time by `next`, which first makes that intent the newest file (gate-intake re-checks the newest, so the sprint's plan links the right gap); `intent new --source mission` needs `--gap` and a goal that passes `goal check` and lists that gap, and `intent check` holds a mission intent to its own `intent.received` record (goal and gap) and to the goal's autonomy (a reviewed goal's intent is not checked as autonomous); the handoff counts only intents whose first `intent.received` names the goal; `intents` and `close` change only the status, never the goal's hashed content (its ethics review, consent and approval stay valid); a goal is met by a newer evaluation of that goal that says `goal_met`, only while it is active (a derived, blocked or retired goal never closes; closing a met goal again is a no-op); `events emit` refuses `goal.derived`, `goal.met`, `state.assessed`, `eval.completed` and `intent.received`, so the lineage `trace` reads and the records a goal is derived and closed from are written only by their commands (edits outside its files: `assess` and `evaluate` add the content's `sha256` to their events, one field each). A gap a hard line fixed is held, by its id and by its summary: listed apart by `options`, never taken by `derive`, so the autonomous loop moves on instead of re-deriving it (`ethics.blocked` names the gaps); a gap's summary is the journey's title, criterion and target (stable across evals); a goal keeps the principles stamp it was derived under (stale after an upgrade, not invalid, since re-stamping would void its ethics review and approval) | | M9.12 | The `/aloop:mission` skill: the supervisor's playbook for the tick loop | M9.8, M9.9 | `plugin/skills/mission/` | after its needs | done: `mission-skill.test.py` (8 checks); a live run needs M9.9's roles | | M9.13 | End to end: an offline scripted to-do-app mission (mission → baseline → eval → goal → intent, zero tokens), then a live `make e2e-mission YES=1` | M9.10–M9.12 | `scripts/test/mission-e2e.test.py`, `scripts/smoke/e2e-mission.sh` | after its needs | blocked | | M9.14 | GA4 live on the owner's account (one Google sign-in) | M9.6 | — | — | needs the owner: `aloop mission telemetry auth` prints the steps, then `init`, `consent`, `setup`, `debug-hit` | diff --git a/docs/TRACKING.md b/docs/TRACKING.md index 64aefe5a..76debde5 100644 --- a/docs/TRACKING.md +++ b/docs/TRACKING.md @@ -70,7 +70,7 @@ Status of every requirement in [ROADMAP.md](ROADMAP.md), in the four states §8 | M9.8 | Mission loop: `tick` from facts, zero-token `due`, cron, budget caps | IMPLEMENTED_TESTED | `scripts/test/mission-tick.test.py` | | M9.9 | Six fixed mission roles, wired; role path guards; artifact check at stop | NOT_IMPLEMENTED | §4.12 | | M9.10 | State Assessor report and Evaluator gap list | IMPLEMENTED_TESTED | `scripts/test/mission-evaluate.test.py` | -| M9.11 | Goals become Aladdin intents; lineage reaches the mission | NOT_IMPLEMENTED | §4.12 | +| M9.11 | Goals become Aladdin intents; lineage reaches the mission | IMPLEMENTED_TESTED | `scripts/test/mission-goal.test.py` | | M9.12 | `/aloop:mission` skill | IMPLEMENTED_TESTED | `scripts/test/mission-skill.test.py` | | M9.13 | Mission end to end (offline in CI, one live cycle measured) | NOT_IMPLEMENTED | §4.12 | | M9.17 | Approvals name the captured human prompt | NOT_IMPLEMENTED | §4.12 | diff --git a/plugin/bin/events b/plugin/bin/events index d5a43c45..363e9997 100755 --- a/plugin/bin/events +++ b/plugin/bin/events @@ -10,7 +10,8 @@ Usage: events lineage TASK_ID the task's chain: dispatch → attempts → gates → limits/decisions → worktree outcome (no transcripts) events trace SHA commit → task → gates/decisions → plan → intent → - approver, from the stream alone (exit 1: unknown commit) + approver, from the stream alone (exit 1: unknown commit); + a mission intent adds goal → eval → ethics → mission events commit-msg TASK_ID a commit message built from the task's title (tasks.md), its recorded decisions and the intent, never model prose events graph [--format json|dot] the stream as a graph (events, tasks, commits, intents; @@ -23,7 +24,7 @@ Usage: structured rationale (plan|split|target| implementation|replan|acceptance) Emitting never fails a hook: an unknown event type is the only error (exit 64), besides the -human approvals and ethics verdicts of the mission layer, which only their own commands write. +human approvals, ethics verdicts and goal lineage of the mission layer, which only their own commands write. """ import glob import json @@ -36,8 +37,11 @@ import _events # noqa: E402 import _mission # noqa: E402 import _paths # noqa: E402 -# events only their own command may write (ROADMAP §4.12): human approvals and the ethics gate's verdicts -GATE_OWNED = set(_mission.APPROVAL_EVENTS) | {"ethics.reviewed", "ethics.blocked"} +# events only their own command may write (ROADMAP §4.12): human approvals, the ethics gate's verdicts +# and the goal lineage (`events trace` and `mission goal close` read them) +GATE_OWNED = set(_mission.APPROVAL_EVENTS) | {"ethics.reviewed", "ethics.blocked", "goal.derived", "goal.met", + "state.assessed", "eval.completed", # what goals are derived and closed from + "intent.received"} # an intent's lineage (`intent new`) def parse(argv): @@ -97,6 +101,7 @@ def trace(cwd, sha): iid = (intent.get("data") or {}).get("intent_id", "") recv = next((e for e in evs if e.get("event") == "intent.received" and (e.get("data") or {}).get("intent_id") == iid), {}) print("intent %s %s raw=%s" % (iid, (intent.get("refs") or {}).get("intent", ""), (recv.get("refs") or {}).get("raw", "-"))) + mission_trace(evs, recv) else: print("intent (missing: the plan names no intent.normalized)") appr = [e for e in evs if e.get("event") == "plan.approved" and e.get("caused_by") == plan.get("event_id")] @@ -108,6 +113,37 @@ def trace(cwd, sha): return 0 +def mission_trace(evs, recv): + """intent → goal → eval → ethics → mission → its approver (ROADMAP §4.12), for an intent a + mission goal handed off (`intent new --source mission --goal`).""" + gid = (recv.get("data") or {}).get("goal_id") + if not gid: + return + gd = None + for e in evs: + if e.get("event") == "goal.derived" and (e.get("data") or {}).get("goal_id") == gid: + gd = e + if not gd: + print("goal %s (missing: no goal.derived)" % gid) + return + d = gd.get("data") or {} + ref = d.get("subject") # the goal file the ledger row covers + print("goal %s from eval %s gaps=%s (%s)" % (gid, d.get("eval_id"), ",".join(d.get("gaps") or []), d.get("autonomy"))) + about = [e for e in evs if (e.get("data") or {}).get("subject") == ref and (e.get("data") or {}).get("sha256") == d.get("sha256")] + tiers = [e["data"].get("tier") for e in about if e.get("event") == "ethics.reviewed"] + consent = [e for e in about if e.get("event") == "ethics.consent"] + if not tiers: + print("ethics (missing: no review of this goal's content)") + else: + tier = "hard" if "hard" in tiers else "gray" if "gray" in tiers else "clear" + print("ethics %s%s" % (tier, " (consent by %s)" % consent[-1]["data"].get("by") if tier == "gray" and consent else "")) + ok = [e for e in evs if e.get("event") == "mission.approved" and (e.get("data") or {}).get("sha256") == d.get("mission_sha256")] + if ok: + print("mission %s approved by %s" % (d.get("mission_id"), ok[-1]["data"].get("by"))) + else: + print("mission %s (missing: no mission.approved for the content the goal served)" % d.get("mission_id")) + + def task_title(cwd, task): m = re.match(r"^S(\d+)-(\d+)-T(\d+)$", task or "") if not m: @@ -187,8 +223,8 @@ def main(argv): print("events emit: need an EVENT", file=sys.stderr) return 64 if pos[0] in GATE_OWNED: - print("events emit: %s is a human approval or a gate's verdict; only its own command writes it " - "(aloop mission … approve / consent / record)" % pos[0], file=sys.stderr) + print("events emit: %s is a human approval or a gate's record; only its own command writes it " + "(aloop mission … approve / consent / record / goal / assess / evaluate; intent new)" % pos[0], file=sys.stderr) return 64 try: eid = _events.emit(cwd, pos[0], task=opts.get("task", ""), attempt=opts.get("attempt", ""), diff --git a/plugin/bin/intent b/plugin/bin/intent index 708f141c..089e6517 100755 --- a/plugin/bin/intent +++ b/plugin/bin/intent @@ -4,12 +4,17 @@ Usage: intent new --source prompt|file|issue|image|mixed [--raw "the request"] [--raw-ref PATH] + intent new --source mission --goal GOAL_ID --gap GAP_ID [--raw "the gap"] [--raw-ref PATH] issue a canonical intent_id (never model text) and write the contract skeleton to - docs/agents/intents/.json (prints the path); emits intent.received + docs/agents/intents/.json (prints the path); emits intent.received. A + mission intent (ROADMAP §4.12) names its goal and gap, so lineage reaches the mission; + it exists only while the goal passes `mission goal check` and lists that gap intent check FILE [--mode reviewed|autonomous] validate the contract: required fields, enums, ≥1 success and ≥1 failure scenario, - and the clarification policy. Exit 0 ok (emits intent.normalized) · 2 invalid · - 3 needs the human (an `ask` question is unanswered — in BOTH modes) + and the clarification policy; a mission intent also passes its goal's gate, matches its + intent.received, and is checked in the goal's autonomy. Exit 0 ok (emits + intent.normalized) · 2 invalid · 3 needs the human (an `ask` question is unanswered — + in BOTH modes) intent summary FILE the approval summary lines: defaults picked, assumptions recorded, spikes created @@ -31,7 +36,8 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__))) import _events # noqa: E402 import _paths # noqa: E402 -SOURCES = ("prompt", "file", "issue", "image", "mixed") +SOURCES = ("prompt", "file", "issue", "image", "mixed", "mission") +GOAL_ID = re.compile(r"^gol_[0-9a-f]{20}$") # a mission goal (plugin/mission/goal.py) CLASSES = {"assume": "assumption", "default": "default", "ask": "answer", "spike": "spike_task"} ID = re.compile(r"^int_[0-9a-f]{12}[0-9a-f]{8}$") TEXT = ("actor", "want", "why", "done_looks_like") @@ -47,21 +53,55 @@ def intents_dir(cwd): return os.path.join(_paths.docs(cwd), "intents") -def cmd_new(cwd, source, raw="", raw_ref=""): +def mission_gate(cwd, goal, gap, mode=None): + """A mission intent's goal gate (ROADMAP §4.12): the goal passes `mission goal check` and lists the + gap, and its intents are checked in the goal's autonomy. Exit 0 · 2 · 3, reported.""" + sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "..", "mission")) + import goal as G + rc, blocks, asks, facts = G.check(cwd, goal) + if rc: + G.report(blocks, asks) + return rc + g, _ = G.M.load(G.M.path_of(cwd, "goal", goal)) + if gap not in [x.get("gap_id") for x in g["gaps"]]: + print("BLOCK[intent]: %s is not a gap of goal %s" % (gap, goal), file=sys.stderr) + return 2 + if mode and mode != facts["autonomy"]: + print("BLOCK[intent]: goal %s is %s: its intents are checked in that mode, not %s" % (goal, facts["autonomy"], mode), + file=sys.stderr) + return 2 + return 0 + + +def cmd_new(cwd, source, raw="", raw_ref="", goal="", gap=""): if source not in SOURCES: print("intent new: --source must be one of %s" % ", ".join(SOURCES), file=sys.stderr) return 64 + if (source == "mission") != bool(goal) or (goal and not GOAL_ID.match(goal)) or bool(goal) != bool(gap): + print("intent new: --goal gol_… --gap GAP_ID go with --source mission, and only with it (a mission intent " + "names its goal and the gap it closes)", file=sys.stderr) + return 64 + rc = mission_gate(cwd, goal, gap) if goal else 0 + if rc: + return rc iid = new_id() doc = {"intent_id": iid, "source": source, "actor": "", "want": "", "why": "", "constraints": [], "success_scenarios": [], "failure_scenarios": [], "connections": [], "done_looks_like": "", "open_questions": [], "assumptions": [], "provenance": {"received_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "raw_ref": raw_ref, "raw_sha256": hashlib.sha256(raw.encode()).hexdigest() if raw else ""}} + if goal: + doc["provenance"].update(goal_id=goal, gap_id=gap) os.makedirs(intents_dir(cwd), exist_ok=True) path = os.path.join(intents_dir(cwd), iid + ".json") with open(path, "w") as f: json.dump(doc, f, indent=1) - _events.emit(cwd, "intent.received", role="intake", refs={"intent": path, "raw": raw_ref}, intent_id=iid, source=source) + derived = None + for e in _events.read(cwd) if goal else (): # the goal this intent serves: intent.received is caused by it + if e.get("event") == "goal.derived" and (e.get("data") or {}).get("goal_id") == goal: + derived = e + _events.emit(cwd, "intent.received", role="intake", refs={"intent": path, "raw": raw_ref}, intent_id=iid, source=source, + caused_by=derived and derived.get("event_id"), **({"goal_id": goal, "gap_id": gap} if goal else {})) print(path) return 0 @@ -87,6 +127,8 @@ def problems(doc, path): prov = doc.get("provenance") or {} if not prov.get("received_at"): p.append("provenance.received_at missing") + if doc.get("source") == "mission" and not GOAL_ID.match(str(prov.get("goal_id", ""))): + p.append("a mission intent names its goal (provenance.goal_id gol_…, from `intent new --goal`)") for i, q in enumerate(doc.get("open_questions") or []): if not isinstance(q, dict) or not str(q.get("question", "")).strip(): p.append("open_questions[%d] needs a question" % i) @@ -122,6 +164,18 @@ def cmd_check(cwd, path, mode="reviewed"): print("BLOCK[intent]: %s" % b, file=sys.stderr) if bad: return 2 + if doc.get("source") == "mission": + prov = doc["provenance"] + rec = [e for e in _events.read(cwd) if e.get("event") == "intent.received" + and (e.get("data") or {}).get("intent_id") == doc["intent_id"]] + d = (rec[0].get("data") or {}) if rec else {} + if d.get("source") != "mission" or (d.get("goal_id"), d.get("gap_id")) != (prov["goal_id"], prov.get("gap_id")): + print("BLOCK[intent]: provenance (goal %s, gap %s) is not what its intent.received recorded (goal %s, gap %s)" + % (prov["goal_id"], prov.get("gap_id"), d.get("goal_id"), d.get("gap_id")), file=sys.stderr) + return 2 + rc = mission_gate(cwd, prov["goal_id"], prov.get("gap_id"), mode) + if rc: + return rc asks = unanswered(doc) if asks: for q in asks: @@ -169,7 +223,8 @@ def main(argv): pos.append(a) cwd = os.getcwd() if argv[0] == "new": - return cmd_new(cwd, opts.get("source", ""), opts.get("raw", ""), opts.get("raw_ref", "")) + return cmd_new(cwd, opts.get("source", ""), opts.get("raw", ""), opts.get("raw_ref", ""), opts.get("goal", ""), + opts.get("gap", "")) if argv[0] == "check" and pos: return cmd_check(cwd, pos[0], opts.get("mode", "reviewed")) if argv[0] == "summary" and pos: diff --git a/plugin/mission/README.md b/plugin/mission/README.md index 234c3669..82c439ef 100644 --- a/plugin/mission/README.md +++ b/plugin/mission/README.md @@ -115,7 +115,8 @@ A gate checks structure and evidence, never whether a judgment is right. `docs/agents/mission/ledger.jsonl` (only the CLI appends to it). - **M9.11 goals.** Add `mission` to the `intent` SOURCES. `gate-intake` checks the newest `int_*.json` by mtime and `ids freeze` links a plan to the last `intent.normalized` anywhere, so - queued mission intents need an explicit link. `record_authority` also approves the latest plan: + queued mission intents need an explicit link (built: `goal next` makes the intent it hands off the + newest file before normalizing it). `record_authority` also approves the latest plan: record a goal's authority only after its sprint's `ids freeze`. `aladdin ""` refuses a bare intent containing `/`; use `aladdin run`. - **M9.17 approvals.** Any session can run `mission approve --by anyone` or `events authority`, or diff --git a/plugin/mission/ethics.py b/plugin/mission/ethics.py index 74f54bcc..aaaa5ef8 100644 --- a/plugin/mission/ethics.py +++ b/plugin/mission/ethics.py @@ -179,6 +179,12 @@ def hard_words(cwd): for w in ((e.get("data") or {}).get("words") or []) if isinstance(w, str)} +def held_gaps(cwd): + """Every gap id a hard line has fixed: the gap is held by its id as well as by its words.""" + return {g for e in _events.read(cwd) if e.get("event") == "ethics.blocked" + for g in ((e.get("data") or {}).get("gaps") or []) if isinstance(g, str)} + + def screen(text): """[(category, matched phrase)] in CATEGORIES order, one per matched category.""" out = [] @@ -275,6 +281,14 @@ def hard_keys(kind, doc, rev): return word_keys(kind, doc, only=named or None) +def hard_gaps(kind, doc, rev): + """The gap ids a hard review fixes: the gaps its hard findings name (a goal of one gap: that gap).""" + if kind != "goal": + return [] + named = {g for f in rev.get("findings", []) if f.get("tier") == "hard" for g in (f.get("gaps") or [])} + return sorted(g for g in goal_gaps(doc) if not named or g in named) + + def review_problems(cwd, rev, path, matched, subject=None): return M.problems(rev, "ethics", path) + (rules(rev, matched, subject) if isinstance(rev, dict) else []) @@ -455,7 +469,8 @@ def cmd_record(cwd, path): cats = [c for c in list(CATEGORIES) + ["other"] if any(f["category"] == c and f["tier"] == "hard" for f in rev["findings"])] M.emit(cwd, "ethics.blocked", role="ethics-reviewer", refs=refs, ethics_id=eid, subject=cur["ref"], - sha256=cur["sha256"], text_sha256=words, categories=cats, kind=kind, words=hard_keys(kind, doc, rev)) + sha256=cur["sha256"], text_sha256=words, categories=cats, kind=kind, words=hard_keys(kind, doc, rev), + gaps=hard_gaps(kind, doc, rev)) print("[ethics] %s recorded: tier %s for %s" % (eid, rev["tier"], cur["ref"]), file=sys.stderr) rc, bad, facts = verdict(cwd, kind, sid) if facts.get("state") == "hard" and kind == "goal" and doc.get("status") != "blocked": diff --git a/plugin/mission/goal.py b/plugin/mission/goal.py new file mode 100644 index 00000000..90fa27f5 --- /dev/null +++ b/plugin/mission/goal.py @@ -0,0 +1,622 @@ +"""mission goal — goals from the gap list, and the Aladdin handoff (ROADMAP §4.12, M9.11). + +The Evaluator's top gaps become the next goal: in an autonomous mission the top N, in a +spoon-fed one the gaps the human picks (the pick is the goal's approval). A goal is bound to +the mission content it serves, the eval it came from, the standing principles for the +mission's mode, and the Aladdin autonomy that mode maps to (autonomous → autonomous, +spoon-fed → reviewed). It passes the ethics gate before any intent exists; then each gap +becomes an Intent Contract (source `mission`), and `events trace` walks a commit back +through intent → goal → mission → the human who approved it. + +Usage: + mission goal options [--eval EVAL_ID] [--json] + the eval's gaps in the Evaluator's order (regressions, then the open gaps closest to + target, then unscored): what a goal can take; spoon-fed offers these. + A gap a hard line fixed (its summary, ethics) is held: listed apart, never taken + mission goal derive [--eval EVAL_ID] [--top N] (autonomous; N default 3) + mission goal derive [--eval EVAL_ID] --pick GAP_ID[,…] --by NAME (spoon-fed) + write goals/.json from the latest recorded eval of the whole mission (no active goal) + whose goal is not met, once the mission is approved and its ethics check passes; the goal + names its targets [{journey_id, criterion_id}], which `mission evaluate` narrows to while + it is active; records goal.derived in the ledger (and goal.approved on a pick) + mission goal check GOAL_ID [--json] + the gate before intents: seal, the content goal.derived recorded, the mission unchanged and + approved, the eval unchanged, the principles stamp well formed (an older block is stale, kept as provenance), mission and goal ethics, and in spoon-fed mode a human + goal.approved for this content. Exit 0 · 2 block · 3 needs the human + mission goal approve GOAL_ID --by NAME [--reason TEXT] + a human approves the goal's current content (goal.approved) + mission goal intents GOAL_ID + the handoff: one Intent Contract per gap (intent new --source mission --goal --gap), filled + from the gap, the mission and the principles; only after `check` passes; idempotent + mission goal next GOAL_ID + the next intent without a plan, made the newest intent file (gate-intake re-checks the newest) + and normalized in the goal's autonomy (intent check --mode), so the sprint's plan.created + links it; exit 1 when every intent has a plan + mission goal close GOAL_ID --eval EVAL_ID + an active goal (sealed, as derived) and a recorded evaluation of it, newer than the goal, + that says goal_met: status met, a goal.met ledger record (a goal already met: exit 0, no new record). Exit 2 + while a target (or a regression) is still open, or for a goal that is not active + mission goal show GOAL_ID [--json] + the goal, its check, and the principles block it carries +""" +import glob +import json +import os +import re +import subprocess +import sys +import time + +HERE = os.path.dirname(os.path.realpath(__file__)) +BIN = os.path.realpath(os.path.join(HERE, "..", "bin")) +sys.path.insert(0, BIN) +sys.path.insert(0, HERE) +import _events # noqa: E402 +import _mission as M # noqa: E402 +import _paths # noqa: E402 +import ethics as E # noqa: E402 +import evaluate as EV # noqa: E402 +import journeys as J # noqa: E402 +import principles as P # noqa: E402 + +AUTONOMY = {"autonomous": "autonomous", "spoon-fed": "reviewed"} # mission mode → Aladdin approval mode (§4.1) +DEFAULT_TOP = 3 # Ship Small: a few gaps per goal, one intent (one sprint) each +GOAL_ID = re.compile(r"^gol_[0-9a-f]{20}$") +EVAL_ID = re.compile(r"^evl_[0-9a-f]{20}$") + + +# ── evals (M9.10's gap list: plugin/mission/evaluate.py) ──────────────────────────── +def first(cwd, event, key, value): + """The first event of a kind whose data[key] is value (gate-owned events: the one its command wrote).""" + for e in _events.read(cwd): + if e.get("event") == event and (e.get("data") or {}).get(key) == value: + return e + return None + + +def eval_record(cwd, eid): + """(doc, eval.completed event, problems) of one gap list: M9.10's own check (it follows from its + state, which a state.assessed ledger record covers, its baseline and goal) and the eval.completed + that recorded this content (written only by `mission evaluate`).""" + if not isinstance(eid, str) or not EVAL_ID.match(eid): + return None, None, ["%r is not an eval id (evl_…)" % (eid,)] + path = M.path_of(cwd, "eval", eid) + doc, err = M.load(path) + if not isinstance(doc, dict): + return None, None, ["no evaluation %s (%s)" % (eid, M.rel(cwd, path))] + try: + bad = ["eval %s: %s" % (eid, b) for b in EV.problems(cwd, path, doc)] + sha = M.digest(doc) + except (KeyError, TypeError, ValueError, AttributeError) as e: + return doc, None, ["eval %s is malformed (%s)" % (eid, e)] + if doc.get("eval_id") != eid: + bad.append("eval %s: eval_id does not match the file name" % eid) + ev = first(cwd, "eval.completed", "eval_id", eid) + if not ev: + bad.append("eval %s is not recorded (no eval.completed event names it)" % eid) + elif ev["data"].get("sha256") != sha: + bad.append("eval %s is not the content its eval.completed recorded" % eid) + return doc, ev, bad + + +def latest_eval(cwd): + """The id of the latest recorded eval (stream order), or None.""" + ids = [(e.get("data") or {}).get("eval_id") for e in _events.read(cwd) if e.get("event") == "eval.completed"] + for eid in reversed(ids): + if eid and os.path.exists(M.path_of(cwd, "eval", eid)): + return eid + return None + + +def gap_of(cwd, g): + """What a goal keeps of an eval gap, with a one-line summary in the journey's own words (stable + across evals, so a hard line on it holds; the measurement `now` is kept apart).""" + jdoc, _ = J.load(cwd, str(g.get("journey_id"))) + title = jdoc.get("title") if isinstance(jdoc, dict) and isinstance(jdoc.get("title"), str) else g["journey_id"] + return {"gap_id": str(g["gap_id"]), "journey_id": str(g["journey_id"]), "criterion_id": str(g["criterion_id"]), + "kind": str(g.get("kind")), "target": str(g.get("target") or ""), "now": str(g.get("now") or ""), + "summary": "%s (%s): %s" % (title.strip(), g["criterion_id"], g.get("target") or "passes")} + + +def eval_gaps(cwd, edoc): + """The eval's gaps as a goal keeps them, in the Evaluator's priority order (regressions, then the + open gaps closest to target, then unscored): a goal takes them as given.""" + return [gap_of(cwd, g) for g in edoc.get("gaps") or [] if isinstance(g, dict) and g.get("gap_id")] + + +def split_held(cwd, gaps): + """(open, held): a gap whose summary a hard line fixed is held for good (ethics.hard_words), so a + goal derived again never takes it and the loop moves on to the next gap.""" + hard, ids = E.hard_words(cwd), E.held_gaps(cwd) + held = [g for g in gaps if E.word_key(g["summary"]) in hard or g["gap_id"] in ids] + return [g for g in gaps if g not in held], held + + +def stream_index(cwd, pred): + """The position of the last event matching pred in the stream, or -1.""" + hit = -1 + for i, e in enumerate(_events.read(cwd)): + if pred(e): + hit = i + return hit + + +# ── the goal's gate ────────────────────────────────────────────────────────────────── +def mission_of(cwd): + path = M.path_of(cwd, "mission") + doc, _ = M.load(path) + return (path, doc) if isinstance(doc, dict) else (path, None) + + +def derived(cwd, gid): + """The goal.derived ledger record `derive` wrote for this goal, or None.""" + ref = M.rel(cwd, M.path_of(cwd, "goal", gid)) + return next((r for r in M.records(cwd, "goal.derived", ref) if r["data"].get("goal_id") == gid), None) + + +def derived_problems(cwd, g): + """A goal is the content `derive` recorded in its goal.derived ledger row, so every derive rule + (the latest eval's open, unheld gaps; the real principles stamp) holds at the gate as well.""" + gid = g["goal_id"] + d = derived(cwd, gid) + if not d: + return ["no goal.derived records %s: only `aloop mission goal derive` makes a goal" % gid] + data = d.get("data") or {} + if data.get("sha256") != M.digest(g): + return ["%s is not the content goal.derived recorded (sha256 %s): derive a new goal" + % (gid, str(data.get("sha256"))[:12])] + want = {"eval_id": g["eval"]["eval_id"], "mission_sha256": g["mission"]["sha256"], + "principles_sha256": g["principles"]["sha256"], "gaps": [x["gap_id"] for x in g["gaps"]]} + return ["%s: %s does not match its goal.derived record" % (gid, k) for k, v in want.items() if data.get(k) != v] + + +def check(cwd, gid): + """(exit, blocks, asks, facts) for one goal.""" + gpath = M.path_of(cwd, "goal", gid) + g, err = M.load(gpath) + if not isinstance(g, dict): + return 2, ["no goal %s (%s)" % (gid, M.rel(cwd, gpath))], [], {} + try: + blocks, asks = list(M.problems(g, "goal", gpath)), [] + sha = M.digest(g) + except ValueError as e: # text that is not valid Unicode (a lone surrogate) has no digest + return 2, ["goal %s cannot be hashed (%s)" % (gid, e)], [], {} + facts = {"goal_id": gid, "path": M.rel(cwd, gpath), "status": g.get("status"), "autonomy": g.get("autonomy"), + "sha256": sha, "ethics": None} + blocks += derived_problems(cwd, g) if not blocks else [] + if blocks: + return 2, blocks, asks, facts + mpath, m = mission_of(cwd) + msha = M.digest(m) if m else "" + if not m or m.get("mission_id") != g["mission"]["mission_id"] or msha != g["mission"]["sha256"]: + blocks.append("the mission changed since %s was derived (or is gone): derive a new goal from the " + "approved mission" % gid) + elif not M.approved(cwd, "mission.approved", M.rel(cwd, mpath), msha): + asks.append("the mission's content has no human approval (aloop mission approve --by NAME)") + if m and g["mode"] != m.get("mode"): + blocks.append("goal mode %s is not the mission's mode %s" % (g["mode"], m.get("mode"))) + if g["autonomy"] != AUTONOMY[g["mode"]]: + blocks.append("autonomy must be %s for a %s mission" % (AUTONOMY[g["mode"]], g["mode"])) + if g["principles"]["mode"] != g["mode"]: + blocks.append("the principles stamp is for %s, the goal is %s" % (g["principles"]["mode"], g["mode"])) + blocks += P.verify(g["principles"], stale_ok=True) # an older block is provenance, not a defect + facts["principles_stale"] = P.stale(g["principles"]) + edoc, _, ebad = eval_record(cwd, g["eval"]["eval_id"]) + if ebad or edoc.get("sha256") != g["eval"]["sha256"]: + blocks.append("the eval %s this goal came from changed or is gone" % g["eval"]["eval_id"]) + for kind, sid in (("mission", ""), ("goal", gid)): + rc, bad, ef = E.verdict(cwd, kind, sid) + if kind == "goal": + facts["ethics"] = {"tier": ef.get("tier"), "state": ef.get("state"), "review": ef.get("review"), + "consent_event": ef.get("consent_event")} + (blocks if rc == 2 else asks).extend("ethics: %s" % b for b in bad) + if g["autonomy"] == "reviewed" and not M.approved(cwd, "goal.approved", facts["path"], facts["sha256"]): + asks.append("spoon-fed: the human approves this goal's content (aloop mission goal approve %s --by NAME)" % gid) + return (2 if blocks else 3 if asks else 0), blocks, asks, facts + + +def report(blocks, asks): + for b in blocks: + print("BLOCK[goal]: %s" % b, file=sys.stderr) + for a in asks: + print("ASK[goal]: %s" % a, file=sys.stderr) + + +# ── commands ───────────────────────────────────────────────────────────────────────── +def pick_eval(cwd, o): + eid = o.get("eval") or latest_eval(cwd) + if not eid: + return None, None, ["no recorded evaluation yet: the Evaluator runs first (mission tick: ASSESS → EVALUATE)"] + doc, ev, bad = eval_record(cwd, eid) + return doc, ev, bad + + +def cmd_options(cwd, o): + doc, _, bad = pick_eval(cwd, o) + if bad: + report(bad, []) + return 2 + gaps, held = split_held(cwd, eval_gaps(cwd, doc)) + opts = [dict(g, rank=i) for i, g in enumerate(gaps, 1)] + if o.get("json"): + print(json.dumps({"eval_id": doc["eval_id"], "goal_met": doc["goal_met"], "goal_id": doc.get("goal_id"), + "options": opts, "held": held}, indent=1)) + else: + for x in opts: + print("%d. %s (%s): %s; now %s" % (x["rank"], x["gap_id"], x["kind"], x["summary"], x["now"] or "-")) + for x in held: + print("- %s held by a hard line (never a goal): %s" % (x["gap_id"], x["summary"])) + return 0 + + +def cmd_derive(cwd, o): + if M.ledger_block(cwd, "goal"): + return 2 + mpath, m = mission_of(cwd) + if not m or m.get("mode") not in AUTONOMY: + report(["no mission yet (aloop mission init)"], []) + return 2 + mode = m["mode"] + if mode == "autonomous" and o.get("pick"): + print("mission goal derive: --pick is for spoon-fed missions (the human picks); autonomous takes --top N", + file=sys.stderr) + return 64 + if mode == "spoon-fed" and (not o.get("pick") or o.get("top")): + print("mission goal derive: in a spoon-fed mission the human picks the gaps: aloop mission goal options, " + "then derive --pick GAP_ID[,…] --by NAME", file=sys.stderr) + return 64 + if mode == "spoon-fed" and not (o.get("by") or "").strip(): + print("mission goal derive: --by NAME (who picked the gaps; the pick is the goal's approval)", file=sys.stderr) + return 64 + top = DEFAULT_TOP + if o.get("top"): + try: + top = int(o["top"]) + except ValueError: + top = 0 + if top < 1: + print("mission goal derive: --top must be a whole number >= 1", file=sys.stderr) + return 64 + edoc, eev, bad = pick_eval(cwd, o) + if bad: + report(bad, []) + return 2 + latest = latest_eval(cwd) + if edoc["eval_id"] != latest: + report(["eval %s is not the latest recorded evaluation (%s): its gaps may be closed since; derive from the " + "latest" % (edoc["eval_id"], latest)], []) + return 2 + msha = M.digest(m) + if not M.approved(cwd, "mission.approved", M.rel(cwd, mpath), msha): + report([], ["the mission's content has no human approval (aloop mission approve --by NAME)"]) + return 3 + rc, ebad, _ = E.verdict(cwd, "mission", "") + if rc: + E.report(rc, ebad) + return rc + if edoc.get("goal_id"): + report(["eval %s is narrowed to goal %s's targets (it was active): a new goal comes from an eval of the " + "whole mission (close goal %s, then mission evaluate)" % (edoc["eval_id"], edoc["goal_id"], edoc["goal_id"])], + []) + return 2 + if edoc["goal_met"]: + report(["the evaluation %s says the goal is met: go back to research and raise the bar (new journeys " + "and a new eval), not a goal from this one" % edoc["eval_id"]], []) + return 2 + gaps, held = split_held(cwd, eval_gaps(cwd, edoc)) + if mode == "spoon-fed": + want = [x.strip() for x in o["pick"].split(",") if x.strip()] + by_id = {g["gap_id"]: g for g in gaps} + stuck = [x for x in want if x in {g["gap_id"] for g in held}] + if stuck: + report(["%s held by a hard line: no goal takes it (aloop mission ethics check)" % ", ".join(stuck)], []) + return 2 + missing = [x for x in want if x not in by_id] + if missing or not want: + report(["not an open gap of eval %s: %s (aloop mission goal options)" % (edoc["eval_id"], ", ".join(missing))], + []) + return 2 + chosen = [by_id[x] for x in dict.fromkeys(want)] + else: + chosen = gaps[:top] + if not chosen: + report(["the evaluation %s lists no open gaps%s" % (edoc["eval_id"], " (%d held by a hard line)" % len(held) + if held else "")], []) + return 2 + gid = M.new_id("goal") + doc = {"goal_id": gid, "mission": {"mission_id": m["mission_id"], "sha256": msha}, + "eval": {"eval_id": edoc["eval_id"], "sha256": edoc["sha256"]}, + "statement": "Close %d gap%s toward the mission: %s" % (len(chosen), "" if len(chosen) == 1 else "s", + "; ".join(g["summary"] for g in chosen)), + "targets": [{"journey_id": g["journey_id"], "criterion_id": g["criterion_id"]} for g in chosen], + "gaps": chosen, "mode": mode, "autonomy": AUTONOMY[mode], "principles": P.stamp(mode), + "status": "derived", "created_at": M.now()} + M.seal(doc) + path = M.path_of(cwd, "goal", gid) + M.save(path, doc) + M.record(cwd, "goal.derived", M.rel(cwd, path), "mission-planner", goal_id=gid, sha256=doc["sha256"], + mission_id=m["mission_id"], mission_sha256=msha, eval_id=edoc["eval_id"], eval_event=(eev or {}).get("event_id"), + gaps=[g["gap_id"] for g in chosen], mode=mode, autonomy=doc["autonomy"], + principles_sha256=doc["principles"]["sha256"]) + if mode == "spoon-fed": + rc = approve(cwd, gid, o["by"], "picked from eval %s" % edoc["eval_id"]) + if rc: + return rc + print(path) + return 0 + + +def approve(cwd, gid, by, reason): + path = M.path_of(cwd, "goal", gid) + doc, err = M.load(path) + if not isinstance(doc, dict): + report(["no goal %s" % gid], []) + return 2 + sha = M.digest(doc) + try: + eid = M.approve(cwd, "goal.approved", M.rel(cwd, path), sha, by, reason, goal_id=gid) + except ValueError as e: + print("mission goal approve: %s" % e, file=sys.stderr) + return 64 + appr = doc.get("approval") if isinstance(doc.get("approval"), dict) else {} + appr["goal"] = {"event_id": eid, "by": by.strip(), "at": M.now()} + doc["approval"] = appr + if doc.get("status") == "derived": + doc["status"] = "approved" + M.save(path, doc) + print("[goal] %s approved by %s (sha256 %s)" % (gid, by.strip(), sha[:12]), file=sys.stderr) + return 0 + + +def intent_tool(): + """plugin/bin/intent as a module (its problems() checks a contract without emitting).""" + import importlib.machinery + import importlib.util + loader = importlib.machinery.SourceFileLoader("aloop_intent_tool", os.path.join(BIN, "intent")) + spec = importlib.util.spec_from_loader(loader.name, loader) + mod = importlib.util.module_from_spec(spec) + loader.exec_module(mod) + return mod + + +def goal_intents(cwd, gid): + """{gap_id: (path, doc)} of the intents already handed off for this goal: the ones whose own + intent.received (written by `intent new --source mission`) names the goal and the gap.""" + out, seen = {}, set() + for e in _events.read(cwd): + d = e.get("data") or {} + if e.get("event") != "intent.received" or d.get("intent_id") in seen: + continue + seen.add(d.get("intent_id")) # an intent's first record is the one `intent new` wrote + if d.get("goal_id") != gid: + continue + p = os.path.join(_paths.docs(cwd), "intents", "%s.json" % d.get("intent_id")) + doc, _ = M.load(p) + if isinstance(doc, dict) and d.get("gap_id") and doc.get("intent_id") == d.get("intent_id"): + out.setdefault(d["gap_id"], (p, doc)) + return out + + +def fill(it, g, gap, m, gref, jref): + st = g["principles"] + it.update( + actor="the app's end users (as the mission's research defines them)", + want=gap["summary"], + why="Closes gap %s (%s; target %s; now %s) of goal %s toward the mission: %s" + % (gap["gap_id"], gap["kind"], gap["target"], gap["now"] or "-", g["goal_id"], m["statement"]), + constraints=list(m.get("constraints") or []) + [ + "Standing principles (%s mode, sha256 %s): `aloop mission principles show --mode %s`" + % (st["mode"], st["sha256"][:12], st["mode"]), + "No principle relaxes an invariant, a gate or a budget cap (ROADMAP §7)"], + success_scenarios=["journey %s meets criterion %s (%s) in its pinned run, so the goal's next evaluation " + "no longer lists gap %s" % (gap["journey_id"], gap["criterion_id"], gap["target"], gap["gap_id"])], + failure_scenarios=["journey %s still misses criterion %s in its pinned run: gap %s stays open and the goal " + "is not met" % (gap["journey_id"], gap["criterion_id"], gap["gap_id"])], + connections=[gref, jref], + done_looks_like="The goal's next recorded evaluation no longer lists gap %s" % gap["gap_id"], + assumptions=["Derived without a model from the Evaluator's gap %s; the Mission Planner may sharpen the " + "want and scenarios, never the goal or gap it serves" % gap["gap_id"]]) + it["provenance"]["gap_id"] = gap["gap_id"] + it["provenance"]["mission_id"] = g["mission"]["mission_id"] + return it + + +def cmd_intents(cwd, gid): + rc, blocks, asks, facts = check(cwd, gid) + if rc: + report(blocks, asks) + return rc + gpath = M.path_of(cwd, "goal", gid) + g, _ = M.load(gpath) + _, m = mission_of(cwd) + have = goal_intents(cwd, gid) + tool = intent_tool() + paths = [] + for gap in g["gaps"]: + if gap["gap_id"] in have: + paths.append(have[gap["gap_id"]][0]) + continue + r = subprocess.run([os.path.join(BIN, "intent"), "new", "--source", "mission", "--goal", gid, "--gap", + gap["gap_id"], "--raw", gap["summary"], "--raw-ref", facts["path"]], cwd=cwd, + capture_output=True, text=True) + if r.returncode: + sys.stderr.write(r.stderr) + return 2 + p = r.stdout.strip() + it, _ = M.load(p) + fill(it, g, gap, m, facts["path"], M.rel(cwd, M.path_of(cwd, "journey", gap["journey_id"]))) + bad = tool.problems(it, p) + if bad: + report(["intent %s: %s" % (it.get("intent_id"), b) for b in bad], []) + return 2 + with open(p, "w") as f: + json.dump(it, f, indent=1) + paths.append(p) + if g.get("status") in ("derived", "approved"): + g["status"] = "active" + M.save(gpath, g) + print("\n".join(paths)) + print("[goal] %s handed off: %d intent(s), %s mode (aloop mission goal next %s)" + % (gid, len(paths), g["autonomy"], gid), file=sys.stderr) + return 0 + + +def cmd_next(cwd, gid): + rc, blocks, asks, facts = check(cwd, gid) + if rc: + report(blocks, asks) + return rc + if facts["status"] not in ("active",): + report(["%s is %s: hand it off first (aloop mission goal intents %s)" % (gid, facts["status"], gid)], []) + return 2 + g, _ = M.load(M.path_of(cwd, "goal", gid)) + have = goal_intents(cwd, gid) + planned = {(e.get("data") or {}).get("intent_id") for e in _events.read(cwd) + if e.get("event") in ("plan.created", "plan.revised")} + for gap in g["gaps"]: + if gap["gap_id"] not in have: + continue + p, it = have[gap["gap_id"]] + if it.get("intent_id") in planned: + continue + others = [os.path.getmtime(x) for x in glob.glob(os.path.join(os.path.dirname(p), "int_*.json")) if x != p] + t = max(others + [time.time()]) + 0.01 + os.utime(p, (t, t)) # gate-intake re-checks the newest intent file: the one handed off + r = subprocess.run([os.path.join(BIN, "intent"), "check", p, "--mode", facts["autonomy"]], cwd=cwd, + capture_output=True, text=True) + sys.stderr.write(r.stderr) + if r.returncode: + return r.returncode + print(p) + return 0 + print("[goal] every intent of %s has a plan; when its sprints close, evaluate and `mission goal close`" % gid, + file=sys.stderr) + return 1 + + +def cmd_close(cwd, gid, eid): + gpath = M.path_of(cwd, "goal", gid) + g, _ = M.load(gpath) + if not isinstance(g, dict): + report(["no goal %s" % gid], []) + return 2 + try: + bad = list(M.problems(g, "goal", gpath)) + except ValueError as e: + bad = ["goal %s cannot be hashed (%s)" % (gid, e)] + bad = bad or derived_problems(cwd, g) + if bad: + report(bad, []) + return 2 + if M.ledger_block(cwd, "goal"): + return 2 + if M.records(cwd, "goal.met", M.rel(cwd, gpath)): # the record decides, never the status field + print("[goal] %s is already met" % gid, file=sys.stderr) + return 0 + if g["status"] != "active": + report(["goal %s is %s: only an active goal closes (handed off with `mission goal intents`)" + % (gid, g["status"])], []) + return 2 + doc, ev, bad = eval_record(cwd, eid) + if bad: + report(bad, []) + return 2 + if doc.get("goal_id") != gid: + report(["eval %s is not an evaluation of goal %s (it evaluates %s): evaluate while the goal is active" + % (eid, gid, doc.get("goal_id") or "the whole mission")], []) + return 2 + born = stream_index(cwd, lambda e: e.get("event_id") == derived(cwd, gid)["event_id"]) + seen = stream_index(cwd, lambda e: e.get("event_id") == ev.get("event_id")) + if born < 0 or seen <= born: + report(["eval %s was recorded before goal %s was derived: it says nothing about the goal's work" % (eid, gid)], []) + return 2 + if not doc["goal_met"]: + still = [x.get("gap_id") for x in doc["gaps"] if isinstance(x, dict)] + report(["goal %s is not met in eval %s; still open: %s" % (gid, eid, ", ".join(map(str, still)))], []) + return 2 + g["status"] = "met" + M.save(gpath, g) + M.record(cwd, "goal.met", M.rel(cwd, gpath), "evaluator", goal_id=gid, eval_id=eid, eval_event=ev.get("event_id"), + sha256=M.digest(g)) + print("[goal] %s met (eval %s)" % (gid, eid), file=sys.stderr) + return 0 + + +def cmd_show(cwd, gid, as_json): + g, _ = M.load(M.path_of(cwd, "goal", gid)) + if not isinstance(g, dict): + report(["no goal %s" % gid], []) + return 2 + rc, blocks, asks, facts = check(cwd, gid) + block = "" + try: + block = P.render(g.get("mode")) + except (ValueError, TypeError): + pass + if as_json: + print(json.dumps({"goal": g, "check": dict(facts, exit=rc, blocks=blocks, asks=asks), "principles": block}, + indent=1)) + return 0 + print("Goal %s (%s, %s): %s" % (gid, g.get("status"), g.get("autonomy"), g.get("statement"))) + for gap in g.get("gaps") if isinstance(g.get("gaps"), list) else []: + if isinstance(gap, dict): + print("- %s (%s, now %s): %s" % (gap.get("gap_id"), gap.get("kind"), gap.get("now") or "-", + gap.get("summary"))) + print("Check: %s" % ("ok" if rc == 0 else "; ".join(blocks + asks))) + sys.stdout.write(block) + return 0 + + +def parse_args(argv): + opts, pos, k = {}, [], None + for a in argv: + if k: + opts[k], k = a, None + elif a == "--json": + opts["json"] = True + elif a.startswith("--"): + k = a[2:].replace("-", "_") + else: + pos.append(a) + if k: + opts[k] = "" + return opts, pos + + +def main(argv, cwd): + cmd, (o, pos) = argv[0], parse_args(argv[1:]) + if cmd == "options": + return cmd_options(cwd, o) + if cmd == "derive": + return cmd_derive(cwd, o) + if cmd not in ("check", "approve", "intents", "next", "close", "show"): + print("mission goal: unknown command %r (options, derive, check, approve, intents, next, close, show)" % cmd, + file=sys.stderr) + return 64 + if not pos or not GOAL_ID.match(pos[0]): + print("mission goal %s GOAL_ID (gol_…)" % cmd, file=sys.stderr) + return 64 + gid = pos[0] + if cmd == "check": + rc, blocks, asks, facts = check(cwd, gid) + if o.get("json"): + print(json.dumps(dict(facts, exit=rc, blocks=blocks, asks=asks), indent=1)) + report(blocks, asks) + if rc == 0 and not o.get("json"): + print("[goal] %s ok (%s)" % (gid, facts["autonomy"]), file=sys.stderr) + return rc + if cmd == "approve": + if not (o.get("by") or "").strip(): + print("mission goal approve: --by NAME is required", file=sys.stderr) + return 64 + return approve(cwd, gid, o["by"], o.get("reason", "")) + if cmd == "intents": + return cmd_intents(cwd, gid) + if cmd == "next": + return cmd_next(cwd, gid) + if cmd == "close": + if not o.get("eval"): + print("mission goal close GOAL_ID --eval EVAL_ID", file=sys.stderr) + return 64 + return cmd_close(cwd, gid, o["eval"]) + return cmd_show(cwd, gid, o.get("json")) diff --git a/plugin/mission/schemas/goal.json b/plugin/mission/schemas/goal.json new file mode 100644 index 00000000..069d53f1 --- /dev/null +++ b/plugin/mission/schemas/goal.json @@ -0,0 +1,75 @@ +{ + "title": "goal — the Mission Planner's next goal, derived from the Evaluator's gap list (ROADMAP §4.12, M9.11): docs/agents/mission/goals/.json", + "type": "object", + "required": ["goal_id", "mission", "eval", "statement", "targets", "gaps", "mode", "autonomy", "principles", "status", "created_at"], + "additionalProperties": false, + "properties": { + "goal_id": {"type": "string", "pattern": "^gol_"}, + "mission": { + "type": "object", + "required": ["mission_id", "sha256"], + "additionalProperties": false, + "properties": { + "mission_id": {"type": "string", "pattern": "^msn_[0-9a-f]{20}$"}, + "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"} + } + }, + "eval": { + "type": "object", + "required": ["eval_id", "sha256"], + "additionalProperties": false, + "properties": { + "eval_id": {"type": "string", "pattern": "^evl_[0-9a-f]{20}$"}, + "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"} + } + }, + "statement": {"type": "string", "minLength": 1}, + "targets": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "required": ["journey_id", "criterion_id"], + "additionalProperties": false, + "properties": { + "journey_id": {"type": "string", "pattern": "^jrn_[0-9a-f]{20}$"}, + "criterion_id": {"type": "string", "minLength": 1} + } + } + }, + "gaps": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "required": ["gap_id", "journey_id", "criterion_id", "kind", "target", "now", "summary"], + "additionalProperties": false, + "properties": { + "gap_id": {"type": "string", "minLength": 1}, + "journey_id": {"type": "string", "pattern": "^jrn_[0-9a-f]{20}$"}, + "criterion_id": {"type": "string", "minLength": 1}, + "kind": {"enum": ["regression", "open", "unscored"]}, + "target": {"type": "string"}, + "now": {"type": "string"}, + "summary": {"type": "string", "minLength": 1} + } + } + }, + "mode": {"enum": ["autonomous", "spoon-fed"]}, + "autonomy": {"enum": ["autonomous", "reviewed"]}, + "principles": { + "type": "object", + "required": ["mode", "sha256", "ids"], + "additionalProperties": false, + "properties": { + "mode": {"enum": ["autonomous", "spoon-fed"]}, + "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"}, + "ids": {"type": "array", "minItems": 1, "items": {"type": "string", "minLength": 1}} + } + }, + "status": {"enum": ["derived", "approved", "active", "met", "retired", "blocked"]}, + "created_at": {"type": "string", "minLength": 1}, + "approval": {"type": "object"}, + "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"} + } +} diff --git a/scripts/test/mission-goal.test.py b/scripts/test/mission-goal.test.py new file mode 100644 index 00000000..9e96986d --- /dev/null +++ b/scripts/test/mission-goal.test.py @@ -0,0 +1,427 @@ +#!/usr/bin/env python3 +"""ROADMAP §4.12 (M9) M9.11: goals and the Aladdin handoff. The Evaluator's gaps (M9.10, in its +own priority order) become a goal (autonomous: the top N; spoon-fed: the gaps the human picks); +the goal names its targets, the mission it serves, the eval it came from, the standing +principles for the mission's mode and the Aladdin autonomy that mode maps to; it passes the +ethics gate before any intent exists; then each gap becomes an Intent Contract (source +`mission`), `events trace` walks a commit back through intent → goal → eval → ethics → +mission → the human who approved it, and an evaluation of the active goal closes it. + python3 scripts/test/mission-goal.test.py +""" +import glob +import json +import os +import shutil +import subprocess +import sys + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "fixtures")) +from mission_fixture import Project, M, BIN # noqa: E402 + +sys.path.insert(0, M.AREAS_DIR) +import _events # noqa: E402 +import ethics as E # noqa: E402 +import goal as G # noqa: E402 +import principles as P # noqa: E402 + +INTENT, EVENTS = os.path.join(BIN, "intent"), os.path.join(BIN, "events") +PASS = [0] +GENV = {"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t", "GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t"} + + +def ok(name): + PASS[0] += 1 + print("ok -", name) + + +def made(r): + assert r.returncode == 0, r.stderr + return json.load(open(r.stdout.strip().splitlines()[-1])) + + +def tool(p, *args): + return subprocess.run(list(args), cwd=p.dir, env=p.env, capture_output=True, text=True) + + +def review(p, subject, findings=(), tier="clear"): + path = p.run("ethics", "new", "--subject", subject).stdout.strip() + doc = json.load(open(path)) + doc.update(findings=list(findings), tier=tier) + json.dump(doc, open(path, "w")) + return p.run("ethics", "record", path) + + +def evaluate(p): + """assess the app as it is now, then evaluate it: the gap list (narrowed to the active goal, if any).""" + return made(p.run("evaluate", made(p.run("assess"))["state_id"])) + + +def setup(mode, extra=()): + """A project with research, the fixture's journey (criteria `all` and `fast`), a passing one, any + extra drafts, a confirmed baseline and a first evaluation of the whole mission.""" + p = Project(mode=mode, veto_window_h=0) + subprocess.run(["git", "add", "-A"], cwd=p.dir, check=True) + subprocess.run(["git", "commit", "-qm", "seed"], cwd=p.dir, env=dict(p.env, **GENV), check=True) + rid = p.research(["c1"]) + js = [p.journey(p.draft(rid)), + p.journey(p.draft(rid, title="The first task stays under 10s", + pass_criteria=[{"id": "ok10", "metric": "secs", "op": "<=", "value": 10}]))] + js += [p.journey(p.draft(rid, **x)) for x in extra] + b = made(p.run("baseline", "new", "--definition", "a first task in under 5 seconds, and it shows up")) + assert p.run("baseline", "measure", b["baseline_id"]).returncode == 0 + assert p.run("baseline", "confirm", b["baseline_id"], "--by", "Adeel").returncode == 0 + return p, js, evaluate(p) + + +# 1) derive refuses until its facts hold: a recorded eval, the mission's ethics, a goal not yet met +p, (j1, j2), e1 = setup("autonomous") +ev1 = e1["eval_id"] +assert [g["gap_id"] for g in e1["gaps"]] == [j1 + "#fast", j1 + "#all"], e1["gaps"] # M9.10's own order +r = p.run("goal", "derive") +assert r.returncode == 2 and "BLOCK[ethics]" in r.stderr and "no ethics review" in r.stderr, r.stderr +assert review(p, "mission").returncode == 0 +copy = dict(e1, eval_id=M.new_id("eval")) # the same gap list under another id, never recorded +copy.pop("sha256") +M.seal(copy) +p.save("eval", copy["eval_id"], copy) +r = p.run("goal", "derive", "--eval", copy["eval_id"]) +assert r.returncode == 2 and "not recorded" in r.stderr, r.stderr +forged = dict(e1, gaps=e1["gaps"][1:]) # a gap list that does not follow from its state +forged.pop("sha256") +M.seal(forged) +p.save("eval", ev1, forged) +r = p.run("goal", "derive", "--eval", ev1) +assert r.returncode == 2 and "does not follow from state" in r.stderr, r.stderr +p.save("eval", ev1, dict(e1, created_at="2020-01-01T00:00:00Z", sha256=M.digest(dict(e1, created_at="2020-01-01T00:00:00Z")))) +r = p.run("goal", "derive", "--eval", ev1) # edited and resealed after it was recorded +assert r.returncode == 2 and "not the content its eval.completed recorded" in r.stderr, r.stderr +p.save("eval", ev1, e1) +st = p.load("state", e1["state_id"]) # a state `mission assess` never measured: all passing +fake = dict(st, state_id=M.new_id("state")) +for run in fake["runs"]: + for rec in run["journeys"].values(): + rec["passed"] = True + for c in rec.get("criteria") or []: + c["passed"] = True +fake.pop("sha256") +M.seal(fake) +p.save("state", fake["state_id"], fake) +fe = p.run("evaluate", fake["state_id"]) +assert fe.returncode in (0, 2), fe.stderr +if fe.returncode == 0: # M9.10 evaluates it; the goal module still refuses it + r = p.run("goal", "derive", "--eval", json.load(open(fe.stdout.strip().splitlines()[-1]))["eval_id"]) + assert r.returncode == 2 and "not measured by mission assess" in r.stderr, r.stderr + e1 = evaluate(p) # the latest eval is a real one again + ev1 = e1["eval_id"] +assert not M.artifacts(p.dir, "goal") +ok("derive refuses with no mission ethics review, an unrecorded eval, a gap list that does not follow from its state, " + "or a state the assessor never measured") + +# 2) options: the eval's gaps as M9.10 ordered them, each with a one-line summary in the journey's words +r = p.run("goal", "options", "--eval", ev1, "--json") +assert r.returncode == 0, r.stderr +opts = json.loads(r.stdout) +assert [(g["gap_id"], g["rank"]) for g in opts["options"]] == [(j1 + "#fast", 1), (j1 + "#all", 2)], opts +assert opts["options"][0]["summary"] == "Add a task and see it (fast): secs <= 5" and opts["held"] == [] +r = p.run("goal", "options") +assert r.returncode == 0 and r.stdout.splitlines()[0].startswith("1. %s#fast (open)" % j1), r.stdout +ok("options: the eval's gaps in the Evaluator's priority order, summarized in the journey's words") + +# 3) derive (autonomous): the top N gaps as given, bound to the mission, the eval and the principles block +assert p.run("goal", "derive", "--eval", ev1, "--pick", j1 + "#all").returncode == 64 # --pick is spoon-fed's +assert p.run("goal", "derive", "--eval", ev1, "--top", "0").returncode == 64 +r = p.run("goal", "derive", "--eval", ev1) +assert r.returncode == 0, r.stderr +gpath = r.stdout.strip() +g = json.load(open(gpath)) +gid = g["goal_id"] +assert gid.startswith("gol_") and M.ID.match(gid) and os.path.basename(gpath) == gid + ".json" +assert [x["gap_id"] for x in g["gaps"]] == [j1 + "#fast", j1 + "#all"], g["gaps"] +assert g["targets"] == [{"journey_id": j1, "criterion_id": "fast"}, {"journey_id": j1, "criterion_id": "all"}] +mdoc = p.load("mission") +assert g["mission"] == {"mission_id": mdoc["mission_id"], "sha256": M.digest(mdoc)}, g["mission"] +assert g["eval"] == {"eval_id": ev1, "sha256": e1["sha256"]} +assert g["mode"] == "autonomous" and g["autonomy"] == "autonomous" and g["status"] == "derived" +assert g["principles"] == P.stamp("autonomous") and g["sha256"] == M.digest(g) +assert "Add a task and see it (fast): secs <= 5" in g["statement"] and M.problems(g, "goal", gpath) == [] +d = p.events("goal.derived") +ec = [e for e in p.events("eval.completed") if e["data"]["eval_id"] == ev1][0] +assert len(d) == 1 and d[0]["data"]["goal_id"] == gid and d[0]["data"]["eval_event"] == ec["event_id"], d +rows = M.records(p.dir, "goal.derived", M.rel(p.dir, gpath)) # a ledger record: it survives a fresh checkout +assert [r["event_id"] for r in rows] == [d[0]["event_id"]] and rows[0]["data"]["sha256"] == g["sha256"], rows +assert d[0]["data"]["mission_id"] == mdoc["mission_id"] and d[0]["data"]["sha256"] == g["sha256"] +assert d[0]["data"]["gaps"] == [j1 + "#fast", j1 + "#all"] +ok("derive: the top N gaps as given; targets, mission, eval, principles stamp and autonomy bound; goal.derived") + +# 4) check: the goal's gate before any intent (ethics; autonomous needs no goal approval) +r = p.run("goal", "check", gid) +assert r.returncode == 2 and "no ethics review" in r.stderr, r.stderr +assert review(p, gid, [{"category": "other", "tier": "gray", "reason": "the list shows more of the user's tasks"}], + "gray").returncode == 3 +r = p.run("goal", "check", gid) +assert r.returncode == 3 and "ASK[goal]: ethics: gray area" in r.stderr, r.stderr +r = p.run("goal", "intents", gid) +assert r.returncode == 3 and not os.path.isdir(os.path.join(p.dir, "docs", "agents", "intents")), r.stderr +assert p.run("ethics", "consent", "--subject", gid, "--by", "Adeel", "--reason", "tasks are the user's own").returncode == 0 +r = p.run("goal", "check", gid, "--json") +assert r.returncode == 0, r.stderr +f = json.loads(r.stdout) +assert f["exit"] == 0 and f["ethics"]["tier"] == "gray" and f["ethics"]["state"] == "ok" and f["autonomy"] == "autonomous" +assert f["principles_stale"] is False +crafted = dict(g, goal_id=M.new_id("goal")) # the same content under an id derive never issued +crafted.pop("sha256") +M.seal(crafted) +p.save("goal", crafted["goal_id"], crafted) +assert review(p, crafted["goal_id"]).returncode == 0 +r = p.run("goal", "check", crafted["goal_id"]) +assert r.returncode == 2 and "no goal.derived" in r.stderr, r.stderr +assert p.run("goal", "intents", crafted["goal_id"]).returncode == 2 +r = p.run("goal", "close", crafted["goal_id"], "--eval", ev1) +assert r.returncode == 2 and "no goal.derived" in r.stderr, r.stderr +os.remove(p.path("goal", crafted["goal_id"])) +edited = dict(g, gaps=g["gaps"][:1], targets=g["targets"][:1]) # derived, then edited and resealed +edited.pop("sha256") +M.seal(edited) +p.save("goal", gid, edited) +assert review(p, gid).returncode == 0 +r = p.run("goal", "check", gid) +assert r.returncode == 2 and "not the content goal.derived recorded" in r.stderr, r.stderr +p.save("goal", gid, dict(p.load("goal", gid), **{k: g[k] for k in ("gaps", "targets", "sha256")})) +assert p.run("goal", "check", gid).returncode == 0 +ok("check: no intent until ethics passes (gray waits for consent); autonomous needs no goal approval; the goal is " + "the one derive recorded (a crafted or edited-and-resealed goal blocks)") + +# 5) intents: one Intent Contract per gap, source mission, filled from the gap, the mission and the principles +r = p.run("goal", "intents", gid) +assert r.returncode == 0, r.stderr +paths = r.stdout.split() +assert len(paths) == 2, paths +intents = [json.load(open(x)) for x in paths] +for it, gap in zip(intents, g["gaps"]): + assert it["source"] == "mission" and it["provenance"]["goal_id"] == gid and it["provenance"]["gap_id"] == gap["gap_id"] + assert it["want"] == gap["summary"] and gid in it["why"] and "world's best to-do app" in it["why"] + assert any(P.digest("autonomous")[:12] in c and "autonomous" in c for c in it["constraints"]), it["constraints"] + assert gap["gap_id"] in it["done_looks_like"] and gap["criterion_id"] in it["success_scenarios"][0] + assert M.rel(p.dir, p.path("journey", j1)) in it["connections"] + assert tool(p, INTENT, "check", paths[intents.index(it)], "--mode", "autonomous").returncode == 0 +rec = p.events("intent.received") +assert [e["data"]["goal_id"] for e in rec] == [gid, gid] and all(e["caused_by"] == d[0]["event_id"] for e in rec) +assert p.load("goal", gid)["status"] == "active" and M.digest(p.load("goal", gid)) == g["sha256"] # content unchanged +r = p.run("goal", "intents", gid) +assert r.returncode == 0 and r.stdout.split() == paths and len(p.events("intent.received")) == 2 # idempotent +assert [e["data"]["gap_id"] for e in rec] == [x["gap_id"] for x in g["gaps"]] +posed = tool(p, INTENT, "new", "--source", "prompt", "--raw", "x").stdout.strip() # a prompt intent posing as one +pd = json.load(open(posed)) +pd.update(source="mission", provenance=dict(pd["provenance"], goal_id=gid, gap_id=g["gaps"][0]["gap_id"])) +json.dump(pd, open(posed, "w")) +assert p.run("goal", "intents", gid).stdout.split() == paths # only intents its own intent.received names +os.remove(posed) +ok("intents: one contract per gap (source mission, goal + gap provenance, principles); goal content unchanged; idempotent") + +# 6) next: the next intent without a plan is normalized (so the sprint's plan.created links it) +r = p.run("goal", "next", gid) +assert r.returncode == 0 and r.stdout.strip() == paths[0], r.stderr +newest = max(glob.glob(os.path.join(p.dir, "docs", "agents", "intents", "int_*.json")), key=os.path.getmtime) +assert os.path.basename(newest) == os.path.basename(paths[0]), newest # gate-intake re-checks the newest: the one handed off +norm = p.events("intent.normalized")[-1] +assert norm["data"]["intent_id"] == intents[0]["intent_id"] and norm["data"]["mode"] == "autonomous" +_events.emit(p.dir, "plan.created", role="planner", caused_by=norm["event_id"], sprint="S1", intent_id=intents[0]["intent_id"]) +assert p.run("goal", "next", gid).stdout.strip() == paths[1] +norm2 = p.events("intent.normalized")[-1] +_events.emit(p.dir, "plan.created", role="planner", caused_by=norm2["event_id"], sprint="S2", intent_id=intents[1]["intent_id"]) +r = p.run("goal", "next", gid) +assert r.returncode == 1 and "every intent of" in r.stderr, r.stderr +ok("next: normalizes the next unplanned intent in the goal's autonomy mode; exit 1 when all are planned") + +# 7) events trace: commit → plan → intent → goal → eval → ethics → mission → the human who approved it +_events.emit(p.dir, "commit.created", task="S1-01-T1", sha="abc123def4567890", subject="feat: faster first task") +r = tool(p, EVENTS, "trace", "abc123def456") +assert r.returncode == 0, r.stderr +for want in ("intent %s" % intents[0]["intent_id"], "goal %s" % gid, "eval %s" % ev1, + "ethics gray (consent by Adeel)", "mission %s approved by Adeel" % mdoc["mission_id"]): + assert want in r.stdout, (want, r.stdout) +for ev in ("goal.derived", "goal.met", "eval.completed", "state.assessed", "intent.received"): # the lineage the trace and close read is written only by goal's commands + r = tool(p, EVENTS, "emit", ev, "--data", "goal_id=%s" % gid) + assert r.returncode == 64 and "only its own command" in r.stderr, (ev, r.stderr) +ok("events trace reaches goal → eval → ethics → mission → approver; `events emit` cannot forge the lineage") + +# 8) close: an evaluation of the active goal (M9.10 narrows to its targets) that says goal_met +p.app(seconds_to_first_task=4.0, working_steps=["open", "add"], tasks_shown=0) # fast now passes; `all` still fails +e2 = evaluate(p) +assert e2["goal_id"] == gid and [x["gap_id"] for x in e2["gaps"]] == [j1 + "#all"], e2 +r = p.run("goal", "close", gid, "--eval", e2["eval_id"]) +assert r.returncode == 2 and j1 + "#all" in r.stderr and "still open" in r.stderr, r.stderr +r = p.run("goal", "close", gid, "--eval", ev1) +assert r.returncode == 2 and "not an evaluation of goal" in r.stderr, r.stderr # the whole-mission eval it came from +p.app(seconds_to_first_task=2.0, working_steps=["open", "add", "see"], tasks_shown=1) +e3 = evaluate(p) +assert e3["goal_id"] == gid and e3["goal_met"] is True +r = p.run("goal", "close", gid, "--eval", e3["eval_id"]) +assert r.returncode == 0, r.stderr +assert p.load("goal", gid)["status"] == "met" and p.events("goal.met")[-1]["data"]["goal_id"] == gid +assert [r["data"]["eval_id"] for r in M.records(p.dir, "goal.met", M.rel(p.dir, gpath))] == [e3["eval_id"]], "goal.met is a ledger record" +assert M.digest(p.load("goal", gid)) == g["sha256"] # closing changes only the status +r = p.run("goal", "close", gid, "--eval", e3["eval_id"]) +assert r.returncode == 0 and len(p.events("goal.met")) == 1, r.stderr # already met: no second goal.met +e4 = evaluate(p) # no active goal: the whole mission again +assert e4["goal_id"] is None and e4["goal_met"] is True +r = p.run("goal", "derive", "--eval", e4["eval_id"]) +assert r.returncode == 2 and "raise the bar" in r.stderr, r.stderr +r = p.run("goal", "derive", "--eval", ev1) # an older eval: its gaps are closed now +assert r.returncode == 2 and "not the latest" in r.stderr, r.stderr +ok("close: met only by a newer evaluation of this goal that says goal_met (once); then the whole mission is met: " + "raise the bar, not a goal from an older eval") + +# 9) the fixed point: a goal is stale once the mission's content changes; a tampered goal fails its seal; +# an eval narrowed to an active goal is not a source of new goals +p.app(seconds_to_first_task=4.0, working_steps=["open", "add"], tasks_shown=0) +gid2 = made(p.run("goal", "derive", "--eval", evaluate(p)["eval_id"], "--top", "1"))["goal_id"] +r = p.run("goal", "close", gid2, "--eval", e3["eval_id"]) +assert r.returncode == 2 and "only an active goal closes" in r.stderr, r.stderr +assert review(p, gid2).returncode == 0 and p.run("goal", "check", gid2).returncode == 0 +assert p.run("goal", "intents", gid2).returncode == 0 +e5 = evaluate(p) +assert e5["goal_id"] == gid2 +r = p.run("goal", "derive", "--eval", e5["eval_id"]) +assert r.returncode == 2 and "narrowed to goal %s" % gid2 in r.stderr, r.stderr +g2 = p.load("goal", gid2) +gp2 = p.path("goal", gid2) +json.dump(dict(g2, principles=dict(g2["principles"], sha256="0" * 64)), open(gp2, "w")) +r = p.run("goal", "check", gid2) +assert r.returncode == 2 and "sha256 does not match" in r.stderr, r.stderr +r = p.run("goal", "close", gid2, "--eval", e5["eval_id"]) +assert r.returncode == 2 and "sha256 does not match" in r.stderr, r.stderr # close checks the seal too +with open(gp2, "w") as fh: + fh.write(json.dumps(dict(g2, statement="x \ud800 y"))) # not valid Unicode: a block, not a crash +r = p.run("goal", "check", gid2) +assert r.returncode == 2 and "Traceback" not in r.stderr, r.stderr +json.dump(g2, open(gp2, "w")) +assert review(p, "mission", [{"category": "other", "tier": "gray", "reason": "a list of the user's tasks"}], + "gray").returncode == 3 +r = p.run("goal", "check", gid2) +assert r.returncode == 3 and "ASK[goal]: ethics:" in r.stderr, r.stderr # the mission's verdict holds its goals +m = p.load("mission") +m["constraints"].append("works offline") +m["status"] = "draft" +p.save("mission", "", m) +assert p.run("approve", "--by", "Adeel").returncode == 0 +r = p.run("goal", "check", gid2) +assert r.returncode == 2 and "the mission changed" in r.stderr, r.stderr +ok("check: a changed mission makes its goals stale (2); an edited goal fails its seal; a narrowed eval derives nothing") + +# 10) spoon-fed: the human picks the gaps (that is the goal's approval); one intent at a time, reviewed mode +s, (s1, _), es = setup("spoon-fed") +assert review(s, "mission").returncode == 0 +ev = es["eval_id"] +r = s.run("goal", "derive", "--eval", ev) +assert r.returncode == 64 and "--pick" in r.stderr, r.stderr +assert s.run("goal", "derive", "--eval", ev, "--pick", s1 + "#all").returncode == 64 # who picked: --by +assert s.run("goal", "derive", "--eval", ev, "--pick", "nope", "--by", "Adeel").returncode == 2 +r = s.run("goal", "derive", "--eval", ev, "--pick", "%s#all,%s#fast" % (s1, s1), "--by", "Adeel") +assert r.returncode == 0, r.stderr +sg = json.load(open(r.stdout.strip())) +assert [x["gap_id"] for x in sg["gaps"]] == [s1 + "#all", s1 + "#fast"] and sg["autonomy"] == "reviewed" +assert sg["status"] == "approved" and sg["principles"] == P.stamp("spoon-fed") +ap = s.events("goal.approved") +assert len(ap) == 1 and ap[0]["data"]["sha256"] == sg["sha256"] and ap[0]["data"]["by"] == "Adeel" +assert review(s, sg["goal_id"]).returncode == 0 +assert s.run("goal", "check", sg["goal_id"]).returncode == 0 +sp = s.path("goal", sg["goal_id"]) +one = dict(sg, gaps=sg["gaps"][:1], targets=sg["targets"][:1]) +one.pop("sha256") +M.seal(one) +json.dump(one, open(sp, "w")) +r = s.run("goal", "check", sg["goal_id"]) +assert r.returncode == 2 and "not the content goal.derived recorded" in r.stderr, r.stderr +json.dump(sg, open(sp, "w")) +r = s.run("goal", "intents", sg["goal_id"]) +assert r.returncode == 0 and len(r.stdout.split()) == 2, r.stderr +s.run("goal", "next", sg["goal_id"]) +assert s.events("intent.normalized")[-1]["data"]["mode"] == "reviewed" +r = s.run("goal", "approve", sg["goal_id"]) +assert r.returncode == 64, r.stderr +ok("spoon-fed: --pick --by is the approval (goal.approved for that content); reviewed autonomy; edits re-ask") + +# 11) a hard line holds its gaps: a goal derived again never takes them (the loop moves on, the human sees +# them as held), and a goal keeps the principles stamp it was derived under (stale, not invalid) +BAD = {"title": "Pay strangers to praise us under made-up names", + "pass_criteria": [{"id": "quick", "metric": "secs", "op": "<=", "value": 8}]} +h, (h1, _, hb), eh = setup("autonomous", extra=[BAD]) +assert review(h, "mission").returncode == 0 +assert eh["gaps"][0]["gap_id"] == hb + "#quick", eh["gaps"] # closest to target: first +gh1 = made(h.run("goal", "derive", "--eval", eh["eval_id"], "--top", "1"))["goal_id"] +assert review(h, gh1, [{"category": "other", "tier": "hard", "reason": "paid fake praise deceives buyers"}], + "hard").returncode == 2 +r = h.run("goal", "check", gh1) +assert r.returncode == 2 and "hard line" in r.stderr, r.stderr +r = h.run("goal", "options", "--eval", eh["eval_id"], "--json") +o2 = json.loads(r.stdout) +assert [x["gap_id"] for x in o2["options"]] == [h1 + "#fast", h1 + "#all"], o2 +assert [x["gap_id"] for x in o2["held"]] == [hb + "#quick"], o2 +assert h.events("ethics.blocked")[-1]["data"]["gaps"] == [hb + "#quick"] +quick = o2["held"][0] +assert G.split_held(h.dir, [dict(quick, summary="Other words for the same gap")])[1] # held by its id as well +r = h.run("goal", "close", gh1, "--eval", eh["eval_id"]) +assert r.returncode == 2 and "blocked" in r.stderr, r.stderr # a hard-lined goal never closes +g2 = made(h.run("goal", "derive", "--eval", eh["eval_id"], "--top", "1")) +assert [x["gap_id"] for x in g2["gaps"]] == [h1 + "#fast"], g2["gaps"] +r = h.run("goal", "close", g2["goal_id"], "--eval", eh["eval_id"]) +assert r.returncode == 2 and "only an active goal closes" in r.stderr, r.stderr # derived, never handed off +newer = os.path.join(h.dir, "principles-next.md") # the standing block changes after derive +with open(P.SOURCE, encoding="utf-8") as fh: + text = fh.read() +with open(newer, "w", encoding="utf-8") as fh: + fh.write(text.replace("Ship Small", "Ship Small, always", 1) if "Ship Small" in text else text + "\n") +was, P.SOURCE = P.SOURCE, newer +try: + rc, blocks, _, facts = G.check(h.dir, g2["goal_id"]) +finally: + P.SOURCE = was +assert P.digest("autonomous", newer) != g2["principles"]["sha256"] +assert facts["principles_stale"] is True and not [b for b in blocks if "principles" in b], (facts, blocks) +g2p = h.path("goal", g2["goal_id"]) +old = dict(g2, principles=dict(g2["principles"], mode="spoon-fed")) +old.pop("sha256") +M.seal(old) +M.save(g2p, old) +r = h.run("goal", "check", g2["goal_id"]) +assert r.returncode == 2 and "not the content goal.derived recorded" in r.stderr, r.stderr +assert E.word_key("Pay strangers to praise us under made-up names (quick): secs <= 8") in E.hard_words(h.dir) +ok("a hard line holds its gaps out of options and derive; an older principles stamp is stale, not invalid") + +# 12) the intent tool: `mission` is a source, and a mission intent must name its goal +r = tool(p, INTENT, "new", "--source", "mission", "--raw", "x") +assert r.returncode == 64 and "--goal" in r.stderr, r.stderr +r = tool(p, INTENT, "new", "--source", "mission", "--goal", "gol_madeup", "--raw", "x") +assert r.returncode == 64, r.stderr +r = tool(p, INTENT, "new", "--source", "mission", "--goal", gid2, "--raw", "x") +assert r.returncode == 64 and "--gap" in r.stderr, r.stderr # a mission intent names its gap +r = tool(h, INTENT, "new", "--source", "mission", "--goal", gh1, "--gap", hb + "#quick", "--raw", "x") +assert r.returncode == 2 and "BLOCK[goal]" in r.stderr, r.stderr # the goal's gate: hard-lined +r = tool(h, INTENT, "new", "--source", "mission", "--goal", "gol_" + "0" * 20, "--gap", "x", "--raw", "x") +assert r.returncode == 2 and "no goal" in r.stderr, r.stderr +r = tool(s, INTENT, "new", "--source", "mission", "--goal", sg["goal_id"], "--gap", "nope", "--raw", "x") +assert r.returncode == 2 and "not a gap of goal" in r.stderr, r.stderr +sp0 = os.path.join(s.dir, "docs", "agents", "intents", s.events("intent.received")[0]["data"]["intent_id"] + ".json") +r = tool(s, INTENT, "check", sp0, "--mode", "autonomous") +assert r.returncode == 2 and "reviewed" in r.stderr, r.stderr # the goal's autonomy, not the caller's +sd = json.load(open(sp0)) +json.dump(dict(sd, provenance=dict(sd["provenance"], gap_id=s1 + "#fast")), open(sp0, "w")) +r = tool(s, INTENT, "check", sp0, "--mode", "reviewed") +assert r.returncode == 2 and "intent.received" in r.stderr, r.stderr # the record, not the file, says which gap +json.dump(sd, open(sp0, "w")) +it = json.load(open(paths[0])) +json.dump(dict(it, provenance=dict(it["provenance"], goal_id="")), open(paths[0], "w")) +r = tool(p, INTENT, "check", paths[0]) +assert r.returncode == 2 and "names its goal" in r.stderr, r.stderr +json.dump(it, open(paths[0], "w")) +ok("intent: a mission intent names its goal and gap, passes the goal's gate, and is checked in the goal's autonomy") + +# 13) the area is discovered; --help is the docstring +assert "goal" in p.run("--list").stdout.split() and "Aladdin handoff" in p.run("goal", "--help").stdout +assert p.run("goal", "nope").returncode == 64 and p.run("goal", "check").returncode == 64 +for x in (p, s, h): + shutil.rmtree(x.dir, ignore_errors=True) +ok("discovered as an area; usage errors are 64") + +print("mission-goal: %d checks passed" % PASS[0])