You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The verification pipeline checks two things: were test files created/modified, and do they pass. This is necessary but not sufficient — an agent can satisfy both conditions with trivial assertions like:
test("it works",()=>{expect(true).toBe(true);});
The test quality rules in the prompt help guide the agent, but enforcement is honor-system. There's no programmatic check that tests actually exercise the code that was written.
Ideas
Coverage delta: run coverage before and after, require that new/changed lines have coverage above a threshold
Assertion density: check that test files have a reasonable ratio of expect() calls to test cases
Mutation testing: run a lightweight mutation pass to verify tests actually catch regressions (probably too slow for the loop, but could work as a willie-style post-pass)
AST analysis: check that test imports actually reference the files that were changed
Context
The current gate catches "forgot to write tests" and "tests are broken." The gap is "wrote tests that don't actually test anything." Coverage delta is probably the most practical first step.
Closed as duplicate of #24 - Phase 4 (Audit Pipeline) will include a test quality agent that verifies test coverage, meaningful assertions, edge cases, and test isolation.
Activity
dominicnunez commentedon Feb 21, 2026
Closed as duplicate of #24 - Phase 4 (Audit Pipeline) will include a test quality agent that verifies test coverage, meaningful assertions, edge cases, and test isolation.