Skip to content

ci: run the soundness oracle on postgres 18 and mysql 8.4 - #38

Merged
colinhacks merged 2 commits into
mainfrom
ci-oracle
Oct 22, 2026
Merged

colinhacks merged 2 commits into
mainfrom
ci-oracle

Conversation

@colinhacks

Copy link
Copy Markdown
Owner

The soundness oracle (oracle/) is the check that catches an unsound inference change: it analyzes every statement in oracle/cases with the engine's sources, runs it on live servers over seeded edge values, and fails a case when a returned value contradicts the inferred type or row count. Until now it ran only by hand. This adds it to test.yml as an oracle job, beside the engine job, so every pull request and every release runs it.

  • Postgres 18 is installed on the runner from the PostgreSQL apt repository rather than run as a service container, because the json_schema_* cases call five JSON Schema validator extensions built into the server. The cluster is initialized with --locale=C, the locale the cases' expected output was recorded under.
  • The validators (pg_jsonschema, jsonschema on boon, postgres-json-schema, is_jsonb_valid and json_schema_validate) are built by .github/scripts/json-schema-validators.sh, each at the commit the cases were measured at, and staged by install path. The build takes about a quarter of an hour, so the staged files are cached under a key on the script, and saved right after they are built rather than only when the job passes.
  • MySQL 8.4 runs as a service container, as in the test job.
  • The oracle runs with --strict-coverage, so a statement that fails or times out on every run fails its case unless the case's unchecked.txt lists it.

Copilot AI balanced review requested due to automatic review settings October 22, 2026 21:29
@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
reylo Ignored Ignored Preview Oct 22, 2026 9:43pm UTC

Request Review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues. One minor suggestion inline.

Reviewed changes

I reviewed the new oracle CI job and the script that builds the five Postgres JSON Schema validators the json_schema_* cases need.

  • oracle job in test.yml: MySQL 8.4 runs as a service. Postgres 18 comes from pgdg and runs on the runner (initdb --locale=C, pg_ctl on 55432). The oracle runs with --strict-coverage over oracle/cases. release.yml calls test.yml through workflow_call, so releases also pass through this gate.
  • json-schema-validators.sh: builds pg_jsonschema v0.3.4 with cargo-pgrx 0.16.1 and pg-jsonschema-boon v0.1.10 with 0.18.1 (both match the upstream Cargo.lock), plus three PGXS extensions at pinned SHAs. All are staged by install path into one directory. I confirmed that cargo pgrx package --out-dir creates the directory but never clears it in either pgrx version, so the two pgrx packages and the make install DESTDIR= trees can share the stage.
  • Validator cache: restore and save are separate, so the quarter hour build is kept even when the oracle fails afterwards.

The oracle check run on e1b10b0 was still in progress when this review was written.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread .github/workflows/test.yml Resolved

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The cache key suggestion is applied cleanly.

Reviewed changes

I reviewed the one commit since the prior review, which changes the validator cache key.

  • Cache key from the installed server: Install Postgres 18 now has id: pg and outputs key=$ImageOS-pg<release>. <release> comes from pg_config --version (e.g. 18.1), and the restore key uses that output in place of pg18. A new 18.x minor release or a different runner OS (ubuntu22 → ubuntu24) now misses the cache and rebuilds the validators once. The test -n guards fail the step under the default bash -e shell if either value is empty, so the key can't silently fall back to a stale prefix.
  • Comment refreshed: the build time is now given as "several minutes", and the comment explains why the key names the Postgres release.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@colinhacks
colinhacks merged commit 299f6e6 into main Oct 22, 2026
6 checks passed
@colinhacks
colinhacks deleted the ci-oracle branch October 22, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants