Skip to content

feat(engine): carry keys through set operations and column lists - #147

Merged
colinhacks merged 7 commits into
mainfrom
more-subquery-keys
Oct 30, 2026
Merged

colinhacks merged 7 commits into
mainfrom
more-subquery-keys

Conversation

@colinhacks

Copy link
Copy Markdown
Owner

More of the unique keys a subquery, a WITH query or a view keeps, on all three databases, so that reading one by such a key infers one row at most.

Rules

Rule Postgres SQLite MySQL
A set operation without ALL keys its whole row each column one table column on both sides, of one type each column one column on every arm, of one affinity and collation each column an exact number on both sides
INTERSECT and EXCEPT keep their left side's keys yes yes yes
A key column the subquery's WHERE pins to one value may be left out of its select list yes yes yes
A column list renames the keys by position: (…) s (a, b), with s (a, b) as (…), a view created with one yes positional already yes, views included
A WITH query reads a sibling WITH query's keys yes inlined already yes
A key column an inner equality makes equal to a kept column (same type, identity equality) yes
A unique index on an expression the select list keeps under an alias yes

Postgres reads a sibling WITH query by deparsing its statement, and so reads it under the WITH queries listed before it.

On MySQL, a parameter compared in a join's ON, inside a derived table or a WITH query, or through a column list or a set operation now takes its column's type (a BIGINT's is int8, not the prepare's int4). A set operation gives the parameter one column per side, whose types must agree.

Checks

  • New statements in subquery_keys, sqlite_subquery_keys and mysql_subquery_keys, adversarial ones included (a swapped column list, a union of mixed types, a recursive sibling, an outer join's equality, an unaliased index expression), run over seeded edge values with no contradiction.
  • Existing answers that move:
    • sqlite_subquery_keys/union.sql and mysql_subquery_keys/view_column_list.sql: many to maybeOne.
    • MySQL parameters compared through a derived table, a WITH query or a set operation: int4 to int8 for BIGINT columns (mysql_subquery_keys, mysql_limits/union_limit_5).
  • The whole oracle with --strict-coverage passes but for the DuckDB cases, which need --features duckdb and which this change does not touch. The engine tests pass, the workspace builds with no warning, and nub run test with both test databases passes 303 tests, none skipped, with the three rebuilt components matching their sources.

Copilot AI balanced review requested due to automatic review settings October 30, 2026 17:19
@vercel

vercel Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
reylo Ready Ready Preview Oct 30, 2026 5:48pm UTC

Request Review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

One soundness regression in the Postgres reading of enclosing WITH queries. A nested WITH RECURSIVE that reuses an outer WITH query's name gets the outer query's keys, so a query that returns many rows is typed maybeOne. Also one docs paragraph that says more than the engine does.

Reviewed changes

This review covers the whole PR at 5655da1: key inference through set operations, column lists, sibling WITH queries and key columns left out of the select list, on Postgres, SQLite and MySQL, plus MySQL parameter typing through derived tables, WITH queries, set operations and ON clauses.

  • Postgres set operations — set_keys keeps the left side's keys for intersect/except (with all or without). A non-all set operation is keyed on its whole row when every column passes one table column through on both sides, with one type.
  • Postgres omitted key columns — a key column left out of the select list still counts when the WHERE pins it to a value (pinned_by_value), when an aliased index expression keeps it (kept_expression), or when an inner equality keeps a same-typed identity column in the list (equated_output).
  • Postgres column lists and sibling WITH queries — renamed_keys/inner_name map keys by position. Catalog::enclosing_bodies holds deparsed sibling and enclosing WITH queries, so Source::Enclosing now has keys.
  • SQLite compound selects — compound_keys adds a whole-row key over arms that are all plain columns of one affinity and collation. It keeps the left side's keys for intersect/except. A key column the WHERE pins may also be left out.
  • MySQL keys — WITH queries and views with column lists are now read by position. set_keys keys the whole row only when every column is an exact number on both sides. A key column the WHERE pins may be left out.
  • MySQL parameter types — compared now looks inside WITH bodies, derived tables, each side of a set operation and join ON clauses. A parameter compared on both sides of a set operation keeps a type only when the two sides agree.
  • Oracle cases, docs and rebuilt components — new statements in subquery_keys, sqlite_subquery_keys and mysql_subquery_keys; reblessed expectations; postgres.mdx and mysql.mdx updates; three rebuilt engine.core.wasm.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread engine/postgres/src/cardinality.rs Resolved
Comment thread site/content/docs/postgres.mdx Resolved

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. Both findings from the previous review are fixed.

Reviewed changes

This run covers 745f233, the one commit since the review at 5655da1.

  • Fixed WITH query shadowing on Postgres — Catalog::scoped now adds an entry for every WITH query in scope, and its statement is None when the query is in a recursive clause, isn't a SELECT, or doesn't deparse. When the nearest entry for a name has no statement, enclosing_body returns None. This means an inner WITH query hides an outer one with the same name even when the engine can't read it. A recursive clause's own Source::With bodies are never read, so the upto cutoff can't skip a forward reference to a later WITH query in the same clause.
  • Added an oracle case — subquery_keys/recursive_shadows_enclosing.sql is the failing statement from the earlier review, and it now infers many.
  • Corrected postgres.mdx — the docs now give the whole-row condition for set operations and say that intersect/except keep the left side's keys with or without all. They also cover the alias rule for an indexed expression, and the "leaves no key" line now exempts an expression that a unique index covers.
  • Rebuilt the Postgres component — packages/postgres/engine/engine.core.wasm was rebuilt in the same commit.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

A set operation without ALL keys its whole row, and INTERSECT and EXCEPT keep their left side's keys. A column list renames a subquery's or WITH query's keys by position, a WITH query reads a sibling's keys, and a key column the WHERE pins to one value, an inner equality makes equal to a kept column, or an aliased index expression keeps, still counts.
A compound SELECT but UNION ALL keys its whole row where every column is one column of one affinity and collation on each arm, INTERSECT and EXCEPT keep their left side's keys, and a key column the WHERE pins to one value still counts.
A set operation without ALL keys its whole row where every column is an exact number on both sides, INTERSECT and EXCEPT keep their left side's keys, a column list renames a derived table's, WITH query's or view's keys by position, and a key column the WHERE pins to one value still counts. A parameter compared in a join's ON, inside a derived table or a WITH query, or through a column list or a set operation takes its column's type.
A recursive WITH query, a write, or one that does not deparse is still in scope, with no statement to read, so a reference to its name gets no keys rather than an outer WITH query's of the same name.
@colinhacks
colinhacks merged commit daf9145 into main Oct 30, 2026
12 checks passed
@colinhacks
colinhacks deleted the more-subquery-keys branch October 30, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants