How a 4B model trained with RL produced query plans faster than Postgres

The premise comes from Leis et al., who asked in 2015 how good query optimizers really are and asked again ten years later. The write-up's summary of both papers is that optimizers continue to leave much to be desired despite a decade of research, and that join ordering on its own is NP-hard. Verification, by contrast, is cheap: a good plan runs fast and a bad one runs slow. That single axis, execution time of a query, is what reduces the problem to reinforcing whatever behaviour produces faster plans.

Postgres already enumerates plans, so the model competes for the choice rather than the execution. For a three-table join the write-up counts 4,608 physical plans: 2 join trees, times 2^2 input orientations, times 3^2 join algorithms (hash, merge, nested loop), times 4^3 scan methods (sequential, index, index-only, bitmap). At four tables that becomes 442,368, at five 33,177,600, at six 1,783,627,776 and at seven 532,030,685,184. Postgres does not evaluate all of them: the article notes it uses dynamic programming to prune the search space, and a genetic algorithm for queries involving 12 or more joins.

The stakes are visible in that query, which runs against a slice of the IMDb dataset. With company_name filtered to roughly 5k Japanese companies and title filtered to roughly 200k rows from the 2000s, joining company_name to movie_companies first yields about 100k rows into the second join; joining title to movie_companies first yields about 400k. The article's conclusion is that the second ordering does 4x the work for the same result. Both estimates assume a uniform distribution, which is the assumption real table statistics violate.

The training loop is four RL rollouts for a single query. Qwen produces a candidate strategy per rollout and sends it to Postgres for measurement against Postgres's own default plan; scalar rewards are assigned to each rollout and flow backwards to update Qwen's weights. Because the environment is inherently noisy, the author lists designing a custom GRPO variant for scoring RL rollouts as one of the project's results.

The cold start is the part worth noting for anyone reproducing this. The 4B model was initially unable to produce a query plan for 99 of the 113 queries. The article also lists running off-policy distillation across half a thousand GPT-6 Astra agent trajectories among its results, and the cost line attributes the ~$400 in OpenAI API fees to generating those Astra trajectory demonstrations; the text available here does not spell out the exact ordering of SFT, distillation and RL.

The rig is split across two machines: vLLM and the trainer on the rented 2x H100 node, and four Postgres containers running on the author's desk. The measurement problem in that setup is Linux page cache contention across concurrent containers, since four databases racing on one host produce timing noise that a reward function will happily learn from. Constructing a Postgres measurement rig that minimizes that contention noise is listed as one of the project's headline results, not an implementation detail.

Read those two figures separately. The geometric mean averages per-query ratios, so a query dropping from 20ms to 10ms counts the same as one dropping from 60s to 30s. The summed latency decrease of 44.7% is the one dominated by the slowest queries in the set. Neither number says how many of the 113 queries got slower, and the source text available here does not report that distribution.

On checking that Postgres honoured the plan it was given, one commenter on Hacker News notes that pg_hint_plan has a debug log you can use to verify whether Postgres actually used the hint, and says this was used during evaluations. The article text supplied here does not describe the hint mechanism or name pg_hint_plan, so treat the plumbing as reported by commenters rather than documented in the source.

One commenter quotes the author's cost line back at the result and argues the ~95 hours of training are not included in the benchmark numbers, adding that every database vendor knows its optimizer could do better given lots of time to derive plans. The same commenter's open question is whether you can afford to spend hours every now and then updating a 4B model so it keeps picking good plans. A reply puts that maintenance in the same bracket as a scheduled backup window, while arguing a deterministic approach, such as analysing actual distributions instead of assuming uniformity, would likely beat re-fine-tuning.

Another commenter answers the equivalence worry by describing where the boundary sits in this setup. It is a practitioner's reading of the design, not a statement from the write-up, but it is the reading that determines how much correctness risk a deployment carries.

If that reading holds, correctness is not the hard part, because Postgres retains it. The hard parts are the latency budget and the regressions. The source reports plan execution time; it does not report how long the 4B model takes to emit a candidate strategy, and a model served on vLLM sitting in the planning path is a fixed cost paid per query. That points the technique offline for anything with sub-second defaults: optimise per query shape, cache the resulting hint, and keep the model out of the request path.

Fallback has a floor but not a ceiling. Postgres ignoring an unusable hint protects you from nonsense, not from a valid hint that happens to be slower than the default on today's data, and the source does not quantify how often that occurred across the 113 queries. Anything placed in front of a real workload would need a per-query gate that compares the hinted plan's measured time against the default and reverts on regression, which is the same comparison the RL reward already performs, run continuously instead of once.

The result is a 4B model tuned against 113 join-heavy queries on one IMDb-derived schema. The source text available here does not say how the model behaves on queries it has not seen, on a different schema, or after the underlying data distribution shifts, and the uniform-distribution assumption used in its own cardinality worked example is exactly the thing that drifts. Commenters raise the currency of statistics and periodic recounts as the practical failure mode; the write-up does not address it.

Three things would change the engineering picture: a per-query regression count rather than aggregate speedups, an inference latency figure to compare against a planning budget, and a run on a schema the model was not trained on. Until those exist, the supported claim is narrow: a 4B model can find plans that beat Postgres's default search on a fixed 113-query workload, measured offline, for roughly $800 in rented H100 time and ~$400 in API fees.