Mercor Labor Index finds 90% of US working hours sit outside AI benchmarks

For engineers, this means published benchmark scores say little about most of the work an enterprise deployment will touch. Mercor also shows that the gaps follow a pattern. That pattern indicates which parts of a workflow your own eval suite is likely to leave untested.

The unit of measurement is the O*NET task. For Lawyers, O*NET lists 22 tasks. Examples include drafting wills and contracts, interpreting laws and regulations, presenting evidence in litigation, and supervising legal assistants. An LLM judge reads each benchmark prompt and tags it with every O*NET task it thinks the prompt corresponds to. This follows earlier work by Wang et al., and one prompt can map to several tasks.

A task counts as covered on the map only if the judge has mapped it to at least 5 prompts from the 29 benchmarks. The map shades each occupation by the share of its tasks that are represented. Its legend separates coverage with at least 1 prompt, coverage with 5 prompts, and coverage from APEX only. To turn task coverage into a share of working hours, Mercor uses the LLM-derived estimates of each task's share of working time from Hatgis-Kessell et al.

Two steps in this pipeline are model judgments. First, the prompt-to-task tags come from an LLM judge, and Mercor validates them against human expert tags only on its own APEX benchmarks. The page excerpt does not report an agreement rate. Second, the working-time shares are also LLM-derived. The 90% figure therefore rests on two layers of model estimation, and the published material does not quantify the error in either.

Mercor selected benchmarks against five criteria. The full list sits in a Benchmark Index tab that is not reproduced in the source excerpt.

Mercor says these choices bias its numbers in known directions. Any incomplete list pushes coverage estimates down, and Mercor calls that bias hard to bound. The per-family cap understates coverage in heavily benchmarked areas such as software engineering. Excluding saturated benchmarks may understate coverage of tasks that LLMs find easy. Mercor argues that including them would be worse, because saturation may reflect low task realism. It concedes that criteria like economic realism involve subjective judgment, and it labels the results provisional.

The family table shows the skew directly. Coverage is high in some families: - Computer & Math: 29 of 36 occupations covered - Business & Financial: 33 of 48 - Legal: 5 of 7 It is low in others: - Healthcare Practitioners: 12 of 89 - Education & Library: 9 of 62 - Sales: 6 of 22 Six of the 22 job families have no occupation with a task at the 5-prompt bar.

Mercor's regressions identify three drivers. First, tasks rated exposed to AI in Eloundou et al. (2023) are about 6x more likely to be benchmarked than unexposed tasks. Mercor says this explains why benchmarks are full of software engineering and financial analysis tasks but sparse on truck driving and gas plant operation. Second, tasks from occupations with larger annual wage bills remain significantly overrepresented after controlling for exposure. Wage bills are computed from BLS mean wages and headcounts, assuming a 40-hour week. Third, within an occupation, tasks with the highest O*NET importance ratings are about 6 to 10x more likely to be covered than those rated least important.

One commenter on Hacker News read the wage finding as a sign that the industry wants AI to take the good jobs. Mercor frames it differently. It calls the pattern a reasonable allocation of the limited resources spent on high-quality, economically relevant benchmarks so far. It also warns that without many more and more diverse benchmarks, the pattern will leave significant blind spots. Mercor's own example involves equally AI-exposed work: benchmarks heavily cover lawyers' tasks but barely cover those of legal secretaries or court clerks.

If your internal evals borrow from public benchmarks, they inherit these gaps. Based on Mercor's patterns, the likely blind spots are:

Commenters on Hacker News raised a different gap. One wrote that about 80% of their software engineering job is fighting disagreements between stakeholders. Another said many things they do are not covered by standard tracking and only become visible when they are not there. A third argued that much human work lacks the tight feedback loop that training and agentic workflows need. Mercor's method cannot settle these points, because it only measures tasks O*NET has written down. Some coordination work does appear in O*NET, such as Supervise legal assistants, but the sources do not say how well such tasks are covered or how much work falls outside O*NET's lists.

A simple average over job-relevant prompts overweights the tasks that benchmarks sample most heavily. Mercor tests a correction on APEX for investment banking, law, accounting and software engineering. Each O*NET task with at least 5 prompts gets a task-level score, and those scores are averaged with equal weight. Mercor reports that the weighted and unweighted scores are generally quite similar.

For Financial and Investment Analysts (APEX-1 plus APEX-Agents, 11 tasks, 176 prompts), the per-model results included:

The shifts are under 4 points, and they go in both directions. Some models drop and others rise. GPT-6 Astra Max and Claude Opus 5 Max tie at 65.5% by prompt but separate by 1.2 points by task. For anyone choosing a model per occupation, task-level aggregation in an internal harness stops prompt-heavy categories from dominating the score, and it needs no new prompts. It does nothing for tasks below the 5-prompt bar, which is all but 924 of the 18,797.

Mercor calls the map a first step toward a continuously updated resource and plans to expand the benchmark list. The excerpt does not give an LLM-judge agreement figure. It has no sensitivity analysis on the 5-prompt threshold and no estimate of how far the 90% would move with a larger benchmark set. Every bias Mercor lists pushes measured coverage down, so read the 90% as a provisional, upper-end estimate of the gap. The more useful output for engineers is the direction of the skew: toward exposed, important, high-wage-bill tasks, and away from the work around them.