The Capability Cliff Nobody Is Talking About
Two data points surfaced this week that should change how operators assign work to AI systems.
First: a model successfully reimplemented a ~16,000-line toolkit by observing its behavior alone — an extraordinary feat of pattern reconstruction. Second: a separate model from the same generation produced an unusable result when asked to customize a library for METR, a task that required matching novel constraints rather than recognizing existing ones. Same capability tier. Radically different outcomes.
Nate Jones frames this as narrow superintelligence — AI capability spiking unevenly across domain types rather than converging toward general competence. An unreleased OpenAI model authoring 700+ math research papers in the same week reinforces the point: these systems are not uniformly strong, they are extraordinarily strong in bounded, pattern-dense domains and unreliable the moment a task requires contextual judgment or constraint interpretation.
What This Means for Task Delegation
Most operators are still delegating based on effort — offloading whatever takes the most human time. That is the wrong axis. The right question is domain type.
High-confidence delegation: Reconstruction, synthesis, pattern matching, summarization at scale, code generation from clear specs. These are domains where AI performance is genuinely superhuman in many contexts.
Lower-confidence delegation: Customization to novel constraints, tasks requiring judgment about unstated priorities, anything where "close enough" produces downstream failures. This is where the capability cliff lives — and where quiet errors accumulate.
RevOps and product teams are particularly exposed here. Workflow automation built on AI judgment calls — routing logic, scoring models, exception handling — sits squarely in the lower-confidence zone if the task definitions are vague or the constraints are implicit.
The Audit You Should Run Before Q4 Planning
Before expanding AI delegation in your next planning cycle, run a simple domain audit on your current AI task assignments:
- List every AI-handled task. Include agents, copilots, and any automated output that feeds a human decision.
- Classify each by domain type. Is this pattern-matching or constraint-interpretation? Reconstruction or customization?
- Flag the constraint-interpretation tasks for human review gates. Do not remove AI from the workflow — add a lightweight checkpoint where the output touches a judgment call.
The failure mode is not that AI is unreliable. It is that operators treat capability as uniform when it is not. The teams that build accurate mental models of where AI is genuinely superhuman — and where it is not — will outperform the ones running on hype-calibrated assumptions.