A high-reliability automation starts by routing the hard cases away

Matthew Jewkes asks leaders to consider their strongest domain experts: people repeatedly interrupted by urgent customer issues and decisions. As routine execution becomes cheaper, he argues, deciding what deserves attention becomes more valuable.

He advocates fully automating a well-defined routine path while reserving human attention for exceptions. In his view, autonomous agents can be useful in development but hard to maintain and capable of generating additional decisions for experts.

An automation should have an escape hatch. AI can check whether an input fits the routine path, classify unusual or business-critical situations and route them for different handling instead of forcing every case through one workflow.

Jewkes describes a biopharma document-indexing system at Clarivate. He says it processed tens of thousands of tasks per month and reached 99.95% accuracy on a selected set of business-critical tasks, after substantial development investment.

The initial agent-like approach reached about 80–83% after a few sprints. The team gathered roughly 1,000 evaluation examples and reviewed them with domain experts, giving the experts concrete cases rather than asking them to fill in a blank spreadsheet.

The review revealed three kinds of problems: genuinely critical errors, ambiguous cases where experts or the organization had no settled answer, and unusual exceptions requiring additional human judgment. These should not all be treated as the same model failure.

Jewkes uses a simplified drug-target example to show ambiguity: experts may disagree about whether an indirect biological effect counts as a target. He contrasts that with emerging drug mechanisms that require humans to update the indexing scheme.

The team invested heavily in routing. Confusing or unusual inputs could be described and set aside; potentially critical cases entered a heavily evaluated path; unresolved outputs went to a person. Routine cases continued through the ordinary process.

After excluding ambiguous and unusual cases, the team reached about 95% accuracy on the critical path with standard approaches. Jewkes says extensive evaluation and iteration raised that figure to 99.95%, and deployment followed a random-sample comparison with historical work.

He claims the automated process outperformed historical expert work at scale, while acknowledging that a PhD expert examining a single case closely could do better. He attributes the difference partly to workload, time pressure and human fatigue.

Beyond saving time, the system groups the remaining work into critical reviews, ambiguous cases and unusual exceptions. Jewkes says this makes it easier for experts to decide where to spend their limited attention.

A backlog of ambiguous, marginally valuable cases can prompt an organization to decide whether to specify the rule or stop doing that work. Grouping unusual cases can reveal patterns that are hard to notice when they arrive one at a time.

He describes an expert using a structured set of exceptions to form an investment view before a major customer raised a related need. Jewkes sees such grouped exceptions as possible leading signals of market demand.

High-reliability automation is expensive to build: engineering time, evaluation review, organizational trust and ownership changes all cost money. He recommends choosing established processes with substantial value and clear customer pain.

Jewkes favors detail-oriented work with a reasonably specifiable process over work dependent on open-ended subjective judgment. For startups still discovering a business, he would retain human flexibility rather than rush into full automation.

His practical advice includes writing and reading evaluations, providing escape paths, versioning inputs and prompts, breaking tasks into smaller parts and capturing failures. He emphasizes that reviewing a large evaluation set is labor-intensive.

Jewkes closes with his prediction that competitive advantage will come less from selecting a particular orchestration framework and more from how organizations manage collective attention and judgment.