Harvey’s agents scaled until their platform and merge queue broke

More than one agent started on Harvey’s platform every minute on average in the preceding month, says Joey Wang. After 10,000 agent-written pull requests, his main lesson is that the difficult failures were in the surrounding platform: scheduling, testing, review and merging.

About 460 people at Harvey used the platform in the previous month. Its work expanded beyond coding: employees ask data questions in Slack, and one marketing colleague requested a storyboard for a product video from a prototype and a talk track.

Wang calls the internal cloud platform Spectre. It runs existing coding agents rather than competing to build a new one. In his view, cloud execution supplies the company’s secrets, real stack, alerts, tickets, shared rules and capacity that individual laptop tools do not provide on their own.

Instead of rebuilding the same integrations for each coding tool, engineers can use their preferred agent and call Spectre underneath. Work can start locally, move to the cloud and return. The shared platform handles the stages after the initial code is written.

For an overnight alert, an agent can write a fix, start a real copy of the stack and attach evidence to a pull request. A follow-up loop reads CI failures and pushes fixes. The request enters the common review queue, its owner is nudged during working hours, and an agent can enable automerge after approval.

Wang says the follow-up loop was enabled for everyone in early October. That month, four out of five agent PRs that merged into the backend had gone through it. Teams also record project knowledge and trusted skills so agents share test instructions and an audit trail.

The next rule was: if a person can click an action, an agent should be able to call it. The command line can start tasks, enable follow-up or automerge, and publish a page. A laptop agent, a sandboxed agent or CI can use the same interface.

In May, only one task originated from the terminal; by October the command line started about 6,500 per month, while the web app continued to be used. Some people started hundreds of tasks a month without opening the app.

Wang used a local agent as a coordinator to split work into ten milestones and send hundreds of tasks to Spectre, each with an isolated environment for end-to-end validation. He also describes an assistant that lists questions waiting for his response and drafts answers for his approval.

The slowest task stage was not model inference but checking out repositories: copying every repository took more than a minute on average. The team kept a copy of each repository on each machine and allowed tasks to borrow it.

In August, a faulty release triggered an alert per workspace. A workflow tagged the agent on every alert, producing around 900 tasks in four hours instead of the usual roughly 50. All chased the same bug until no node capacity remained. The fix joins repeated alerts to an existing investigation.

In October, an engineer sent roughly 950 legitimate tasks from the command line in about three hours. One laptop filled the cluster and other people’s work queued behind it. Wang treats this as a platform design problem, not misuse by the engineer.

Four days later the team introduced a fair launch queue. Interactive users receive priority, users get a fair share and can borrow unused capacity, and queued tasks show their place in line. In a synthetic burst of 300 tasks, task-list p95 latency fell from 1.7 seconds to 71 milliseconds with the queue enabled.

Wang says a later busy day handled more than twice the task volume of an August peak with better completion. Several changes happened at once, so he cannot attribute the improvement to one fix. Larger bursts continued to arrive while the platform kept up.

The bottleneck then moved to merging. Company-wide PR volume rose about 60% in one month, and a merge that once took 15–30 minutes exceeded an hour. The merge queue had to be switched off; PRs now rebase on main and rerun CI, leaving everyone exposed if main breaks. Wang says this problem remains unsolved.

Automations that start on a schedule or event are a fast-growing source of work. Around 110 people had an automation running in the previous month. A job switches itself off after five consecutive failures and can carry a budget or expiry date, limiting unattended task creation.

Wang is still asking how to run persistent channel agents, require evidence before a person reviews a PR, and merge when agents continuously open requests. Only about one merge in twenty currently carries that kind of proof, he says.

His advice is to own what happens after code is written, make actions callable by agents, and cap how much work an agent, person or alert can launch. Engineers can choose their preferred coding agent; the shared system still has to run, check and merge the work.