En 30 secondes
Cognition says merged pull requests rose about 40-fold in a year while its engineering team grew 2.7-fold. Co-founder Walden Yan attributes the change to more than adopting a coding agent: teams now own customer problems end to end, while isolated virtual machines let agents test changes and send evidence back through Slack. He describes agents improving GPU inference through repeated measurements and building an Android app against an iOS reference over 44 days. The figures are Cognition’s account of its own workflow; merged changes measure throughput, not quality by themselves.
- 1Merged changes rose faster than engineering headcountYan reports a roughly 40-fold rise in merged pull requests alongside 2.7-fold headcount growth over a year. He believes quality held up, while the chart measures the number of merged changes.
- 2Small pods own the customer problem, not just implementationTwo or three engineers handle bug reports, product decisions and quality checks, delegating described work to agents.
- 3Agents test against local services instead of live dependenciesCognition moved services into a virtual machine with mocked external dependencies and rebuilds the environment hourly.
- 4Agents can improve the process that runs themA daily channel surfaces proposed changes to build speed, tests and other sources of wasted time.
- 5Give agents measurable jobs with reporting and escalationFor latency, Yan proposes instrumentation, alerts and recurring updates rather than a separate prompt for every fix.
- 6GPU optimization combined repeated trials with human approvalAn agent tested throughput changes on allocated GPUs and presented its best candidates to an engineer.
- 7An Android app used the iOS app as a continuing referenceYan describes a 44-day effort with hooks, feedback and checks for parity; he says the result was planned for release.
À retenir
- Give agents isolated environments and repeatable tests before expecting them to complete work end to end.
- Organize engineers around customer problems, testing and product decisions as implementation becomes more automated.
- Turn an ongoing goal into measurements, repeated experiments and escalation points for human judgment.
- Cognition’s merged pull-request growth is a throughput measure; assess the quality and impact of changes separately.
Lire le texte remanié, chapitre par chapitre
A software factory changes the engineer’s work
Cognition reports a sharp increase in merged changes. Yan describes a default agent-driven workflow and smaller teams with wider responsibility.
Cognition's merged pull requests grew roughly 40-fold over a year while engineering headcount grew 2.7-fold. I believe the changes are at least as high quality as before. That is our experience using merged changes as a measure of throughput; the important question is what we changed in our codebase and teams to achieve it.

The basic idea of a software factory is that development runs itself by default. At Cognition, Devin can take a change from a bug report through a merged pull request and subsequent monitoring, escalating when human judgment is needed. That default changes what engineers need to do.
If an engineer only picks a ticket and asks an agent to implement it, I don't think they are set up for success. Once work is clearly described, we can send it to Devin. Engineers need to take responsibility for defining and directing more of the work.
We split larger teams into small pods of two or three engineers and gave them wider ownership as well as groups of agents. Engineers listen to customer bug reports, test the product, do product management and maintain their own list of work for agents to implement.
Make testing possible without live services
Agents receive customer reports through Slack and return evidence of fixes. Local services, mocks and hourly VM builds let them test independently.
We work from Slack throughout the process. A customer can send a bug report with a video; we forward it to an internal channel and tag Devin. It proposes a fix and returns a recording as evidence, which we can share with the customer without opening an editor or GitHub most of the time.
That workflow required changes to the infrastructure underneath it. Previously, testing meant connecting to staging services and a database, then having engineers click through the flows. We moved services into a local Docker Compose setup and mocked third-party dependencies so an agent can run the application inside one virtual machine.

We rebuild these virtual machines hourly against the latest code. If a snapshot build fails, Devin is the first responder and fixes it. An agent started from Slack therefore gets a fresh machine with a recent version of the code and can test without touching live services.
Agents also need instructions for testing the codebase. Our cloud agents write and update their own testing skills as the code changes. As a result, some engineers can review and test work without opening a laptop, and several use iPads in their development workflow.
I think controlled cloud virtual machines can also reduce the risk from secrets left on individual developer machines. Testing need not be simple: when we built voice and Zoom participation, Devin created multiple accounts inside a VM, joined a call and exchanged voice messages to check the feature.
Let agents improve the factory
Agents analyze development history and propose improvements to builds, tests and scripts.
A cloud agent setup records code changes, pull requests and decisions. We run agents over that history to find wasted time or tokens, opportunities for scripts and missing tests. They can then suggest improvements to the process that produces the software.
Every day Devin posts ideas to a channel. One suggestion might save a minute in continuous integration by changing the Node setup; another might catch more bugs with a lint script before AI code review. The agents can help optimize the factory, not just work inside it.
Move from individual tasks to measurable jobs
Yan suggests giving agents an outcome and the measurement, alerts and reporting needed to pursue it over time.
The next shift is from giving an agent a task to giving it a job. If I care about the 99th-percentile latency of an application—the slow end of users' response times—I want the agent to establish the measurement, create alerts, break down the work and report progress, rather than wait for me to specify every step.
Make a goal measurable wherever possible. That might mean measuring click latency, layout shifts or onboarding retention. If a team can express a useful outcome as a number, I encourage it to set up an agent to work toward improving that measure continuously.
Testing GPU inference improvements
An agent tried throughput strategies on allocated hardware and escalated its strongest candidates for human approval.
Cognition received Nvidia Vera Rubin GPUs before inference kernels had been built for them. We allocated a set of GPUs to an agent and gave it a goal: maximize output tokens per minute. It tried approaches over about a week and sent its best candidates to a human engineer for final approval.

The agent found eight or nine changes. In the workload shown, the cumulative result moved throughput from below our GB300 comparison to significantly above it. The point is not a single clever prompt: the system had hardware, a measurable target, repeated tests and a human approval step.
Maintaining an Android app against an iOS reference
A long-running agent used code changes, customer feedback and QA to pursue parity. Yan says a working app emerged after 44 days.
A second example began because our small team lacked capacity to staff an Android application fully. An engineer, Samuel, asked whether the existing iOS app could act as a blueprint and gave Devin responsibility for maintaining Android parity.
It was not one short session. He set up continuing hooks to watch the iOS codebase, listen for user feedback in Slack, improve the Android app and test it against the existing product. The agent could compare details and escalate the most important decisions.
After 44 days we had a working application that we planned to release. Our timeline shows days with few or no human messages and as many as 40 or 50 pull requests merged. Engineers could inspect recordings in Slack rather than repeatedly opening the agent session.
The lesson I take from this is that agent results depend on the loops around them. With testing, feedback and goals in place, a team can ask agents to sustain work over time rather than issue a fresh instruction for every change.
Sur la même chaîne
58:03EN
19:38EN
19:55EN
23:19EN
17:31EN
17:31EN