Why AI coding gains depend on more than faster code generation

Martin Harrysson, Prakhar Dixit and their colleague Shilpa describe patterns they say they have observed while advising large organizations on AI in software development. Their focus is the way people and operating models change.

Harrysson says expectations initially outpaced results: many large companies reported modest improvements despite widespread AI use. In a later survey of several hundred clients and large enterprises, he says the top 20–25% were beginning to see much larger end-to-end gains, around twofold.

The speakers organize their observations around four areas: redesigning the development workflow, changing team structure and roles, building a suitable technology and context layer, and managing organizational change.

Dixit describes some clients moving from two-week sprints toward one- or two-day workflows: agents work through tasks, people validate results and the loop repeats. Teams decide where human review belongs and adjust that boundary as confidence grows.

The workflow can extend beyond building and testing into product work and release. Dixit says standardized artifacts, such as requirements and specifications, help agents work consistently when large organizations store context in many different ways.

On staffing, Dixit reports a shift among some clients toward pods of two to four people. He says smaller teams do not necessarily mean reduced headcount; organizations may create more teams while broadening individual responsibilities.

He describes product-oriented roles converging toward a “definer” and development and quality roles toward a “builder.” He says there is not yet an agreed industry vocabulary for these roles.

Dixit argues the challenge is assembling tools around an organization’s own context. Knowledge held by long-serving subject-matter experts must be documented so agents can use it without repeatedly depending on the same people.

He also emphasizes trust: governance, security and stability must accompany agent adoption. Scaling beyond a successful pilot requires preparation across the operating model, staffing, technology and costs, including AI financial operations.

A colleague identified in the talk as Shilpa says the difficulty is increasingly selecting and curating useful skills from a large supply. The relevant set depends on the organization’s work and context.

She warns that faster code generation may coincide with worse quality. She says the reports they reviewed show increases in bugs and incidents even when measured per developer or per pull request, and calls for equal attention to review skills.

The team recommends curating strong examples of desired work rather than giving agents only average examples from the existing codebase. Policies, standards and quality checks should be available throughout the development process.

Examples of checks include whether code is maintainable, appropriately simple and compatible with other branches. Applying expectations earlier can catch issues before a final quality gate.

As build and test become faster, bottlenecks can move upstream to requirements and design, downstream to release, and sideways to risk, legal, compliance and design teams.

For the upstream stage, the speakers suggest connecting agents to relevant enterprise context, including customer information and logs, while linking that context to what needs to be built. Release requirements also need to be mapped in advance.

Shilpa describes a bank application where color and accessibility rules required coordination with another team. Such parallel requirements can become bottlenecks unless they are identified and made available to the workflow.

Harrysson says measuring impact remains unfinished work. Organizations spend on tools, tokens and training but often lack confidence that they are measuring the right outcomes.

Some clients are exploring AI-assisted ways to assess the value of work and allocate resources, he says. He considers the idea promising but difficult; existing engineering measures still have a role.

He describes a near-term management heuristic: splitting an existing group into smaller pods and more teams can increase parallel capacity. He says leaders will need more sophisticated measurement over time.

The speakers close by arguing that results depend on many coordinated changes rather than introducing a single tool. Harrysson reports seeing substantial improvements in some clients and remains optimistic about the direction.