Microsoft’s Jay Parikh on AI Infrastructure, Model Choice and Agent Security

Matthew Berman interviews Jay Parikh, Microsoft’s executive vice president of CoreAI, about the developer stack, office culture, infrastructure, model choice and security. The conversation took place around Microsoft Ignite in November 2025; product claims, plans and forecasts should be read in that historical context.

Parikh says Microsoft formed CoreAI in January and presented its product strategy at Build in May. With models, frameworks and protocols such as MCP evolving rapidly, he sees the AI stack as unfinished. The goal is to help builders and enterprises assemble these pieces into useful applications.

He describes a layer of reimagined developer tools above Microsoft Foundry, which he calls an agent factory for building, deploying and observing agents. Security and trust must be present from the start, he argues, because agents may plan, call tools and access data in ways that are less deterministic than traditional software.

Deployment must also span cloud and edge devices, Parikh says, reflecting the needs of different sectors and geographies. Asked why Microsoft combined the teams, he says the common objective is to serve builders across the stack. He wants products, measurement and customer engagement to work together so developers are not left with disconnected, insecure or unobservable components.

Berman asks about CoreAI’s planned return to in-person work in the new year. Parikh argues that face-to-face collaboration helps people mentor one another, share discoveries about Copilot, and jointly learn how to provide context or scaffolding for difficult AI tasks. This is his rationale for the team’s policy, not evidence that every Microsoft employee had the same requirement.

Berman notes that many enterprises and individuals are still learning how to use AI, and asks whether that is also true inside Microsoft. Parikh says the company serves many roles, including HR, finance, engineering and security, and must evolve as quickly as the technology. He describes an internal program aimed at reducing administrative work and freeing time for creative product development.

Parikh offers product prototyping as an example: specify an idea in a document, ask agents to build five alternatives in parallel, and use customer feedback to improve the best elements. He believes this can broaden the creative search and accelerate feedback. He says cultural change and learning how to work differently arise in most of his customer and partner conversations.

Asked about the blurring of product, development and design roles, Parikh says AI tools let people contribute beyond their traditional specialties. A systems engineer might generate a UI concept; a product manager or designer might use Copilot to propose a bug fix. An engineer may still review the resulting pull request against code standards.

Parikh thinks the biggest differences are between individuals rather than job titles. He contrasts infrequent users who are delighted whenever AI works with frequent users who are frustrated because they push it on harder tasks. In his view, the latter group tends to experiment with models, context engineering, evaluation, fine-tuning and reinforcement learning.

Parikh describes some teams orchestrating groups of agents on complex tasks and using verification systems to check outputs. He says some teams treat code as an intermediate artifact and do not inspect every line, with verification agents feeding problems back to coding agents. This is his description of particular workflows, not proof that automated verification is sufficient in every setting.

Berman turns to the Fairwater data center and asks whether power has become a greater constraint than GPUs. Parikh says the answer depends on ambition and location: power, land, transformers and other heavy equipment all affect expansion. Some jurisdictions have paused data-center growth while others are encouraging it, he says.

Hardware requirements also change from one generation to the next, including cooling, power and networking. Parikh argues that agent workloads raise demand for ordinary compute, storage and network capacity as they call enterprise tools and systems. Scaling AI therefore involves the whole system, not only GPUs.

Berman raises the AI infrastructure bubble debate and compares unused fiber capacity with claims about idle GPUs. He cites Satya Nadella’s remarks about power shortages leaving some GPUs waiting. Asked for Microsoft’s utilization, Parikh does not provide a utilization percentage; he instead emphasizes continuous measurement and optimization across CPUs, GPUs, memory bandwidth and networks.

Microsoft can learn from both first-party workloads such as Microsoft 365 and GitHub and third-party customer workloads, Parikh says. Its teams tune the hardware, software and application stack to get more use from the fleet. Berman presses again on whether there will be enough electricity over the next 12 to 24 months.

Parikh says physical capacity must be aligned with demand and that surpluses or shortages can occur. He acknowledges Nadella’s statement that power is short and says teams are seeking more power while optimizing existing infrastructure. He gives no specific forecast or figure for powered versus idle GPUs.

Parikh argues that improving utilization of already-powered GPUs can be faster and cheaper than waiting for new data centers. Because Microsoft operates a large fleet, even a fraction of a percentage point of improvement can release useful capacity, he says. This is a complement to building more infrastructure, which takes time.

On model efficiency, Parikh expects enterprises to use a mix: larger, slower and more expensive models for some tasks and smaller, targeted models for others. Teams may start with a large model and then fine-tune or distill a smaller one, while optimizing frameworks and runtimes. He says cost and latency improvements must remain within acceptable accuracy, security and safety tolerances.

Parikh describes Microsoft’s model router as a way for applications to select among models according to preferences such as cost, speed and quality. He presents it as a platform capability to reduce the burden on enterprises choosing models for different workloads. The interview does not independently test its routing performance.

He also argues that enterprise data and context can make applications more useful. Organizations can bring customer, supply-chain or marketing data alongside a model and adapt it through fine-tuning or reinforcement learning. He expects these combinations to support more complex workflows and potentially better returns, a claim dependent on the deployment.

Berman asks how customers should choose between open and closed models, including models from Chinese developers. Parikh says Microsoft wants to offer choice, claiming that Foundry listed 11,000 models at the time. He advises beginning with the customer’s objectives and experience, then using examples and proof points to find a suitable approach rather than treating the decision as binary.

Berman asks whether a single omni-model will eventually handle everything. Parikh does not rule it out forever but does not expect one model to be best at every task soon. He anticipates continued demand for different sizes and customized models across fields such as healthcare, finance, climate and industrial processes.

On OpenAI, Parikh says Microsoft both incorporates technology from the partnership into its products and develops its own models through Microsoft AI. Product work can involve post-training and combining models, while the MAI team pursues Microsoft’s own model capabilities. He presents these as parallel efforts.

Asked how CoreAI works with Mustafa Suleyman’s MAI team, Parikh describes collaboration on infrastructure, algorithms, evaluation, safety and trust. Work is divided according to the teams best placed to do it; for example, software-development models may be closely tied to GitHub Copilot and developer-tool teams.

Berman raises model-weight theft, poisoning and AI-enabled hacking, and asks which attack vector worries Parikh most. Parikh says unknown threats are especially concerning: known threats can have mitigations, while the technology’s speed and continuing advances may challenge established security practices.

Parikh says security is built into Microsoft’s agent platform. As he describes it, an agent receives an identity tracked in Microsoft Entra, with policies and access controls, and can be deactivated if it misbehaves. He also describes tracing tool calls, data access and human approvals for an agent’s actions; these are product claims rather than independently verified guarantees.

For his contrarian view, Parikh rejects lines of AI-generated code as a useful primary metric. He prefers to ask what an organization can do now that it could not do before. He points to financial-services teams with years of technical debt: if AI helps them modernize code and infrastructure and improve security, those outcomes matter more than the volume of generated lines.