Microsoft Build 2024: Copilot+ PCs, Phi-3 and AI Demos

Microsoft Build 2024 has begun, and I am watching selected highlights from the first day’s lengthy keynote. Satya Nadella and Sam Altman discuss AI platforms, local models and developer tools. This is an account of the May 2024 announcements and my reactions at the time.

I praise Nadella’s leadership and argue that Microsoft is investing across cloud AI, model partners, hardware and data centers. My belief that the strategy will pay off is an opinion, not an established result.

In the keynote Nadella describes advances in model architecture, data and compute, and envisions a multimodal interface that works with text, speech, images and video. He also discusses memory and reasoning as ways to support complex tasks.

Nadella introduces Microsoft Copilot and the Copilot stack, then describes Copilot+ PCs as a new category. I focus on local AI acceleration and Microsoft’s simultaneous investments in models, software and chips.

Microsoft presents the Windows Copilot Runtime and a library of local APIs for application developers. Nadella compares its importance for AI experiences with Win32’s role in graphical applications; that comparison expresses Microsoft’s ambition, not a measured outcome.

I welcome Microsoft’s emphasis on smaller models and on-device AI alongside hosted models. Nadella says developers can call models through APIs and highlights Phi Silica, intended to run locally on the neural processing units in Copilot+ PCs.

Microsoft also describes incorporating retrieval-augmented generation and semantic search over on-device data. I explain the proposed Recall experience as a way to find past PC activity and raise concerns about privacy and deletion controls.

I mention Microsoft’s relationship with NVIDIA, including both hardware and software for training and fine-tuning. I see Microsoft placing bets across the AI stack; my positive assessment is commentary, not investment advice.

Nadella calls OpenAI Microsoft’s most strategic partnership. I speculate about how that relationship might feel to OpenAI while Microsoft also works with other model providers. The video does not establish the companies’ private views or verify the ownership figure I mention.

A staged Copilot demonstration shows a Minecraft player asking for help crafting a sword. The assistant describes what it sees, asks the player to open the inventory and recommends gathering materials; a zombie encounter prompts advice on finding shelter.

The demonstration appears responsive to me, but I explicitly assume that the clip runs at normal speed without edits. I also wonder whether the voice gains an echo when the player enters a cave; the video does not establish either claim.

Another demonstration shows a shopping assistant evaluating footwear for a cold overnight hike. The presenter interrupts the voice response and asks for a recommendation, and the assistant says it will add boots to a cart.

I interpret the shopping demo as an example of an assistant acting through a computer interface and imagine broader use in everyday work. A prepared demonstration does not establish reliability across websites or prove that all users already have these features.

The presenter then switches to Spanish and the assistant follows. I find the language switching notable, although it is one selected demonstration rather than a comprehensive test of multilingual performance.

The keynote displays a range of models from providers including Mistral, Databricks, Cohere, Meta and Microsoft. I see the selection as evidence that Microsoft is supporting several approaches rather than relying on one model provider.

Microsoft highlights an expanded Hugging Face partnership, including additional models and integration with Azure AI tooling. The offerings and their availability described here reflect the 2024 event.

I examine Microsoft’s Phi-3 quality-versus-size chart and say the smaller models look promising. The displayed comparison is Microsoft’s own benchmark presentation, not an independent or universal ranking of model quality.

Nadella introduces Phi-3 Vision as a 4.2-billion-parameter multimodal model for reasoning about images and answering questions, alongside 7-billion-parameter Phi-3 Small and 14-billion-parameter Phi-3 Medium models.

Microsoft describes applications spanning web, mobile, Windows and edge environments, using local hardware where available and cloud resources where needed. These are product capabilities and plans as presented at Build, not a guarantee for every device.

I turn to developer tools and mention the announced Microsoft partnership with Devin alongside GitHub Copilot Workspace. I interpret this as Microsoft supporting multiple approaches even when products appear to compete.

Nadella cites 1.8 million GitHub Copilot subscribers across 50,000 organizations. I say coding assistants have changed my own workflow: I write less code from scratch and use the tools to speed up development.

The keynote also describes Copilot in tools such as Teams, Loop and Planner, with proposed uses including meeting agendas, notes, action items and project coordination. I joke about the implications for project managers; the examples are not evidence that these tasks are fully automated.

A Microsoft executive brings Sam Altman onstage. Their conversation begins with Altman’s view that developers have adopted and adapted OpenAI’s models quickly, building uses the company itself did not anticipate.

Altman argues that an API allows many developers to incorporate model capabilities into products and services. He describes the platform’s purpose as making that integration accessible beyond a single company’s own applications.

Asked what to expect in the coming months, Altman predicts models will become generally more capable, while improvements in speed, cost and modalities will also matter. His comments concern future models generally; the clip does not announce or demonstrate GPT-5.

Altman cites GPT-4o’s price and speed relative to its predecessor at the time and says voice mode has surprised him with its usefulness. These are 2024 product comparisons and opinions, not a guide to current offerings or prices.

The discussion turns to safe deployment. Altman says OpenAI and Microsoft invested substantial work in safety systems, policy, monitoring and research to make models usable, while acknowledging that GPT-4 remained imperfect.

He expects more powerful systems to introduce additional safety questions and require further research. Altman presents safety work as a condition for deployment; the interview does not demonstrate that any model is safe for every use.

I close by saying Windows looks more compelling to me and predicting that Apple will also make major AI announcements for its operating systems. That final expectation is my May 2024 forecast, not a claim about what happened later.