Apple Intelligence at WWDC 2024: Siri, Privacy and ChatGPT

At WWDC in June 2024, Apple introduced Apple Intelligence and described a collection of generative AI features for its devices. I am watching the presentation and giving my reactions to what Apple announced. The examples are demonstrations and future plans from that event, not evidence that every feature was already available or worked as shown.

Apple emphasizes deep integration, personal context, and privacy. I think integration is its biggest advantage: Apple controls the hardware and operating system and can connect an assistant to the apps and information people use every day. Emails, photos, messages, and routines could provide useful context if handled responsibly.

Apple contrasts its personal intelligence approach with chat tools that have broad world knowledge but little knowledge of the user. I interpret the later OpenAI partnership as a way to add those broader capabilities while Apple develops and controls the surrounding experience. Apple announced iPhone, iPad, and Mac support; I wonder about Vision Pro, which the presentation did not mention here.

Apple describes language and image generation as well as the ability to take actions across apps. Language and image generation are familiar from other products, but I see action across the operating system as the most important differentiator. Apple has its own models and presents ChatGPT as an external integration; Apple also says it intends to support other models in the future.

The presentation shows prioritized notifications and systemwide writing tools that can rewrite, proofread, or summarize text in Apple and third-party apps. I do not find these functions inherently novel compared with existing tools, but putting them directly where people work could make them much more convenient.

Apple also shows personalized generated images using people in a photo library. This is not a feature I personally care about, though I can imagine other people enjoying it. The feature I want most is the ability to ask Siri to do something regardless of which app holds the necessary information.

The examples include finding files someone shared, locating particular photos, and playing a podcast sent by a spouse. These sound simple but could save time. I recently spent about half an hour searching for a person I had spoken with months earlier; a system that can find relevant information across a device could help with that sort of task.

Apple describes drawing on information from apps and the screen to answer questions in context. In one hypothetical example, the system considers a rescheduled meeting, a child’s performance, and traffic to judge whether the user can arrive on time. That kind of assistance requires sensitive personal information, which makes the privacy architecture central to its usefulness.

I prefer local models because I do not want all my personal activity sent to a cloud service for storage and analysis. Apple says its architecture uses on-device processing when possible, with larger server models for more complex tasks. The distinction matters: Apple Intelligence was not presented as a system where every request necessarily runs locally.

Apple says its on-device models and semantic index are integrated into the system and supported by its advanced chips, including A17 Pro and M-series chips in the 2024 announcement. I expect Apple to focus its public messaging on user-facing capabilities rather than model size and technical benchmarks, because most consumers care more about what a feature does.

For more demanding requests, Apple introduced Private Cloud Compute, using larger models on Apple silicon servers. Apple says personal data is used only to fulfill the request and is not retained or made accessible to Apple. Those are Apple’s privacy claims about its architecture, not something I independently tested in this video.

Apple also says independent researchers can inspect server software and that devices will not send requests to servers whose software has not been publicly logged for inspection. I find the idea interesting, though at the time of the presentation I am not sure exactly how verification will work. Apple published a separate technical explanation of the design.

Siri gets a new visual treatment and is demonstrated understanding a speaker who corrects a place name mid-request. Apple also shows typing to Siri by double-tapping near the bottom of the screen, with the option to switch between text and voice. Typing is useful in public, even though I think voice is central to the long-term experience.

Another announced feature lets users respond to certain Siri interactions through AirPods by nodding or shaking their head. I think this understated feature hints at a more ambient assistant: a device in your ear could let you interact without taking out a phone or speaking every response aloud.

I am surprised that the presentation does not emphasize a more natural Siri voice, especially after recent voice demonstrations from other AI companies. I imagine voice and gesture becoming increasingly important, and even speculate about future cameras in earbuds that could detect more gestures. That is my idea, not an announced AirPods capability.

Apple says Siri will be able to perform hundreds of actions in and across apps. One example finds a photo of a person wearing a particular coat, enhances it, and adds it to a note. The individual steps look basic, but coordinating them through a single request could be valuable.

Apple attributes these actions to enhancements to App Intents, a framework that lets developers expose actions for Siri, shortcuts, and other system experiences. Third-party developers can define actions in their own apps. I suspect outside AI assistants will not get equivalent unrestricted control, though that is my interpretation, not an announced restriction.

Imagine an app exposing a ride-booking action so I can ask Siri to arrange a trip to the airport. That is a hypothetical use case, but it shows how voice could gradually reduce the need to navigate screens. Touch and silent interaction will still matter; I expect a gradual shift rather than an immediate replacement.

I also imagine a more proactive assistant that knows about a flight and asks whether to book a ride, with a nod or a shake providing the answer. Apple’s examples here generally start with a user request, so this proactive behavior is my prediction, not something demonstrated in that sequence.

Apple describes a semantic index spanning photos, calendar events, files, messages, and emails. It shows how Siri might retrieve a recommendation without remembering which app contained it or extract a driver’s license number from a photo for a form. I would like Android to offer similar app actions while letting users choose among AI providers.

In Apple’s airport example, Siri checks a mother’s flight information from email against live flight status, finds a lunch plan in a text message, and estimates travel time to the restaurant. The intended benefit is completing a multi-step task without manually switching between Mail, Messages, and Maps. This is a staged demonstration of proposed functionality.

The writing tools show different rewrites, tones, proofreading suggestions, and explanations. Comparable features already exist in products such as Grammarly, but native integration can make them easier to use. I am more impressed by Smart Reply, which identifies questions in an invitation, offers selectable answers, and uses those choices to prepare a response.

Mail also shows summaries in the inbox and surfaces messages deemed urgent, such as a boarding pass or a dinner invitation. I do not think summarization or prioritization is unprecedented; what matters is whether these capabilities work reliably in the application people already use.

Apple’s Genmoji lets users create an emoji-like image from a description on device. I expect it to attract misuse, as earlier generative sticker features did. Apple will need to decide how its safeguards handle potentially offensive requests; my prediction is not a test of its actual safeguards.

Image Playground is Apple’s image-generation interface. It offers concepts such as themes, costumes, and places, along with descriptions and styles such as animation, sketch, and illustration. Apple presents generation as occurring on device. The interface encourages choosing and combining options instead of requiring users to perfect a long prompt.

The presentation does not show photorealistic output in these examples. Apple also describes a standalone Image Playground app and an API for developers. Image Wand in Notes can use a circled sketch and surrounding words to generate an image, or suggest one from nearby context when the user circles an empty area.

Apple then introduces ChatGPT integration for requests involving broad knowledge or specialized expertise. In the Siri demonstration, the system suggests using ChatGPT for meal ideas, asks permission before sharing the question, and returns the answer in the same interface. Apple says it is starting with OpenAI’s GPT-4o and plans support for other models later.

I am surprised Apple did not highlight the kind of natural voice interaction that OpenAI had recently demonstrated. I suspect Apple wants Siri to retain its own voice, but that is my interpretation. Apple also shows asking about a photo, with permission before it is shared, and describes using ChatGPT with documents and presentations.

In Writing Tools, ChatGPT’s Compose feature is shown generating a bedtime story and accompanying images. Apple said users could access ChatGPT for free without an account, while subscribers could connect their accounts for paid features. The company said users would be asked before information was shared; account-specific privacy terms and service availability may differ.

I wonder about the economics of the arrangement and whether distribution could eventually lead model providers to pay Apple. That is a prediction, not a disclosed agreement. I was disappointed by the OpenAI partnership because I wanted stronger competition, though separating ChatGPT from Apple’s deeper personal context and app actions is a strategically sensible integration in my view.

Apple’s statement that it intends to add other models matters: it keeps the external model layer open to alternatives. Overall, I find much of the on-device approach compelling, even if some image features feel gimmicky. The most promising part is the possibility that Siri can actually carry out actions across apps on a user’s behalf.