Ten Thousand Certified Engineers in a Year: Google Cloud's Order of Operations for Enterprise AI

Sarah Guo introduces Kawal Gandhi as lead for generative AI in the Office of the CTO at Google Cloud, after years on search and ads. Gandhi says the path was customer demand. Working with advertisers on shopping and travel, especially commercial queries, he kept hearing requests for storage, compute, and infrastructure to make ads perform better. That pulled him into Cloud, then into analytics, machine-learning pipelines, Document AI, and conversational AI. Generative AI is the latest turn on the same question: how do you make the customer experience better with the information they already have.

Asked for the origin story of AI inside Google Cloud, he does not start with TPUs. Customers leaned toward Google, in his telling, because they wanted lower latency, faster response, and a better experience with their data. From the beginning he saw machine learning and AI as the differentiator. The Cloud journey was data, AI, storage, privacy, and security: take the deep technology Google used internally and offer it in market. Internal tools and frameworks took years to make rich enough for both regulated and unregulated customers, and to let those customers bring current IT investments onto the platform. Those scars became product.

Google's pattern, Elad Gil notes, is to dogfood its own APIs and then launch them as a service. Gandhi says the first generative applications were in Workspace: better document summaries, more personalized email, suggestions in spreadsheets. The team treats them as experiments. As research leans in, features get added to the platform, efficiency and productivity numbers get checked internally, and only then does the company open them up. The suite now lives under the name Duet AI.

Uptake, he says, is across the board, not a single hero feature. Documents get generation and summarization. Slides now suggest images and create new ones from a prompt, work that used to mean briefing a studio or an agency, waiting on a calendar, and iterating a first draft. Email generation has been especially helpful for productivity, for consumers and enterprises alike. He flags security even though the company does not talk about it much: how content is sent, how links are used, taken seriously.

Gil has a personal stake. He spends maybe four hours of a workday just keeping up with inbox, and wants to know how far email generation can go. Gandhi refuses to predict; technology and models have a way of surprising you, and he would rather talk again in six months. What he will say is a family story. His mother speaks Hindi. He used to translate mail by hand. Now he can set Gmail so that when she opens the message it is already in Hindi. Features like that, he says, just bridge gaps for a lot of users, inside the company and outside it.

Google Cloud's stack, as Gandhi lays it out, starts with AI infrastructure. Customers train on first-party models or bring their own. One level up is Vertex AI and what he calls the model garden, a shelf of capabilities you can pick from. Domain models such as Med-PaLM 2 and Sec-PaLM are still early. You can chat with them and get validation: notes a nurse or doctor writes can be checked and dropped into the right format to save. Foundation models, customer-trained models, and open source are all landing on the platform. He is excited to see Llama, Stable Diffusion, and other variants come in. The differentiators he is willing to bet the next few quarters on are simpler: data stays secure, fine-tuning on the platform is straightforward, and model operations, drift monitoring, tooling, and safety are easy.

Gil cites Ankur Goyal of Braintrust: large enterprises often jump straight into fine-tuning, then wake up and prove the idea out on a GPT-4 or Bard API, iterate until it works, and only later consider training their own model. Gandhi says history always looks like that. You get an API, you build a prototype, excitement explodes. The serious thinking starts when you deploy, manage, monitor, and chain different models so responses actually answer for a capability. He wants to decouple that feeling from hype. It is real excitement. Engineers, and he has been one, want to grab something without restrictions and show the art of the possible. From those experiments, over the next year, he expects differentiated output inside the enterprise.

On when a customer should train their own model or use a domain one, he gets responsible fast. Guardrails, use cases, and the cost you are willing to put behind a continuous learning process all matter, and so does organizational maturity. Do they have a team that understands how models are built, tuned, and brought forward? Investment is allowed; where you sit in the cycle is what you cannot fake. Google Cloud takes that conversation to the board, into strategy, and into cultural transformation. It is not a one-dimension technology decision.

Guo asks how use cases break out: experiments, internal tools such as customer success and ops, or generative features in external products. Gandhi's sequence is strict. People inside an enterprise want to be creative, but they start with efficiency gains in workflows so they can invest back into the group, then productivity, then creativity. Gil's customer-success example is the right first move. Make support better, and do not drop CSAT. That is the KPI. If it holds, you go into productivity: promotions, next-best recommendations. Trust increases. He adds trust as its own dimension. The world he is looking at is systems of intelligence, or agents, in which trust goes up until you can really free human capital to be creative.

The whole industry, in his view, is on that trust cycle: do the models do what we think, do they go off, do they hallucinate. Progress should be gradual. Getting super excited and then discovering the KPIs do not work is the failure mode. What predicts success is less the technology than where a company places itself in the industry: vision-quadrant first-of-a-kind versus fast follower, how much they will invest, how much they believe, whether they will keep investing. He has spoken with a lot of customers, even governments. He has not seen interest this high. They have just scratched the surface.

The first wave, horizontal across regulated and unregulated industries, was sales and marketing. Every department had bottlenecks in content creation and distribution; efficiency and creative gains showed up immediately and shortened the workflow. Customer care is the second cut, even in B2B, because a supplier still wants a better experience with the provider. The third is internal experience. Employees start asking why looking up HR benefits has to be so hard, and the same technology turns inward.

Core use is still language, text, and conversation. Turning content into a script, running translations, and hitting distribution used to be really hard three or four years ago and now sounds easy. Multimodal is early: text, then images, then images and text with audio, then those media combined. Trust has to rise before that stack comes forward. Two problems sit in front of scale. One is safety around identification, deepfakes, and voices used out of band, especially internally. The other is cost: how you store, retrieve, and scale the data you just generated. Gaming is the industry actually scaling multimodality on Google Cloud today. He asks when shopping sites will stop showing a progress bar on video. That is still centralized in a few experimenting organizations.

The expensive part of the investment cycle is becoming cheap: models, platform availability, usage. A cost curve going down is what lets more people adopt. Training classes are one sign. In a year, more than 10,000 people have been certified on GCP generative AI globally, ready to write code and take advantage of coding capabilities, including system migration. The surprise, for Gandhi personally, is engineering. He thought engineers were already productive. Design and UX discussion used to take a long time; now a bot in the room offers an approach or writes the code. Resource shortages that bottlenecked projects are loosening. The code and proposals those organizations generate are IP. Gil sees the same pattern in startups: technologists building for themselves, then mid-market tech companies whose developer-CEOs can just decide. Gandhi's version of the stack is a platform with models, a coding model, capital to experiment, then Google Cloud for secure scale and cost.

Where does the bot live in a Google product team, Gil asks: chat, docs, the IDE, version control? Today, Gandhi says, it lives in docs. Any project, hypothesis, or experiment starts with a document. As platforms mature, docs will make suggestions: here is output based on what you specced, and you can ignore it. It could be a notebook. It could be code in an IDE with the API key filled in. Scale is what scares him. Guardrails have to exist: who in the organization is entitled to do this, is logging on, who accessed what and when. If you need a monitoring tool for model drift and it knows the schema, why should an engineer spend four weeks writing it?

Asked for anti-patterns, Gandhi will not call them mistakes. The team thinks deeply about data customers bring onto the platform and runs drills so that early trained models do not leak, and so adapter models and certification stay in the customer's tenant. From the beginning: all of their data, all of their models, all of their weights are their IP, and Google Cloud's job is to safeguard that. Moving too fast without checkpoints creates rollbacks that are expensive, because then you have to educate the industry all over again. Guo notes Google has always been strong on customer data security. On which sensitive vertical moves first, healthcare versus education versus financial services, he sees early adoption in all of them, plus the horizontal of sales and marketing.

Google pioneered custom silicon with TPUs internally seven or eight years ago. Gandhi's platform answer is optionality, not a religious choice. The job is to abstract infrastructure the way old Windows and Java apps eventually did, so teams stop thinking about where something is trained and deployed and get fungibility and scale. The real tradeoff today is how many people know how to use each stack. GPUs have been external for years, so a lot of people are trained; people who are truly comfortable on TPUs are often inside Google. Spreading that experience is a future investment. He is more focused on inference. Character.AI is thinking about scaling out; so are examples around Quora's Poe. Google Cloud is investing heavily in TPU v5e to push the cost curve the other way, including for education sites that do not have big-lab luxury when multimodal arrives and an app goes from five users to 100.

The NVIDIA GPU shortage comes up in every customer conversation, then the conversation moves. Once you look at the application they are trying to solve, you get to data security, the right teams having access, data residency, questions other clouds cannot answer easily. Training demand is concentrated; the world will then focus on the last mile of deploying inside the organization. At that point, he says, people stop talking about chip shortage. Customers should have availability and optionality: if Llama is trained, put it in the model garden so an application can use it without new scaffolding. Starting customer-first beats handing over another set of models to wrap. Models are 50 to 60 percent of the work; leveraging current investment is 30 to 40 percent; upkeep and operations still sit after that.

What he is most excited about is working with customers on next-generation multimodal user experience and the first use cases worth partnering on deeply. Second is shared data for training: datasets the platform can offer, and synthetic data that could recreate simulation modes for regulated industries missing pieces of a picture. Labeling today runs through a partner ecosystem integrated into the platform, including RLHF services. The non-negotiable is that whatever a partner gets from the customer stays in that customer's project, not reused on another. He would like to come back in six months. Guo says she will hold him to better Gmail suggestions, which is as concrete a footnote as enterprise AI gets: start by giving back some of those four inbox hours.