Don't Do AI for AI's Sake: Marily Nika's Starter Guide to AI Product Management

Marily Nika's tip for staying informed is newsletters, including some that are not AI-centric, since she believes everything will be AI by default and AI will be sprinkled into all technology coverage. On hype, she says writers fearing that ChatGPT will end online writing have it wrong, because it enhances work instead of stealing it. She also urges people to read beyond ChatGPT, since AI can do far more.

She uses ChatGPT in three ways. First, she asks it to rewrite her mission statements, which must be understood by everyone from leadership to junior staff and even a child, and she says the first try is often fantastic. Second, it generates user segments with motivations and pain points that her own mind would not reach, for example who would want a fitness band with no screen. Third, it suggests AI-enhanced ideas. In each case she brings her own idea first and does not make it do her job.

Her bigger claim is that PMs will become AI product managers, because products need personalised experiences and good recommendation systems, as with Netflix after you finish *White Lotus*. For her it means PMs must get comfortable partnering with research scientists who build models. Many PMs dislike the uncertainty of research, which is more "we'll try this and in a year shut it down if it fails" than the ship-and-launch rhythm they are used to.

She frames the product as the intersection of three things: desirable to users, viable as a business, and feasible for a research scientist technically. Lenny notes he had not considered that every cross-functional team might end up with a research scientist.

For PMs without a technical background, Nika says not to be overwhelmed: you will never need to train or code, and no-code approaches to training exist. Any product with data on user behaviour can be improved with something smarter, like personalisation, fraud detection, better accuracy in healthcare or better recommendations in shopping. She has met PMs who were not collecting any data yet, and for them setting up a dashboard is already a first step towards AI. A data-science intern can then show what is possible.

She warns about the "shiny object trap": make sure there is a real pain point before using AI. Her definition is that a generalist PM helps ship the right product, but an AI PM helps solve the right problem, and there must be a user and a pain behind it. Asked for signs AI is a bad fit, she says never use it for an MVP.

Instead, build a Figma prototype and fake what the AI would do, then show it to users to get buy-in. Training models on powerful machines can take weeks and eat data scientists' time. Use AI where you already have data, or data from an adjacent product, to build something meaningful such as recommendation or automation.

How much data you need depends on the task. A classifier for dog photos might work with 15 or 20 labelled images, but speech recognisers and complex NLP need thousands. Some teams synthesise fake data to get started. Big companies, she says, should collect their own diverse data, because if everyone trains on the same purchased dataset, everyone's quality will be identical.

She also says that deciding when quality is good enough to launch is the PM's responsibility, for instance whether 70 or 80 percent accuracy is sufficient. That, she says, is what makes the role exciting, because it involves problems no one has tackled before.

To explain a model, Nika uses her three-year-old daughter. You show her a rhino and an elephant repeatedly until she can spot one on the street and say it looks like a rhino. A model is like that brain: it takes an input, such as an image, and says it is 70 percent sure it is a rhino. Lenny adds that inputs could also be text, voice or video.

Training means feeding the model thousands of labelled examples while it finds patterns. She says those patterns are not simple rules like "if it is gray then it is that". The model learns, in a way we do not fully understand, to identify things and outputs a probability. For speech, the output is a transcription of the audio.

Her most impactful project example is Google's AR glasses work, shown at Google I/O, where a person's speech in one language is transcribed, translated and displayed to the wearer in their own language. She says it unlocks borders of communication and shows the technology is no longer science fiction. It is a matter of connecting the pieces.

Asked whether GPT-style tools will make PMs unnecessary, Nika says absolutely not. They make everything better and free time for strategic work, for example by writing the repetitive parts shared across the many projects she runs. She thinks they will also unlock new areas of product management.

She wants PMs to be less intimidated by learning to code and training a small model. Even if no-code tools can do it, hands-on experience gives a different mindset and confidence. She compares it to her piano teacher making her learn classical music first: understanding the fundamentals helps you create later and helps you understand how your tools were built, instead of trusting them blindly.

For resources, she recommends Coursera courses, including a Stanford introduction to AI, for self-paced learners. Those who like learning with others might try Career Foundry, General Assembly or Coding Dojo, and she has given a Python talk at the latter. In her view it only takes a few weeks of passion to realise it is not intimidating.

To become a strong AI PM, she says to learn how AI product development differs from general product management, since you sometimes manage the problem more than the product. If your company has research scientists, shadow them for an hour a week. That gives context and shows the potential.

Nika lists the challenges of AI product work. The first is uncertainty: after training, results may not answer your hypothesis, and the PM, as captain of the ship, must keep the team motivated. The second is managing change from a leadership perspective, which can be tricky. The third is that good data is hard to get and may require creativity, even asking people on the street to contribute.

The fourth is career trajectory. PMs usually advance by launching, but research work launches less often, so she advises clarifying early with hiring managers what progress means. She encourages PMs to flex this zero-to-one muscle anyway.

On getting buy-in for uncertain ML bets, Nika points to adjacent products as a way to de-risk: show leadership that something that once seemed crazy worked and that the new proposal is similar. She adds a contingency plan with a rollback and the maximum negative impact. Trust builds over time, and in cultures that welcome failure it is easier.

Lenny suspects most ML investments fail, and wonders whether public models change that. She agrees and says research and product are less siloed than before, so companies are adding PMs to the layer between research and production. She suggests reading arXiv papers, and notes PMs are needed to find monetisation, as with ChatGPT's survey about what people would pay.

Nika's Maven course runs three weeks with nine workshops. The first week covers how the AI product lifecycle differs from regular products and how to come up with ideas, citing Steve Jobs's point that users don't know what they want until shown. Later parts cover productionising, partnering with research scientists, persuading them to trust you, and interview preparation. Students build an AI product end to end, and two students who paired up went on to raise funding.

One student group built a model that took X-ray images found online and flagged what might be wrong with the patient, and went on to a recommendation step, though she stresses they were not pretending to be doctors. They did not know how to code. She recommends Google's AutoML for training custom models with minimal effort, provided you have labelled images.

She describes a company that used drones to photograph wind turbines and AutoML to detect which needed maintenance, cutting three weeks of work to a few hours. She built the course as a product: she formed hypotheses about the audience, asked people what they wanted to learn, and shifted from software engineers to PMs. She found three weeks ideal and messaged every applicant personally.

She added a bonus section on ChatGPT after her December cohort asked how it was trained on day one. She urges listeners to try creating their own courses, since people may want to learn what you take for granted. Lenny adds that teaching crystallises your own understanding, and Nika agrees, citing uncomfortable questions from students.