A Million DALL-E Images, Distilled into Twelve Minutes of Film

Nathan Labenz opens with old colleagues. Stephen Parker and Josh Rubin lead creative at Waymark and made *The Frost*, a short assembled entirely from DALL-E 2 stills. From 2017 to 2021 the company had built what Labenz calls the easiest video-creation app on the market, with template quality as the standout. Users still wanted something else: an app that would make the content *for* them. Labenz had been hunting for AI that actually worked. That changed when OpenAI released GPT-3, and in September 2021, after the team successfully fine-tuned the Curie model, he became convinced generative AI was the answer.

He used the CEO’s prerogative to pause almost everything else, board meetings included, run a generative-AI crash course, and swing the product roadmap around the new stack. Some on the team thought he had lost it. Parker and Rubin came along and went deep on AI art while Labenz split his own “10,000 hours” across language, vision, and speech. As an OpenAI innovation-partner customer, Waymark got first-wave DALL-E 2 access in early 2022. Labenz says Parker has personally generated over a million images with it since then.

The result, he insists, is not a proof of concept. It is a 12-minute film with a coherent narrative and a consistent look. Only seven people are credited: Parker and Rubin from Waymark, Tommy Herman, Zach Poly, and Lexi Deets, plus collaborators Matt Sessions and Robert McFalls. *MIT Technology Review* wrote the project up. A trailer for *The Frost 2* is already out, made with an entirely new generation of tools. Labenz’s ask is simple: watch the film as a film.

Parker describes the pipeline in one breath: prompt for stills, curate, take them into After Effects, cut them up, puppet them, run images through D-ID, and otherwise manufacture video from photographs, then assemble a short. Rubin, the director, is blunter about the bet. Plenty of people already make image montages set to music, some of them better than this team. *The Frost* was a “happy wonderful accident” born of a narrower question: can you make a *narrative* out of AI-generated imagery? They treated text-to-image as the beginning of AI cinema. Three and a half months later they had roughly thirteen minutes and, “for better or worse,” they felt they had hit the goal. The standard was not “look what AI spit out.” It was something the filmmakers would be proud of and that people would watch on the merits.

The process inverted ordinary development. Normally you start with an idea, then a script, then storyboards, then — if you have the money — locations and a shoot. DALL-E gave them a starting world instead. Parker had gone “gung-ho” on photoreal cinematic stills; Rubin saw an early Frost series of twenty or thirty images, faces and mountains and close-ups of gear, and the story arrived to tie the world together. That is not how original films usually begin unless you are already stealing from a pile of references and then have to make them your own.

Parker’s prompt discipline starts before the keyboard. He asks people to imagine how the image would have been captioned in the training set. Someone who types “gray alien skull” into a generator gets an illustration or a painting. His rewrite is to picture the thing as real: it would sit in the Museum of Natural History in New York, accessioned around 1946, photographed for a 1960s *National Geographic* collection. That contextual frame tells the model what you are looking for and where it might have been found. Applied to *The Frost*, the same idea becomes a reusable wrapper. You ask for portraits from a film. You give the film an imaginary name, *Tundar*, a tweak on “tundra” pointed enough to set the mood and obscure enough not to collide with something already in the data. An Oxford comma, then the subject — people climbing a snowy mountainside. Another comma, then “filmed by” a famous Hollywood cinematographer or director. Extra consistency tags at the end, in their case blue and gray. You are writing the imaginary IMDb still caption, the blog-post still, the film-site grab. Each clause narrows where in the dataset the model should look.

Compositional continuity was the unglamorous fight. Shot/reverse-shot for a conversation was the hardest cinematic grammar to prompt. Once they had master shots, Rubin’s film knowledge took over: you can only live on wides for so long, you need a punch-in, a profile, a way into the world audiences already know how to watch. Results were mixed. Sometimes they got exactly the angle; sometimes they let DALL-E take the reins. Parker’s team mined raw materials; Rubin built story from them and sent notes — this scene needs more reverses — which sent the team back into the generator. He keeps saying “tools”; Parker keeps correcting toward artist. The model is a cinematographer you do not fully control. In a typical production you own the set, the talent, and the lines. Here the substrate is active, and you manage what it offers that day.

The outdoor snow saved them. Cutting from shot to shot, tents and mountains do not match, and the brain still buys it because there is snow and there are peaks. Indoors, that generosity disappears. Character faces should have been worse. Every DALL-E 2 frame was generated from scratch — no reference image, unlike later Midjourney blends and variations. Parker spent a long stretch trying to talk Rubin into a group story so no single face had to hold. Rubin would not move on Dr. Ulrich, who had to carry time on screen, including a United Nations speech to save the world. The workaround was an archetype the dataset is drowning in: gray-haired, glasses, white lab coat, “wily” mad scientist. The faces still change. Rubin admits some shots look like Santa Claus, maybe thirty pounds heavier, others trimmer, features off. A consistent voice for Ulrich plus the score made him buy it. Labenz had watched the film several times and had not noticed. Rubin’s explanation is ordinary storytelling: people forgive boom mics and swapped extras on Reddit boards because they came to feel something. Human priors also help. Viewers are not primed for a person to become a different person between cuts, so the mismatch has to get loud before it registers.

The cutting-room difference is the part Rubin calls revolutionary. In a traditional shoot, even animation, going back is a luxury; once it is in the can you live with it. Mid-edit he wanted a hand clutching a rope, to raise the tension after a slip off the mountain. The team returned an image in about twenty minutes. Into the timeline: fingers wrong. Outpaint it twice, pull back. “Ah, that’s it.” That, he says, is how the polish phase actually happened. Parker calls their board an *active* storyboard: a shared whiteboard site, a row per scene, images in order — and every still is the real asset, not a concept sketch, swapped or inserted for the whole project. The trade, Parker is careful to say, is control. You can reshoot forever. You still do not own the cinematographer.

Most frames still got local surgery: DALL-E started small, they scaled up, then corrected an eye, a hand, a steel pallet it had dropped into the background for no reason. The object they could not prompt into every shot was a transponder that had to travel up the mountain with the climbers. Parker borrows “MacGuffin” a little loosely: not the plot-object people hunt, but an idea of consistency, designed through many prop iterations, then composited in from a few angles. Broader composition is harder than a rope. You cannot mask the whole layout. Inpainting and collage from several DALL-E stills helped, and knowing form tells you what to erase so the model has space to paint back. Dataset gravity still wins. Rubin wanted fires in daytime overhead camp shots; those photographs barely exist, so night fire is easy and noon fire is not. A blue-graded world would not take an electric-red or purple scarf, because graded cinema does not keep costume that saturated. Parker’s picture of latent space is a library of what humanity has already found. Generation interpolates between those coordinates — avocados, chairs, avocado-ish pillows, avocado-adjacent chair designs, hence the avocado chair. It does not teleport to a point nobody has visited. Open prompts blend more freely; a cinematic-8K-portrait wrapper triangulates into a tighter neighborhood. DALL-E, he adds, photobombs the Mona Lisa constantly, as background extra, photoreal portrait, drawing, sculpture, a ghost you only notice after a hundred thousand images. It also invents clothes that do not care about our categories: a button-down that becomes a poncho below the top button.

A reviewer called the vibe “grotesque and unsettling,” which happens to fit the story. Labenz guessed the stills were photoreal and the motion layer was the uncanny part. Rubin does not argue. DALL-E’s people are missing fingers, incomplete feet, joints that do not bend like joints. Human climbing had to look rudimentary. Animator Matt Sessions, a longtime Waymark collaborator, puppeteered in After Effects, dragging an arm, a knee, up and down, slowly. The reference that made the sluggishness feel like a choice was the blizzard sequence in Kurosawa’s *Dreams*, then getting a Criterion remaster. Early motion tests read to Rubin like a graphic novel that only needed to move a little, not 24-frame realism. For scale — a whole party moving up a mountain without animating every limb — they sprinkled Mixamo 3D figures through the frame for coarser, more fluid crowd motion.

Same-tech romantic comedy? Parker’s first reaction is that he cannot think of anything harder: subtle faces, constant reverses. The genre of *The Frost* — disaster, expedition, world-building at a distance — was doing them a favor. Mining emotion was already one of the biggest fights. For the avalanche they prompted utter terror, panic, “about to die,” and often got a romantic gaze at the mountain. They tweaked eyebrows and mouths by hand, inpainted smiles and frowns, and only at the tail of the project started using Photoshop’s new AI emotion controls plus After Effects. Parker later argues that a lot of the missing fear is RLHF sanding danger out of the data — understandable if you do not want fake disasters, lethal if you are making drama, action, or sci-fi and need people who look scared, angry, or in love.

They began *The Frost* around December or January thinking text-to-video was two years out. By the time they were cutting a *Frost 2* trailer, Runway Gen-2 was giving 18-second clips instead of four — enough, Rubin says, to tell a shot or even hold a scene. The image is no longer a still you have to breathe life into; the life is already there, and the new bugs are different. A head starts to resemble a tomato; you throw the take out and generate again. Inpainting a video is more tax than prompting another clip, so flexibility drops. The emerging flow is often: make the still in DALL-E or Midjourney, then ask Runway to move it. Three knobs exist now — pure text, image plus a motion prompt, or image alone. Parker likes the idea of *The Frost* as IP, a storyline people can follow while the tools morph underneath, because the field otherwise gives you one look at a trick and then ten new ones the same day. Voice crossed a threshold too. For part one they auditioned a pile of AI voices and dumped them as robotic. On part two, in a dramatic register, Rubin says they were blown away. Labenz flags ElevenLabs’ pro clones and PlayHT’s new emotion prompts — anger versus surprise on the same line.

The conversation lands in the 2023 Hollywood writers’ and actors’ strikes. Labenz quotes Waymark’s then-CEO, Alex: they would never have made this the traditional way, because they did not have that budget and would never have dreamed it. Production methods change; the set of things that can exist changes too. Parker is careful about the rest. They are a technology company whose mission is letting people make their own commercials. *The Frost* was a luxurious byproduct of learning the tools on behalf of customers, done for fun and not for profit. He thinks there is still room for a lot of people. Smaller groups will get to pursue films they could not afford. Some jobs will go away, which is what technology does. He does not buy a blanket death of the artist: if you have one modality and refuse to move, you will have trouble; if you put the tools in the workflow, new paradigms show up. Fine-tuning a model on your own work, then exploring inside that territory, is the example he wishes more individual artists would try.

Prompting “in the style of” a named artist, he notes, is already a mélange — sometimes credited, often not, and in film it collapses screenwriter, grip, and unnamed armies into one famous name. The economic idea he is trying to advocate in public is different: directors author a definitive fine-tune of their thumbprint, license it, and let other people mix those authored models. That, he thinks, could be a healthier market than scraping everyone into an amalgam. He also wants a quieter cultural permission. Running his own numbers, he was a couple of months from a million images generated that year, and he publishes a small fraction. He has private worlds he walks alone. A lot of people will, and he would like society to understand that without treating it as a crisis.

Rubin has Hollywood friends and family asking, nervously, what this is. His answer is not that it is easy. Directing this picture was exceptionally hard, “maybe even harder than a traditional animation,” because you are in a room with an unknown artist who might give you exactly what you asked for, or random wonderfulness, or the grotesque image other people talk about. For a film he would put his name on, there still has to be “a big 500-foot vision” belonging to a human. You cannot ask ChatGPT for a screenplay, feed that into another model, and receive a movie. There is tweaking, argument, critical thought. Ten or twenty years from now a typed prompt might return a finished picture. Right now the storyteller is still the person.