Four of the Top Five Defense Primes Still Run Hardware Tests on PowerPoint

Four of the five largest U.S. defense primes manage hardware-test data on the same startup’s platform. The customer list also includes Anduril and the Corvette racing team, across aerospace, defense, robotics, and autonomy. The company is Nominal. On Sequoia Capital’s Training Data podcast, cofounders Cameron McCord and Jason Hoch sit down with partners Sonya Huang and Alfred Lin after a recent fundraise.

The topic sounds narrow: hardware testing. The claim underneath is not. Software spent two decades turning version control, CI/CD, and test automation into products that companies no longer have to invent. Hardware engineers working on software-defined systems, autonomy, and robotics still do not have that luxury. If AI is going to enter the physical world, McCord and Hoch argue, the first gate is here.

McCord frames a set of tailwinds Nominal did not create: compressed timelines, more money into hardware, and a rapid shift in how products are developed, especially in aerospace and defense. People are trying to field hardware faster than they ever have. Testing, in their view, is not a back-office chore in that race. It is where the new paradigm either holds or breaks.

Huang says her calendar now fills with hardware companies every week. A generation of founders trained at Anduril, SpaceX, and Tesla looks regalvanized. She asks McCord what is in the water. He gives a positive frame and a negative one. The positive: after two decades of turning the world into SaaS, people are reconciling with the fact that most of what they actually want to exist is physical, and they are excited to build real things again. SpaceX, Anduril, and Tesla also proved that betting hard on the tools and infrastructure of hardware development is a massive competitive advantage.

The negative frame is blunter. Hardware is hard. It is capital intensive. Bending metal, steel, and electronics is difficult, and that difficulty is itself a moat, which is why investors are interested. McCord’s GitHub vignette is the industry’s missing chapter. Pure software teams once kept versions on local machines, then centralized inside the company, then productized the whole stack until venture money created GitHub and the DevOps tools everyone now takes for granted. Software testing, he says, is a solved problem. Hundreds of thousands of hardware engineers at the frontier do not have the equivalent.

The status quo is, in his word, shocking. Test data still lives locally by default. Network storage is common; the cloud is not. An avionics engineer, a GNC engineer, and a thermal engineer each download a file to a laptop, run their own MATLAB or Python, and try to post the result back — often as a screenshot. “PDFs, PowerPoint engineering is still the bleeding edge for many of these companies.” In Nominal’s early days they talked about dragging the industry from 2003 to 2019. Hoch adds that today’s product has to get ten or eleven hard software problems right before a user’s ordinary week even works.

SpaceX is the exception that proves the scarcity. Unlike most players, it decided from the beginning that it had hired some of the most talented engineers on the planet and that commercial test-analysis software was not good enough. It built a proprietary internal platform. Studying that decision, Hoch says, they concluded it was a huge reason for SpaceX’s later speed. A thousand hardware companies starting this year and next cannot each rebuild that stack. That is Nominal’s original motivation. At the traditional primes the problem is different: a thirty-year specialist in one proprietary simulation technology, sitting apart from prototype and manufacturing, with no common data platform, while Anduril-class newcomers move faster.

The episode’s title is McCord’s joke from his Anduril years. Around 2018 and 2019 it was in vogue to simulate everything, as early in the design cycle as possible. He watched the pendulum swing that way, then back. The power, he says, is not choosing one side. It is blending model outputs with real-world telemetry and doing that continuously. “Physics gets a vote. It still gets a vote.” A simulation can be beautiful. A physical system will still have an opinion.

That is why Nominal used hardware testing as the narrow wedge. It is where software-defined hardware first touches reality, the tip of the spear for how software will hit physical AI. Spreading into simulation and design can come later. The advantage is in the merge. There is also a business reason: testing is where ROI shows up fast. Seconds saved compound into minutes and hours for a customer trying to field a product. Everyone who has lived a major hardware program also knows the other half — something hidden in the data that becomes an all-hands hunt and can halt the program. Nominal’s pitch is to surface those long-tail risks before they do.

Testing is iterative by definition, the classic independent-variable loop, which is the cycle Nominal wants to drive. The product vision follows the same logic. Validation written for a test should be the same logic that runs as an end-of-line quality checklist in manufacturing, then deploys to the edge on the machine itself. McCord describes the core product becoming “the authoring hub of all logic that governs the performance of physical systems,” version-controlled and deployable. For software listeners, Hoch translates the scale. A unit test is simple and deterministic. Even rich end-to-end software tests pale next to a flight test: a physical machine, hundreds of people, someone in the cockpit, and a pre-test just to make the attempt safe. That is already closer to production than to a lab.

Customer org charts confirm the gap. On a robotic-arm company McCord usually sees three teams with three stacks: manufacturing and assembly, R&D and prototyping, and fleet operations in the field. Nominal wants the invisible thread — an anomaly on a deployed robot that comes back to R&D, gets triaged, and traces to a physical part out of calibration. The word users care about, Hoch says, is traceability: where this part came from, which tests it underwent. You cannot run every subsystem through every test on an aircraft. Budget, time, and resources will not allow it. Keeping track is fundamental hardware engineering.

Lin later boils the business down to collect, visualize, analyze, iterate, report, and asks whether that loop is tailor-made for agentic AI. Hoch’s answer is practical. Every point in the loop can be accelerated. The prerequisite is that the data is in one place. A person who uses AI tools every day might naturally ask what happened in the last fifty tests that is relevant to this one. That question assumes those fifty tests even live together. For years the primes have been stuck on that step. Some, he says, are tired of building it in-house and are finally willing to take a partner.

McCord hears the same fact from chief engineers and CTOs: insights are trapped in the hardware. Data-acquisition systems, test stands, lab benches, power supplies, instrumentation — that is the bread and butter of bringing a product to life — and only a small percentage ever reaches a central repository with metadata, structure, and a catalog. He used to call the missing layer digital engineering. The conversation now rhymes with physical AI. The building blocks are the same.

The episode gives three concrete uses. Anomaly detection: a robotic arm does a 120-second laundry-folding test; servos, motors, and physical performance can drift, and dumping all of that into model training is dumping dirty data. Nominal runs in the background and marks which telemetry was actually in calibration, so only the trustworthy slices go into training. English-to-code: a flight-test check needs to run at a remote range even when the engineer is not there; the old friction was learning a domain-specific language, and a natural-language prompt can be what gets the check over the line, then versioned and deployed. Control-room copilot comes later in the conversation: ten thousand sensors at a million points a second is more than a human can watch, so an agent that says this run is out of family versus the last fifty is worth a look.

Hoch has run the arithmetic. Catch the issue in the moment and it is a thirty-minute fix. Miss it and something can explode, then the whole company loses two days of its most critical test campaign. Putting more data in front of more eyeballs is not a nice-to-have at that ratio. It is the difference between a footnote and a halt.

McCord offers a staffing ratio. It used to take about fifty people to test and validate one physical product. The ratio is now closer to one-to-one. The target is one person, with agentic tools, running fifty systems in parallel. Users already feel the first version of that: more eyes on more data, a thirty-minute catch instead of a two-day freeze. Hoch jokes about a Moneyball for hardware testing — obscure stats that surface what a human would not notice. He also wants operator audio from the test floor in the platform. Before AI tools, the bang for the buck was not there. Now, of course you collect it, and you wait to see what the asset unlocks in a couple of years.

Lin asks when you can vibe-code an airplane. Hoch’s answer is: a long way. When he recruits software engineers he likes to say that we spent decades making the internet work pretty well, and we are still far from generating an airplane in natural language. The missing steps are data collection, cleaning, and standardization. His domestic version of the gap: he still has to assemble IKEA furniture himself. He cannot even design his own IKEA furniture at home. Do not talk about the airplane yet.

Huang’s analogy is the one Hoch says he is most excited about. Software has verification agents. Could Nominal be the verification agent next to each company’s design agent? Part of why coding agents got good so fast is that code is verifiable, so the learning loop is tight. Hardware will not copy that path until it has test and validation infrastructure — unit tests for the physical world.

The more frontier example is a program with the U.S. Air Force and DARPA called CYPHER, for cyber-physical systems executing in real time. Major aircraft and weapon-system tests are still a giant matrix of deterministic points, burned down sequentially, often over years. CYPHER pairs AI agents with digital twins, faster than real time, to recommend the next most informative test condition. Instead of fly, land, check one box, fly again, the campaign becomes a gradient: adjust the vector, retrain a physics-informed surrogate, keep going. McCord says that only becomes possible as the systems themselves get more autonomous and higher-endurance, and that government ranges are the infrastructure that makes the experiment real.

Common sense says better AI means fewer real-world tests. McCord’s judgment is the opposite, and it is the claim he wants remembered. “We’re entering a period where there is going to be much more hardware testing.” The pendulum, he thinks, is swinging back. Teams are coming to grips with how little they understand about how physical systems operate, and how lacking they are from a data perspective. More companies are building more hardware. Next comes a race to collect that data and train the models.

Only after that loop closes does the industry get the world people actually want: agents on design, manufacturing, testing, and monitoring, everything pre-simulated, the physical test preferably run once. He thinks that world is far away. The path through the next few years is not. Demand for fusion of real-world test data and model outputs will explode. Lin asks whether every hardware company becomes a physical AI company. McCord’s answer is yes, at least in the sense that design, generation, and manufacturing should be accelerated until people’s creativity in the physical world is unlocked the way it has been in software.

Hoch’s closing analogy is the one that lands. If AGI designs a video game for your child, you might let them play it without rigorous testing. It is just a video game. If AGI builds a physical toy, you would want to be sure it was not dangerous. “The physical world will just always be different, because it’s what we live in.” McCord’s present-day exhibit is the F/A-18. Any aviator, he says, will tell you it is an inefficient jet — extra stuff on the airframe, the way the rear fins are mounted — leftovers of the process by which it was tested. That is what the worst test process writes into metal. The best test process, squinted at, would let you build more flexibility and versatility into the product. Lin asks why not feed those inefficiencies into a model and strip the extras off. McCord has talked to some “pretty emboldened people” who are trying.

The Department of Defense’s posture is shifting too. This administration, McCord says, has been forward-leaning on AI. A few years ago an AI mention in a proposal could be close to a disqualifier, because mission-critical work cannot afford experimental components. Nominal’s own team uses coding agents happily for front-end React, and laughs at some of the back-end suggestions: merge that and it would be really bad for the customer. Skepticism had a reason. It is changing. Collaborative combat autonomous aircraft are pushing the frontier, and Nominal has worked closely with Anduril and other vendors on that project.

Both cofounders came out of MIT, and Huang asks why a new graduate should join them. Hoch’s pitch is that their customers’ problems still have to obey physics, including the computing kind. If a data set is too expensive to ship to AWS, and you still have to crunch it to know whether the physical system worked, you are back in a set of software principles a lot of people have left behind. He thinks software engineers will spend a disproportionate share of the next decade on exactly those physical-world problems, and that Nominal is on the leading edge of that shift.

Lin asks whether they will ever build hardware themselves. Hoch says yes. The supply chain of hardware data runs from a sensor, through processing and analysis, to a report in the hands of someone who has to decide whether a satellite is safe to launch. Getting better at that chain probably means touching every part of it. The principle is that they have to earn the right to capture data: make the user’s day an hour shorter, and the cataloging comes along as a side effect. You cannot lead with “use this because it files the metadata correctly.” McCord is already smiling. He thinks it will happen sooner than outsiders expect. They are, he adds, further along than they are letting on.

Huang mentions a Sequoia partner who constantly reminds the firm that hardware is the only moat. She then wishes them the GitHub of hardware — a layer that would change the professionalism, reliability, and speed of engineers now “galvanized to go off to space.” McCord answers with the aviation status call that is also the company’s name: all systems nominal. A company named for “everything is fine” is in the business of making anomalies visible earlier. That is the joke, and the product.