A Tiny Network Grew a Salt Crystal the Training Data Never Showed

A few months before this conversation, Tim Duignan posted a short video on Twitter: salt water, a tiny crystal, and the crystal getting slowly larger. He called it the most exciting result of his career. Nearly two million people watched. Behind the clip there was no foundation model. Generating the training data took a few GPUs for a day. The network itself had tens of thousands of parameters. Training took a few hours. Against that, the large language models of 2024 were eating trillions of tokens and billions to trillions of weights.

Duignan is an applied mathematician at the University of Queensland who has been in love with electrolyte solutions — the technical name for salt water — since his PhD more than a decade ago. Nathan Labenz, host of Cognitive Revolution, found the work the way he finds most things: a viral tweet, then a rabbit hole. The episode title, *Fluid Intelligence*, is a pun. It is the psychologist's term for flexible problem-solving, and it is also intelligence about fluids. Labenz wants listeners who live in AI to sit still for computational chemistry, because the clip is not a toy visualization. It is a demonstration that a small, physically constrained network can invent a phase of matter it was never shown.

The rest of the hour is a crash course in how that demonstration was built. Duignan walks through ab initio molecular dynamics, femtosecond time steps, equivariance, and coarse-graining — stripping the water out of the simulation so the network has to learn the solvent as an average. The compute, by the standards of frontier language models, is almost embarrassingly modest. The scientific claim is not. If a network trained on a liquid can grow a crystal, the question is no longer whether machine learning can speed up a force calculation. It is whether it can guess a phase of matter.

Tell a stranger you study electrolyte solutions, Duignan says, and they think Gatorade, then politely lose interest. The default intuition is that salt water is a solved problem. It is not. The ocean is a salt solution; carbon dioxide dissolves into it as carbonate, which is a load-bearing term in every climate model, and aerosols form from the same class of liquids. In a lithium-ion battery the electrolyte is what connects the terminals, and it sets how fast the cell can charge and how long it lasts. Biology is the third pillar. The electrical signals that run a brain are ions moving in and out of cells, which is why evolution built ion channels and ion pumps — molecular machines whose only job is to move salt.

From climate to batteries to life, electrolyte solutions are among the most important substances on Earth, and science still cannot predict their most basic properties from first principles. First principles, in Duignan's usage, means starting from the Schrödinger equation and Newton's laws and computing solubility, activity coefficients, and chemical potentials. Those numbers then get plugged into larger models of oceans, minerals processing, and cells. Today they are measured, not derived. High-temperature work needs expensive kit. A new, unstudied electrolyte means going back to the lab and sweeping temperature and pressure from scratch. That work does not get you *Nature* or *Science*, so people mostly do not do it. The databases that exist were filled decades ago, and machine learning is hungry for exactly the data that is no longer being made.

The failure mode that makes the gap visceral is a battery fire. On the first charge — the formation step — a salt layer precipitates on the electrode. If that layer, the solid-electrolyte interphase, forms badly, lithium dendrites can grow, pierce the separator, short the cell, and start a fire. Jeff Dahn's million-mile battery program is, in practice, a hunt for electrolyte additives that control that film, followed by cycling the cell for the equivalent of a million miles to see what happened. It is trial and error on a timescale that hurts. Machine learning can already extrapolate from a few early cycles, Duignan notes, but the dream is molecular-scale insight into which crystals form on the surface and why. Until the Schrödinger step is cheap, that dream stays a dream.

Between the lab and the neural net sits a loop chemists call ab initio molecular dynamics — AIMD, an acronym that, Duignan notes dryly, has nothing to do with AI. You solve the Schrödinger equation for the forces on every atom, then step Newton's laws to move the nuclei, then solve again. The movie that comes out is a trajectory of atoms. He ran these on Department of Energy supercomputers in the United States. Even there the systems were tiny: something like a lithium chloride solution with about a hundred water molecules, simulated for picoseconds. The spatial and temporal window is a feature as well as a bug. Nothing else can watch matter at those scales. It is also why the method cannot be the production tool. Most of the chemistry you care about lives at larger sizes and longer times.

The time step is a femtosecond, 10^−15 seconds, because that is the rhythm of molecular vibration; thousands of those steps make a picosecond. They solve the time-independent Schrödinger equation, not the full time-dependent one, using the Born–Oppenheimer split: electrons are fast, nuclei are slow, so you compute a ground-state energy and wavefunction with the nuclei frozen. The practical basis sets are Gaussians centered on atoms, or plane waves, or a mix; Duignan's group uses CP2K. A brute-force treatment of every electron coupling to every other electron — full configuration interaction — scales exponentially. Decades of quantum-chemistry approximations have beaten that down to a polynomial, still not cheap, still not something a normal lab can throw at every electrolyte.

The nuclei themselves can mostly be treated classically. They are heavy enough, and room temperature is hot enough, that Newton's laws are the large, warm limit of quantum mechanics. Hydrogen is the occasional exception. Electrons are not: they are light enough that you have no choice. At room temperature, decoherence also does the user a favor. Superpositions wash out as soon as a system talks to a thermal environment, which is why a quantum computer has to be isolated from cosmic rays and why Duignan does not have to worry about wavefunction collapse. He is hunting the lowest eigenvalue of the Hamiltonian — the ground state — which he calls almost the kindest problem quantum mechanics will give you. Kind is not the same as cheap. The bottleneck is clear: the quantum force call is too expensive to scale.

The expensive half of the loop is the quantum force call. Neural network potentials replace it. Coordinates of every atom go in; total energy and per-atom forces come out. You still generate some training labels with quantum chemistry, then do supervised learning, then keep the cheap Newtonian half of the tick-tock. The idea dates to the 1990s. The founding paper is Jörg Behler and Michele Parrinello in 2007, using an ordinary multilayer perceptron. The current generation is a thousand times faster or more. Systems a thousand times larger, run a thousand times longer, become routine. Duignan's architecture is NequIP, an equivariant graph neural network with message passing from Boris Kozinsky's group at Harvard, the same lab that has simulated an entire HIV capsid at millions of atoms — the scale where you finally do want a GPU cluster. TorchMD, a Transformer, is another option that performs well.

The model is strikingly small. Tens of thousands of parameters, Labenz makes him confirm more than once. Classical force fields get by with a few dozen; frontier language models live in the billions to trillions. Duignan's explanation is that the distribution being fit is vastly simpler than "all knowledge on the internet." Data generation is the costly step, a few GPUs for a day on Australia's ordinary supercomputer allocation. Training itself is hours, not days. He has not seen grokking. Universal force fields, aimed at any element in any arrangement, will push the parameter count up toward language-model territory. For a single electrolyte, they do not have to.

The leap that made him jump was equivariance. Raw coordinates are arbitrary: rotate a water molecule and every number changes, but it is still water. NequIP builds the rotational symmetries of three-dimensional space into the architecture with tensors, the same mathematical objects chemists have used for decades in the multipole expansion — dipoles, quadrupoles, octupoles. The math comes from the group theory and representation theory that remade particle physics. In the NequIP paper's water-box table, force error dropped by about four times on a thousand times less data. "It's not often in science you see something go down by three orders of magnitude," Duignan says. "That's the kind of thing you jump on."

The field is arguing about whether you even need that prior. AlphaFold 2 used equivariance; AlphaFold 3 dropped it and recovered comparable metrics with data augmentation, rotating the set a thousand ways and feeding a plainer net. The same trick has been shown for network potentials on loss curves. Equivariance is slower, because tensors talking to tensors are more work than matrices. An Apple paper recently swallowed the bitter lesson whole: stop being clever, stack parameters and data. Duignan is rooting the other way. If any inductive bias is worth hard-coding, it should be fundamental physical laws and the symmetries of nature. He suspects the data-side version will struggle to generalize into crystallization. The debate is not settled. It is worth watching.

The viral clip was not even an all-atom movie. After training a network on quantum forces, Duignan's group coarse-grained: they threw the water molecules out of the simulation and asked a second network to predict energies and forces on the ions alone, with the solvent present only as an average. Their solution was about four molar; pure water is about 55 moles per liter, so roughly ten waters per ion. In statistical mechanics that average force is a free energy. Training it needs longer trajectories, because a given ion arrangement does not have a unique force — it depends on how the waters happen to sit. People had run all-atom network MD of electrolytes before. Crystallization almost never shows up there. It is a rare event on very long timescales, the sort of thing you usually have to drag out with enhanced sampling.

They checked the training trajectories. The radial distribution function looked liquid. Ions paired, occasionally made a triplet, and never built a large cluster. Then, in the coarse-grained run, a crystal started to grow. Water's dielectric screening weakens ion–ion electrostatics by a factor of about eighty; in the crystal there is no water and the forces are a different world. Duignan had spent years of his PhD hand-building continuum solvent models that ignore water and treat only ions. He fitted equations, tuned parameters, and produced models that would not generalize. He gave up. He thought it was impossible. "This is the kind of thing I'd been trying to do for years. I had given up and thought it was impossible, and yet it just dropped out for free." His working guess is that the net had already seen waters squeezed out of transient pairs and triplets, felt the forces jump, and treated the crystal as the end of that curve. Quantitatively it will not nail solubility. The repair is active learning: add the crystal structures to the set and train again.

Labenz asks whether this is the same phenomenon as an LLM learning higher-order concepts en route to next-token prediction. Duignan's frame is a high-dimensional surface. Positions in, energy out: a potential-energy surface, or a free-energy surface once the solvent is integrated out. Train on the liquid region and, somehow, the net extrapolates into the crystal well. Language models fit a different surface, the probability of the next word. The two are closer than they look. In statistical mechanics the log of a probability density is an energy; the softmax layer in a Transformer is a Boltzmann distribution. Log probabilities, in that language, are free energies. He does not claim to have the mapping fully in his head. He does think the link between statistical mechanics and large language models is a research program, not a metaphor.

The second free gift was self-ionization, back in the all-atom water simulations. Run long enough and a hydrogen hops from one water to its neighbor, making OH− and H3O+. In real water about one molecule in ten million is in that ionized state; that is where pH 7 comes from. The net keeps hydronium in its pyramidal shape and produces chains of hops — the Grotthuss mechanism, the reason acid diffuses through water so fast. None of that was in the training data. The rate is too high, so the quantitative pH is wrong, but people have already fixed it by adding proton-transfer geometries to the set and recovering pH 7. Temperature, meanwhile, does not belong in the network. The energy surface does not know about temperature; a thermostat on the Newtonian side adds or removes energy. Turn it down too fast and you get a glass, atoms frozen before they can order. Ice in a kitchen freezer is a real crystal, held up by hydrogen bonds, and ice has a whole phase diagram of forms. Other groups have already used network potentials on ice–water equilibrium and seen a cousin of Duignan's result: train on the liquid, get something to say about the crystal.

Temperature is almost a free lunch because it lives in the physics engine, not the net. Microsoft's MatterSim paper pushes that across huge temperature ranges, which is exactly where experiment gets expensive. The next ambition is a universal machine-learning potential: one model, any element, any arrangement of atoms, forces on all of them. ByteDance has already published electrolyte work that extrapolates to organic electrolytes absent from the training set. Google DeepMind's GNoME, in *Nature*, reported millions of new crystal structures. Meta is in the race. On the open side, MACE-MP-0 was trained only on crystals and then simulated liquids — the mirror image of Duignan's liquid-to-crystal surprise. The pattern looks like language models: a strong foundation plus a little extra data on a new system beats training from scratch.

Labenz borrows a friend's phrase, the "big tech singularity": if the only way forward is to scale, a handful of companies could end up owning materials science. Duignan treats that as a real risk, not a sci-fi plot. His prescription is public money, organized the way CERN organized the Large Hadron Collider, funding teams outside the firms and releasing the models. A professor plus a few PhD students cannot run this as a cottage industry; GNoME's hundred-author collaboration is closer to the template. AlphaFold 3's delayed release already previewed the fight. The sting in his observation is practical. Inside the companies, he says, there are not that many people with the time to run the simulations that would actually matter. The scientific value of releasing the tools is larger than the scientific value of sitting on them. Recouping a giant GPU bill is a fair counter-argument. It does not make the gap smaller.

China is not behind on this track. ByteDance's electrolyte paper is the exhibit Labenz keeps for himself and for the audience: do not talk yourself into a lead you do not have. Dual use travels with the method. These potentials are not just materials tools. David Baker's group recently used one in *Science* to design new drug molecules, and a preprint put a network potential at the state of the art for the core of drug–protein binding. Design a drug and you can, in principle, design a harmful molecule. Defense materials are a quieter concern; Duignan is not losing sleep over them yet. Labenz floats KANs, the Tegmark group's architecture that puts learnable functions on edges instead of scalar weights. Duignan likes the idea for fitting high-dimensional surfaces and for keeping old knowledge when new data arrives. The machine-learning community's best habit, he says, is empirical: try it, time it, and do not pick a team in advance. Computational efficiency will decide.

Asked how this changes the pace of discovery, Duignan is optimistic and allergic to hype. Plenty of senior professors are still skeptical. His own view is that the tools are transformatively useful across fundamental science, and that the central constraint is the one it has always been: training data. AI is a very hungry beast. The sweet spot is a problem where you can generate large, diverse, high-quality data efficiently. He stayed away from machine learning for years because electrolyte databases were too small; AlphaFold was the late conversion. Now he sees two data flywheels pointing at the same place. One is the automated lab, the New York Times story of a pharmaceutical company running robots around the clock. The other is quantum chemistry on demand. He does not yet see a hard ceiling as the models get bigger. Drug discovery's current texture — intuition, trial and error, no view of the molecular scale — is what he thinks starts to age out.

The practical checklist is four items. First, ask whether your domain can mint training data cheaply, whether right and wrong can be checked without a human, and whether the most expensive computational step can be swapped for a network; if all three, you are in the dessert. Second, run the active-learning loop: train on a little, simulate for a long time, watch an uncertainty signal, spend a first-principles call wherever the net is unsure, and retrain. The ground truth is always computable, which is the luxury language-model hallucination never gets. Third, if you work in materials, chemistry, or drugs, start with an open universal potential such as MACE-MP-0 as a first estimate, then fine-tune on a little system-specific data. Fourth, leave temperature and pressure to the physics engine. The energy surface does not depend on them; a thermostat is almost free. Do not make the network learn what it does not have to.

The longer horizon he wants is hierarchical coarse-graining, not one network to rule every length scale. A drug binding a protein needs every atom in the pocket and can treat distant water as an average, possibly jumping resolution on the fly. He has not trained a single net on both all-atom and coarse-grained descriptions yet; he thinks someone should. Jack Oliver's *The Incomplete Guide to the Art of Discovery* is his recommended hack: steal a tool from another field. Newton stole calculus for gravity. X-ray crystallography stole physics for protein structure. Diffusion models stole nonequilibrium statistical mechanics — thermal annealing, Langevin dynamics — and then a Microsoft paper showed that a diffusion model trained on molecular trajectories is learning a score mathematically identical to a force field. Full circle, with better tricks. Statistical mechanics, he suspects, still has unused language for grokking, for the Boltzmann distribution of trained weights, for why a learning curve looks like an energy dropping into a new phase.

He closes with a request to the machine-learning community. Superintelligence, he says, is not obviously what society asked for. Ask people whether they want something smarter than a human in every way and they will probably say no: they want the dishes done, not better art than they can make. He does not want a bot that reads the literature and generates his hypotheses. Those are the parts he likes. Sam Altman's picture of an AGI that finds a grand unified theory and then explains it is, to Duignan, looking in the back of the book. Build tools that let humans solve the stuck problems — including making simulations a thousand times faster. Labenz is mostly with him, with one split: he would take a narrow superhuman physics solver, an AlphaGo of unifying gravity and quantum mechanics, even if a particular human lost a eureka. What worries him is generality, and the Anthropic results in which flattery generalizes into uglier deception. A path to a narrow materials discoverer is not obvious. For now, Duignan's salt crystal is the exhibit that a small, physically biased net can already surprise its author.