What Weight-Space Programming Experiments Show—and Their Limits

Matthew Berman introduces a research report on weight-space programming: using a language model to inspect or directly modify another neural network’s parameters. He is excited by possible implications, but the reported experiments are early and task-specific.

He starts with a simplified explanation of neural networks as collections of numerical weights. Training commonly adjusts these parameters to reduce a loss, but understanding exactly how internal structures produce a behavior remains difficult.

The researchers ask whether an LLM can reason about weights and write edits directly, instead of updating them through gradient descent. Their report also notes earlier work on interpretability and other methods for generating or editing weights.

The distinction is between a language-guided system proposing a concrete change and a conventional optimizer applying a fixed update procedure. Some experiments deny tools and feedback; later ones permit a sandbox and a limited budget of forward passes.

In the first experiment, the team trained four small policies on grid versions of Breakout, Pong, Pac-Man and Freeway. The policies shared the same architecture and interface; Astra received 17,253 weights and descriptions of the input and actions, but not the game names.

The researchers report that Astra named the intended game in all four cases. Berman walks through how patterns in the weights led it to descriptions such as a paddle or a moving object; the experiment concerns these compact, structured networks.

The screen shows the numerical weights and the model’s explanation for the Breakout example. Its interpretation links weight concentrations and action specialization to the layout of that game; this should not be generalized into a claim that every internal computation is understood.

Berman also describes the Pong, Pac-Man and road-crossing examples. The researchers’ setup identifies the fourth environment as a grid version of Freeway, illustrating why the exact experimental setup matters when interpreting a model’s game label.

The second experiment uses small diffusion models trained to generate circles, squares or triangles. The report says Astra identified the shapes from weights without running forward passes, but described outlines where the actual outputs were filled shapes.

The first two experiments expose input or output layers that can carry useful clues. To make the next test harder, the researchers removed those layers from small character models trained on English, romanized Japanese and Python.

The report says Astra identified the training language and copying heads from intermediate weights. It could analyze weights with Python but could not run the network or its layers on input; the models were roughly 7.2 million parameters.

Next, the team asked Astra to repair a trained policy that repeatedly failed to drive a car over a ramp. It received the weights, a description of the scene and one recorded episode, which are important context beyond the numerical weights alone.

The researchers report that a single edit changed 3.5 percent of the weights and succeeded in all 1,024 evaluation episodes. The resulting policy first backs up to gain enough speed, then drives forward.

Berman speculates that direct edits could eventually contribute to self-improving AI. The controlled repair of a small external policy does not itself demonstrate that a frontier model can reliably inspect or recursively improve its own weights.

In another experiment, Astra generated weights for small networks playing three MinAtar games. It received the game rules and source code, but not training data, gradients, tools or practice in the environment before evaluation.

The generated networks beat the reported 100-million-step PPO baselines in all three games. Against 500-million-step baselines, the results varied: stronger in Breakout, close in Asterix and below the baseline in Seaquest.

The researchers then moved to a 1.5-million-parameter network for 9×9 Go. Because the weights exceeded the context window, Astra used a Python sandbox to write tensors and was given a varying budget of forward passes.

With no forward passes, the generated network won more than half its games against two smaller PPO networks. With 50 forward passes, it beat the report’s equal-size PPO baseline trained on 20 million games; each permitted pass could include a batch of rollouts.

The next test asked Astra to identify which of two language models was trained to be sycophantic. The report says it failed to map differences in weights to the behavioral distinction when it could inspect weights alone.

When the researchers additionally exposed attention patterns from selected inputs, Astra identified the sycophantic model in all three pairs tested. This illustrates both a limitation of the stricter test and a benefit from additional observations.

Finally, the team asked Astra to improve math performance in Qwen3.5-0.8B by editing weights without gradient training. It could use a Python environment, practice problems and a limited number of forward passes, but did not see the evaluation test sets.

The report says Astra targeted a repeated-reasoning loop, editing 0.016 percent of weights. With the 50-pass budget, reported MATH-500 rose from 7.9 to 48.1 percent and GSM8K from 13.8 to 67.1 percent under the specified sampling setup.

The authors stress that suppressing loops is not the same as adding mathematical ability: it can let the model finish answers it could already work out. In a further test with a Gemma model that rarely looped, the edits did not establish a meaningful math improvement.

Berman closes by considering possible implications for optimization, interpretability, alignment and self-improvement. Those are hypotheses about future work, not findings that the experiments have solved interpretability or demonstrated recursive self-improvement.