Compose small decisions before reaching for a generative model

Raymond Weitekamp introduces recursive decision models and his broader goal of an interface he calls Operating System One. He asks which tasks can be accomplished with many small decisions instead of continuous text generation.

He notes that names such as system-one model, decision model and zero-shot classifier are contested. In his framing, a model ranks or judges choices while code decides what action to take.

The central question is whether some behavior that appears to require extended reasoning can be organized as many fast, limited judgments.

His recursive decision model idea combines recursive language models with decision models. He contrasts open-ended generation, which expands possibilities, with choosing among available options.

A recursive language model can call smaller instances until a depth limit. Weitekamp applies a similar tree to decision calls and also considers combinations with ordinary agents.

A common pattern makes a decision model a tool for an LLM, for example to route a request or assess a tool call. His focus is the reverse: a decision model at the root, which may call an LLM when needed.

He revisits a Doom demonstration in which one decision composition handles immediate movement and another handles map exploration. Together they control a game-playing agent.

Weitekamp sets three design rules: keep context outside the model so code can slice it; allow decisions to invoke further decisions; and first make the system work without generation while still allowing generative components to be composed.

A Galton-board analogy illustrates successive choices sorting items by properties. He calls it a toy example and notes that the same simple task could be done in one classifier call.

In hierarchical product search, a model chooses which branch of a large category tree to explore next rather than seeing all 12,000 categories at once. Code can choose how many candidates to follow.

He shows visual search examples that repeatedly narrow an image region, while acknowledging that a capable vision model might solve some examples in one pass.

In a Factorio demonstration, the system identifies a problematic production line and an element to repair using decisions without a generative language-model call. He notes that its path through the game was not optimal.

A decision-only system has a limitation: available actions must already be enumerated by an application, script or another component. It cannot invent arbitrary text on its own.

Weitekamp suggests that some games, browser interfaces and computer-use tasks expose actions that can be enumerated, making decision trees useful even without generation.

Generation can complement fixed decision trees by creating or modifying the available structure. He frames the two kinds of models as complementary rather than interchangeable.

A smart-home example expands a simple command into actions for rooms and devices. The decision system can select actions while a language model handles cases requiring generation.

He calls the shared interface between them a map: named options connect descriptions to functions, while the language model can add or revise entries as user behavior changes.

An arbiter governs which proposed actions may change shared state. This matters when several fast decision paths run speculatively in parallel and could otherwise collide.

He uses streaming audio as an example: several paths may form hypotheses before a sentence ends, and the arbiter decides when there is enough confidence to act.

The initial implementation used DSPy; he also describes a standalone Rust package for experimenting with the approach.

Operating System One is his longer-term effort to control computers with less physical strain. He describes discomfort from prolonged keyboard and mouse use.

He looks back at music experiments in which laptop sensors and the Princeton Laptop Orchestra turned gestures and devices into instruments. This was also how he learned programming and machine learning.

In his Earshot prototype, one layer transcribes audio and selects an action while a tiny model learns to map short audio slices to the same actions.

A teacher supervises the small model, and another decision layer determines when its predictions are trustworthy enough to use. In his demonstration, the response becomes faster as it learns.

He mentions a separate local visual-classification experiment to illustrate that small models can support responsive interfaces beyond audio.

His planned OSO computer-use agent starts with voice, uses decision-first coordination and can hand suitable tasks to ordinary coding agents. He says the project is not yet ready.

The longer-term idea is to learn user preferences and move familiar decisions to smaller, faster models when the system has enough evidence to trust them.

Weitekamp invites viewers to explore the recursive decision model implementations and follow the OSO project.