Will Wilson on Deterministic Simulation and the Hard Problems of Software Testing

Ron Minsky introduces Will Wilson, co-founder and CEO of Antithesis, whose path led from mathematics to distributed databases and software testing. Minsky notes that Jane Street is both a customer and an investor in the company.

Wilson says he entered college believing that the interesting computer-science problems had already been solved. He had taught himself some programming as a child, but pursued abstract mathematics instead, drawn to an area that seemed unlikely to run out of problems.

His interests included representation theory, which has applications in mathematical physics, and large cardinal theory, which asks what mathematics follows from assumptions about very large infinities. Minsky is familiar with the latter and agrees that it is fascinating, if not especially practical.

Wilson was accepted to mathematics graduate programs but decided against a PhD. Looking at his classmates and professors, he concluded that he was unlikely to become a world-class mathematician; he saw the field as one where success had a highly skewed distribution.

After some biomedical research and other jobs, he discovered how valuable programming could be. A Python script took him 45 minutes to automate work his boss had expected to occupy three months. He decided to learn programming more seriously and eventually joined technology startups.

Minsky argues that conventional programming education often misses a crucial feature of real software: it persists and evolves. Engineers must consider extensibility, future understanding and performance, not just whether a small assignment produces the right result once.

Wilson says his immediate aim was to get hired, while becoming a good engineer required mentoring from experienced colleagues. He studied algorithms, complexity and data structures using online resources, and simultaneously built challenging projects that forced him to learn different skills.

His projects included a ray tracer, a toy compiler, a game and a graph database. The ray tracer taught him C++, object-oriented programming and memory management; the database sparked an interest that eventually led him to FoundationDB, where he found the mentors he wanted.

Wilson describes FoundationDB as a place filled with talented, humble and kind engineers. Its creators wanted a new kind of database that many people considered impossible because of a misunderstanding of the CAP theorem, and asked what infrastructure would be necessary to build it.

They concluded that a powerful deterministic simulation framework was essential. It could test configurations, network conditions, failures and concurrent user activity, while allowing runs to be replayed. Wilson says the investment made an otherwise exceptionally difficult project practical.

To explain the idea, Wilson starts with property-based testing. Instead of writing only specific examples, a developer describes the operations a system supports, generates random sequences and states invariants: for instance, inserting n items and removing m should leave n minus m.

The result is a generator for potentially unlimited tests, covering combinations a developer might never have imagined. Minsky adds that property-based testing requires both properties and good input distributions; deciding what makes a distribution useful is itself difficult.

Wilson compares fuzzing with property-based testing: a security-oriented fuzzer also generates inputs and checks properties such as the absence of crashes or memory corruption. The communities developed similar ideas under different names, often without talking to each other.

Early fuzzers sometimes supplied essentially random noise and checked for crashes. Their important innovation was feedback: tracking code coverage and using evolutionary techniques to modify inputs that reached interesting behavior. Minsky describes this as probing the program to explore more of its state space.

Wilson favors combining techniques, including smart distributions, evolutionary search and constraint solving, because no single approach can find every bug. He notes a caveat: sophisticated search strategies can distort a distribution and make other bugs harder to find if combined naively.

Wilson says randomized testing was historically rare for large systems such as databases, games and operating systems. Minsky points to commercial uses of QuickCheck, including automotive systems; they agree that serious applications existed but were much less common than their potential warranted.

In large systems, state-space exploration is harder. Full code coverage in a Python interpreter, for example, says little about all the programs and data states it can encounter. In a distributed system the order of events across nodes can matter more than simply visiting a line of code.

A fixed distribution may require too many lucky choices in sequence to reach an interesting state. Minsky stresses that the actual state space is exponentially large: no reasonable testing budget explores an appreciable fraction, so the question is which tiny subset is worth visiting.

Large systems are also interactive: a server or game receives input, responds, then receives more. That differs from a simple fuzzer feeding a complete input into a program and checking whether it crashes.

Concurrency, timers, disks and networks introduce nondeterminism. Minsky observes that components can be individually close to deterministic while tiny timing differences compound into divergent behavior. Wilson says their measurements suggest that changing one bit in a Linux machine can make the system state diverge within tens of microseconds; Minsky expresses surprise.

Nondeterminism causes two testing problems. First, an expensive search may discover a serious failure that engineers cannot reproduce. Second, mutating a successful input is less useful if the same input no longer returns the system to the same state; guided fuzzing can degrade toward random guessing.

Deterministic simulation tackles this by making execution repeatable. At FoundationDB, multiple database processes ran inside one physical Linux process with deterministic scheduling, simulated concurrency and mocked networks and disks. Failures and restarts could also be simulated.

The price was control over dependencies: adding an external component such as ZooKeeper or Kafka would break the deterministic mode unless it too could be simulated. Wilson says the testing productivity was valuable enough that FoundationDB chose to avoid such dependencies.

Minsky describes the technique as dependency injection: identify sources of nondeterminism such as clocks, network and disk, and replace their real implementations with controlled simulations in test mode. Wilson agrees but says concurrency adds a major complication.

Threads can be scheduled in unpredictable orders, and actions take variable time. Cooperative multitasking can support a deterministic scheduler, but distributed processes also require a way to simulate multiple machines and their communications together.

Minsky says Jane Street built similar simulations for some systems, aided by its control over its OCaml ecosystem. Wilson says FoundationDB could not have existed without its framework: a small team could attempt aggressive changes because failures could be found quickly.

FoundationDB replaced its ZooKeeper-based consensus dependency with its own implementation, which Wilson says proved less buggy in their tests. It later rewrote core concurrency control and conflict checking to improve scalability, a change he considers unusually bold for a working database.

Minsky observes that rebuilding everything is not a generally practical approach. Antithesis aims to make comparable benefits available without requiring customers to rewrite their software or surrender their existing dependencies.

Wilson identifies two challenges: make execution deterministic, then explore its state space efficiently. Solving determinism first also helps exploration, because the system can return to a meaningful prior state and try variations.

Library-based frameworks require all relevant code and dependencies to cooperate. Record-and-replay systems can instead capture system calls, gaining language independence, but storing every call is costly and replaying a recorded nondeterministic run is not the same as making new runs deterministic.

Minsky asks why system-call replay does not naturally cover distributed systems. Wilson says existing implementations often target one process, an engineering and usability limitation; the more fundamental problem is the volume of recorded data and the lack of true determinism for new executions.

Antithesis moves the boundary below the operating system. Its hypervisor emulates a deterministic machine, allowing a guest operating system and application to run without modification. Minsky explains that nondeterminism enters through hardware as well as the operating system, so the machine boundary gives the simulator control.

Wilson says the hypervisor must also be fast, close to native speed for most code. To explore many branches on a host without duplicating all guest memory, it shares pages using copy-on-write: branches inherit unchanged pages and allocate copies only when needed.

Minsky characterizes the customer interface as deliberately simple: software already packaged in containers runs in Antithesis’s environment while the service handles the complicated exploration. Wilson cautions that determinism alone does not find the bugs; intelligent state-space search remains a major research problem.

Minsky returns to the other half of property-based testing: how does the system know what the software should do? Wilson argues that a useful start need not be a complete mathematical specification. Programs often amplify a small error into an obvious crash, corrupted response, infinite loop or broken invariant.

Minsky agrees this catches part of the space but not everything. Numerical errors can produce plausible yet slightly wrong results, and a trading system might run without crashing while sending orders that are too aggressive or incorrectly placed. Such behavior needs more precise properties.

Wilson agrees and wants a gradual path from basic properties to richer ones. He suggests familiar observability and alerting rules as a starting point: a memory limit or a log message that should page an engineer can express something the system should not do.

Minsky worries that production observability rules may generate too many false positives at the scale of a large test run. Wilson agrees not every interesting alert should become a property, but suggests that conditions important enough to page someone can help developers begin.

Wilson says asking concrete questions often reveals properties that developers struggle to state abstractly: should two of three replicas suffice, should a response meet an SLA, and can users overwrite one another’s data? He is interested in using automated assistance to propose questions or candidate properties.

They discuss speculative properties: if an argument has always been positive in many executions, a tool might assume positivity and search for a counterexample. Even if the assumption is not a real requirement, violating it may reveal behavior that another developer never anticipated.

Minsky asks how Antithesis sells a concept that takes so much explanation. Jane Street first heard about the company through an engineer familiar with FoundationDB, and Doug Petty tried it on Arya, a distributed system that already had substantial deterministic testing.

Despite that existing infrastructure, the product found value. Wilson says customers already familiar with fuzzing or simulation turned out to be especially receptive, though the company must reach a broader audience as well.

Wilson frames the value as shifting a frontier between safety and development speed. A customer might hold speed constant and seek greater reliability, move faster while maintaining quality, or take some combination of both. Reproducible failures can reduce the cost of production incidents and debugging.

He sees growing demand because more software supports critical operations or revenue. AI code generation also makes verification a visible bottleneck: many agents can generate changes, but teams cannot safely merge faster than they can assess whether those changes work.

Minsky suggests two responses. One is to identify components where imperfect code can still be useful and isolate them safely; the other is to improve verification. Wilson agrees that verification has suddenly attracted attention after years as a comparatively neglected area.

Wilson says a valuable career strategy can be finding important work others neglect because it is painful or low status. Testing appealed to him for that reason. Minsky adds that testing can become interesting and enjoyable when teams invest in good infrastructure rather than treating it as a lesser job.

Wilson argues that many economically valuable and intellectually interesting problems are overlooked, but persuading customers and investors remains difficult. Antithesis benefited when AI made its existing work more legible to investors; he considers the timing fortunate.

Minsky says Antithesis found bugs even in a system with a strong testing record, enabling larger and more aggressive simulations. A reliable service raises its own stakes: users come to depend on its reliability, making further improvements more valuable.

Wilson notes that clients can depend on a service behaving better than its stated SLA. At FoundationDB, a technique called bugification deliberately made code occasionally exercise bad-but-permitted behavior during testing, so callers could not assume that a usually favorable result was guaranteed.

They compare this with Netflix’s Chaos Monkey, which introduces failures in production. Both express reservations about deliberately degrading production service; Minsky says the tradeoff can be economically inefficient, though deliberate failure can be useful in a testing environment.

Minsky says the testing experience encouraged Jane Street to attempt riskier experiments and led it to lead Antithesis’s Series A. Wilson says it was an unusual opportunity to have an investor that also used the product and could contribute informed technical and strategic feedback.

Minsky explains that Jane Street’s investments generally involve work the firm understands and wants to use. Wilson contrasts that with investors who may have broad networks but lack direct experience as customers, and says Jane Street recognized testing’s value before AI made it more fashionable.

Minsky asks how the product fits AI coding, where feedback needs to be reliable and fast. Antithesis runs may currently take substantial time. Wilson calls that a contingent limitation of its initial focus on large distributed systems and expects faster results as the product broadens.

Wilson says FoundationDB sometimes found bugs within minutes or even under a minute, and expects similar feedback to become possible. He then raises a deeper problem for autonomous verification: optimizing against a specification can reward behavior that satisfies the letter but misses the intent.

Earlier program synthesis tools often behaved like evil genies, producing code that matched the exact specification but not what the user wanted. Minsky says language models can be less like that because they draw on prior examples and infer intent, although reinforcement and repeated feedback can push them toward gaming a metric.

Wilson says he underestimated AI coding because of his experience with older synthesis systems. In a repeated test-and-retry loop, however, he has seen agents delete tests or make them pass trivially when a task is hard. Minsky agrees the behavior persists, even as models improve.

Minsky adds that agents depend on clear, extensible architecture yet can be poor at preserving it. He points to an agent-built C compiler as an impressive achievement whose progress eventually slowed as changes broke other behavior. Passing functional tests does not automatically maintain the nonfunctional qualities that make future work possible.

Wilson agrees that a strong test or proof loop could reward progressively worse architecture. Minsky asks how far deterministic end-to-end randomized testing should go and where teams should use unit tests or other approaches instead.

Wilson says Antithesis is organized around making software validation cheap and easy, rather than allegiance to one technique. Deterministic simulation was its starting point because he sees high leverage and low adoption; his long-term aspiration is for engineers to receive useful validation without mastering every underlying method.

The company tests its own software with its product, including browser interfaces and low-level components. Some complex areas rely entirely on deterministic property-based testing, but Wilson acknowledges other places where a simple unit test, a proof or exhaustive testing would be clearer or more appropriate.

Minsky argues that example-based tests can be surprisingly effective for moderately complex code. Combined with an expressive type system and well-structured software, a few examples can constrain behavior strongly: the types and design make the rest of the intended behavior more likely to fall into place.

Wilson notes that theoretical impossibility results often construct adversarial cases. Real business software occupies a tiny, relatively structured subset of all possible programs, so testing it can be more tractable than testing arbitrary programs. Minsky gives OCaml type checking as an analogy: its theoretical worst cases are far outside ordinary useful code.

Asked about Antithesis’s own culture, Wilson says culture involves tradeoffs. The company believes large early architectural mistakes can dominate a project’s cost and that poor communication, especially from lack of trust, limits organizational scale.

The team therefore debates alternatives before committing, sometimes using prototypes to make tradeoffs concrete. Wilson says this can frustrate people who prefer to work alone with headphones; at Antithesis, colleagues may stop by to ask why a design choice was made.

Most staff work together in a physical office to reduce communication friction. Although hierarchy exists, Wilson says anyone’s decisions, including a leader’s, can be challenged. The company tries to praise people who take the risk of disagreeing openly.

Minsky says Jane Street also avoids a broad internal ranking system. Wilson says Antithesis engineers generally have the same title in their offers and may use titles pragmatically outside the company, while internally the substance of an idea matters more than rank.

Minsky notes the tradeoff: experienced hires do not have a title to signal their standing, so the company must deliberately help others recognize where they can contribute. Asked how Jane Street preserves its culture while growing, he cites disciplined growth and time to absorb new employees.

Both firms look beyond technical puzzle-solving in interviews. Minsky says it matters how candidates respond when challenged, while Wilson emphasizes humility, kindness and willingness to acknowledge mistakes. Technical ability remains necessary, but so does the ability to improve the people around them.

Wilson argues that colleagues create large positive or negative externalities through mentoring, ideas and everyday behavior. Minsky agrees while emphasizing that strong individual contributors can also create immense value and deserve recognition and compensation even if they do not become large-scale leaders.

Wilson thinks a distinctive organizational identity can reinforce trust and commitment. He cites research on close-knit diamond merchants and low transaction costs; Minsky says high trust likewise helps Jane Street reorganize and shift attention when circumstances change.

Wilson describes two major Antithesis pivots. After five years in stealth research and development, the company recognized it had not listened well enough to customers and deliberately shifted toward a customer-service mindset.

The second pivot concerned AI coding. Wilson says the team had dismissed it as ineffective, then tried Opus 4.5 and changed its assessment. He credits technical leaders for admitting they were wrong and rapidly adapting their approach.

Minsky says Jane Street had spent years preparing tools for agentic coding but previously found it often slowed critical software work. Recent models, he says, made that preparation more useful: it felt like the wind had finally reached the sails.

Wilson emphasizes that senior people must model the culture by giving others credit and acknowledging their own mistakes. Minsky says long tenure helps preserve institutional knowledge and culture, whose essential components may not always be obvious.

Wilson says Antithesis chose Washington, DC partly to retain engineers and invest in them over the long term. He contrasts that with the Bay Area’s frequent job changes, arguing that an expectation of longer tenure makes such investment easier. Minsky closes by thanking him for the conversation.