Jane Street and the OCaml Compiler

Ron Minsky describes Jane Street's journey from an ordinary user of OCaml to a contributor to its compiler and ecosystem. The firm adopted an unusual language it did not create, so its relationship with the community differs from a company controlling a language of its own.

Minsky says he arrived without compiler-implementation expertise; his background was in distributed systems. He had learned OCaml in graduate school and began using it for quantitative research after finding Python too slow, assuming the code was temporary and could later be rewritten.

Instead, he stayed, and roughly 80,000 lines of research code became the beginning of the firm's OCaml codebase. He calls much of that early code terrible. What began as a tactical choice eventually required a more deliberate organizational decision.

The firm needed to replace infrastructure that had largely been carefully reviewed VBA behind spreadsheets. The initial OCaml experiments worked well, and the company chose it for production trading systems, valuing readability as well as the resulting systems.

Around 2005–06 roughly ten people programmed in OCaml; by 2009 there were about thirty. In 2010 traders began learning it too. Traders are technically sophisticated, Minsky says, but people who program part-time place different demands on the language and tools than full-time developers.

As the organization grew, it created a dedicated tools-and-compilers group in 2015. Work on developer tooling and the compiler became a formal investment rather than occasional contributions. Minsky presents the numbers as a snapshot of the firm at the time of the talk.

The language affects performance and its predictability. Minsky contrasts straightforward code with SQL, where the optimizer and data statistics can determine a plan with very different costs. A programmer cannot always infer performance from source text alone.

Safety matters when programs connect both to markets and to a firm's money. Tests are essential, but expressive type systems can also catch classes of mistakes and make changes safer and faster. Their value depends on how developers use them.

Usability and discoverability also matter, particularly as a codebase grows and users range from full-time developers to traders programming part-time. Interactive help and navigation became more valuable as more people needed to understand unfamiliar systems.

Minsky divides his examples into features the firm did not know it needed and improvements it later pursued itself. First-class modules were an unexpected addition that allowed modules, normally a separate static language of types and values, to be packaged as ordinary values.

OCaml modules expose types and operations, and functors transform modules. First-class modules allow matching modules to be placed in a table, selected at runtime and combined dynamically, bringing powerful module abstractions into a more flexible setting.

The feature helped the firm rewrite a trading system that assembled a computation graph to estimate security values. Its manual wiring grew so complicated that developers feared extending it. First-class modules made component responsibilities easier to express and connect, though a solution without them was possible.

Generalized algebraic data types, or GADTs, extend ordinary algebraic data types. In a pattern match, a particular case can reveal additional type information. Minsky had initially assumed this would mainly help people designing domain-specific languages, not systems programmers.

He found unexpected uses in performance-sensitive systems. A packet processor may need direct control over memory and minimal allocation, while developers still want rich type-level structure and safety. GADTs can provide a typed interface around a representation chosen for speed.

GADTs can also express existentially quantified types: a value has some particular type even if the caller does not know which one. Minsky says this became useful for high-performance message-oriented systems, another application the team had not anticipated.

Merlin was another tool the firm had not requested. It supplies IDE-like features such as discovering a value's type, finding its definition and opening documentation. Minsky admits he once dismissed such tools because the existing environment felt adequate.

He calls this a form of habituation: developers' mental models are shaped by their current tools. Navigation also became genuinely more important as the organization and codebase grew, especially for newcomers or people who program only part of the time.

The next unexpected improvement was PPX, a metaprogramming system. Metaprogramming generates code from code; it can remove repetitive and error-prone work, though generated programs can be harder to reason about.

The older Camlp4 system used an extensible parser. Jane Street used syntax extensions to generate S-expression serializers and deserializers automatically from type declarations. That eliminated substantial hand-written code, but unrestricted syntax extensions conflicted with tooling.

If different projects change concrete syntax arbitrarily, an editor, formatter or syntax highlighter cannot rely on a shared grammar. Minsky says Merlin had to recognize popular extensions through its own modified parser, illustrating the resulting technical debt.

PPX instead transforms OCaml abstract syntax trees into abstract syntax trees. The language provides shared extension points and annotations that transformations can interpret, preserving a common concrete syntax. Minsky compares the approach to Lisp macros while stopping short of calling OCaml's metaprogramming perfect.

He credits other members of the OCaml community with recognizing the importance of a coherent tooling story. He also praises the language's comparatively conservative evolution, while noting his own reservations about older features such as the object system.

Jane Street moved from discussing problems with compiler developers to hiring people with relevant expertise and working on the compiler. One project, Spacetime, aimed to provide precise memory profiling: which allocations remain on the heap and the call paths that produced them.

A source location alone can be unhelpful when a shared list-construction function accounts for many allocations. A stack trace provides more context, but storing a full trace on every object would be costly. Spacetime uses a tree of traces and compact identifiers to reduce that cost.

Even with optimization, precise heap profiling and analysis remain expensive; Minsky says the firm uses machines with extra memory to analyze runs. Sampling is cheaper when exact information is unnecessary, and the team is exploring a range of profiling options.

The firm also worked on debugging. At the time, GDB could show OCaml stack traces, but inspecting values or calling program functions in a stopped process was limited. The project involved compiler instrumentation and debugger extensions, with some changes hoped for in OCaml 4.08.

Flambda is an intermediate representation designed for optimization, especially inlining. Compilers transform programs through layers of representations; expanding a function call can make a larger region visible to analysis, enabling optimizations beyond eliminating the call itself.

For example, one function may allocate a tuple that another immediately deconstructs. With enough visibility, the compiler can remove the allocation. Minsky says the goal is not just faster average code but predictable conditions under which allocation disappears, important for latency-sensitive systems.

Flambda was a multiyear investment. At the time, the team often saw programs run 10–20% faster without extra work; Minsky hoped a more sophisticated second version would make it easier to write reliably allocation-free code. He presents that future improvement as a goal, not an achieved result.

An audience member asks whether Flambda and Spacetime conflict. Minsky says they mostly interoperate, though using both made compilation very slow for technical reasons involving instrumentation and the cost of optimization. He regards that as an engineering problem rather than a fundamental incompatibility.

Returning to GADTs, Minsky notes that mature language features and their implementations do not always age equally. Parts of the compiler were written before current best practices were established, and some less-used paths had rough edges.

A concise inference feature worked for ordinary variants but not GADTs. The reason, he says, was two implementations of pattern matching: the GADT-capable one had bugs and performance problems. Engineers repaired it and made it the single implementation, bringing the features together.

The lesson is not that every improvement requires a new theoretical idea. A company can contribute engineering time to fix unglamorous problems that a research-oriented community may not have the resources to polish, alongside work on genuinely new features.

Modular implicits were one longer-term project. Minsky compares them to Haskell type classes: type information can select an appropriate implementation implicitly, allowing more general code with less explicit plumbing. He says the design and engineering work may take years.

Typed algebraic effects are another proposed direction. Effects can represent operations such as mutation or network communication; type-level tracking could distinguish effectful and pure code, while effects can also support flexible concurrency mechanisms.

Minsky notes that some team members helped develop the original proposals, although the feature was not necessarily an internal priority. He sees theoretical advantages but remains unsure whether they justify the added language complexity. The outcome was unresolved at the time of the talk.

A broader priority is usability across the ecosystem. Minsky recalls interns finding Jane Street's internal OCaml tooling stronger than what they had used at university, while externally developed tools such as Merlin and OCamlformat also benefited the firm.

Build tooling provides a collaboration example. The firm had an internal system called Jenga and wanted to avoid rewriting build rules when releasing open-source packages. Jeremy Dimino began a portable interpreter for JBuild files, initially without intending to create a full build system.

The new tool was unexpectedly fast: Minsky says some end-to-end builds ran about five times faster than with OCamlbuild. It gained users, evolved from JBuilder into Dune and became a collaborative build system. At the time the firm planned to replace its internal engine with it.

Minsky values sharing suitable infrastructure with outside contributors while recognizing that some tools benefit from close internal integration. A healthy ecosystem matters to the firm: it depends on OCaml and needs talented people to want to work with the language.

Other priorities include simplifying compiler and language implementation and improving performance engineering. Minsky sees this as a long-term responsibility, not merely a way to meet the firm's immediate needs.

At the time, he says, Jane Street had roughly 12 million lines of OCaml and hundreds of users. That dependence gives the firm a strong incentive to help the language thrive. He stresses that Jane Street does not own OCaml; the wider community and maintainers make the decisions.

In questions, Minsky says polymorphic variants are old and were used early at Jane Street, but the firm has used them less as other ways to achieve concision appeared. He questions whether the feature's benefits justify its complexity.

Asked about native compilation, he says OCaml already had a good native compiler before Jane Street's involvement. Its straightforward compilation could produce efficient code even without aggressive optimization. He also credits the independently developed Js_of_ocaml backend for bringing OCaml libraries to JavaScript.

Asked for resources to learn compiler internals, Minsky recommends Benjamin Pierce's Types and Programming Languages for compilers and type systems generally. He says he does not know of a comparably good guide to the OCaml compiler itself, and considers that a gap.