Laurent Mazare on Python, OCaml and Machine Learning

Ron Minsky talks with Laurent Mazare about how Python and OCaml fit together at Jane Street. Although the firm is known for OCaml, Python is important for data analysis and machine learning, and the two languages connect to a shared infrastructure.

Mazare began at LexiFi, using OCaml to model financial contracts and their payoffs. At Goldman Sachs he worked with Slang, an internal scripting language; he joined Jane Street in 2013 to develop trading systems in OCaml.

He later spent a year at DeepMind, working primarily in Python and TensorFlow and experimenting with Swift for machine learning, before returning to Jane Street to focus more on research. He now divides his time roughly equally between Python and OCaml, in very different modes of work.

His open-source projects include OCaml and Rust bindings for PyTorch, providing access to GPU computation and automatic differentiation from those languages. He has also contributed to Rust's Clippy and worked on bindings and data-frame-related projects in OCaml.

Mazare uses Python notebooks for research. A notebook lets him run a cell, inspect the result, change the code and run it again. Plotting libraries make it easy to compare algorithm output with intuition and investigate surprises in a tight feedback loop.

His OCaml work tends to be production-critical development. He has seen jobs continue running years after deployment. He attributes resilience to both language features and engineering practices: tests, explicit handling of errors and attention to corner cases.

Minsky separates language, ecosystem and organizational practice. Mazare says Python has become the de facto environment for machine learning and data analysis, with plentiful plotting libraries, tutorials and Python-first interfaces to major frameworks. A large ecosystem attracts still more users.

OCaml has fewer libraries, though Mazare considers many of them high quality. Bindings can bridge the gap: OCaml can call Python's Matplotlib for plots or use bindings to TensorFlow and PyTorch. Minsky emphasizes how visualization makes exploratory notebook work particularly compelling.

Mazare sees fast feedback in both environments, but of different kinds: a notebook displays results on data, while OCaml's build system reports type mismatches as code changes. In Python the researcher relies more on experiments; in OCaml the type checker provides additional feedback before execution.

Minsky points out that OCaml can also support exploratory work through expect tests. Code and captured output live together; a test regenerates output and the developer can accept changes. He uses the loop for tasks such as web scraping, though it lacks the notebook's graphical component.

Mazare agrees expect tests are interactive but says they often focus on edge cases in lasting code. A notebook experiment may instead aim to work on a particular dataset and may never be run again. The goals influence how much generality is worth paying for.

Minsky asks whether Python would retain an advantage if OCaml had equally good notebooks, visualization, editor support and libraries. Mazare says it is hard to know, but dynamic typing can be useful: a small algorithm written for integers may work on floats without changing its operators.

Duck typing also allows an algorithm to work with different objects that offer the needed methods. This flexibility trades away guarantees: the programmer gains confidence from experiments on relevant inputs, but mistakes may only appear at runtime.

Minsky says static types establish properties across executions, but obtaining those guarantees requires fitting code into the type system. OCaml's inference makes that negotiation relatively light; for one-off exploratory research, avoiding it altogether can still be attractive.

Mazare initially describes types as proving code correct. Minsky narrows the claim: a type system proves certain properties, not overall correctness. Those properties can eliminate large classes of bugs, but they do not replace all other reasoning and testing.

Numerical code is a case where types may help less. A matrix or floating-point value can have the right type while representing an incorrect computation; a decomposition may converge poorly or produce a subtly suboptimal answer. Finding these problems often requires examining data and results.

Mazare notes that even determining whether a floating-point value will be NaN is difficult. Experiments can catch issues on the data at hand, whereas checking every conceivable input throughout a numerical program may impose too much friction.

Minsky contrasts quick exploration with long model-training runs: discovering a trivial failure after days of GPU work is expensive. Mazare says researchers commonly test a smaller run end to end, including model and checkpoint output, but an untested branch can still fail later.

A type checker can catch mistakes such as passing an integer where a filename is expected before a long job runs. Mazare suggests a division in which Python remains useful for rapidly iterating on the numerical model, while more resilient infrastructure handles real training and production concerns.

Production infrastructure must account for failures such as unavailable machines, filesystem problems and ordinary programming mistakes. Rewriting a Python model in OCaml can yield speed and guarantees, but it creates two implementations that may diverge; calling Python from OCaml is another option.

Where the firm has maintained separate implementations, a produced model can include both weights and sample inputs and outputs. The OCaml deployment runs the sample values with the saved weights and checks that its results match, providing a practical consistency check.

Mazare describes PyML, an open-source library for interoperability. It can call Python from OCaml, for example to use Matplotlib, but Jane Street often uses it in the other direction: Python notebooks need to reach OCaml services for market data, other data and actions.

OCaml code can be compiled into a shared library loaded by Python. Starting the OCaml runtime registers functions that Python can call, making it possible to provide wrappers with very little handwritten Python. This lowers the burden on OCaml developers who do not work in Python every day.

Minsky compares this to Ecaml, which lets the firm write Emacs extensions in OCaml while interacting with Emacs Lisp. Both approaches pass control repeatedly between runtimes; similar work has also been done for Vim. Mazare says it is striking that many alternating calls can work transparently.

Interoperability introduces memory-management hazards. OCaml uses a tracing garbage collector; Python primarily uses reference counting and a collector for cycles. A cycle spanning the two runtimes can look reachable to each collector even when the combined objects are no longer useful.

Mazare describes a cross-runtime cycle that wasted tens of megabytes until it was found and fixed. Minsky says Ecaml encountered the same class of problem. Neither collector alone has a complete view of the cycle, making it a fundamental integration difficulty.

Asked how he chooses a language, Mazare favors OCaml for code that should last weeks or months and for symbolic manipulation. A syntax extension is an example: types and pattern matching can help ensure that the program handles the language's different syntax cases.

Minsky broadens the example beyond compilers. Many systems, including trading systems, contain combinatorial structures with many possible cases. Exhaustive pattern matching can help developers account for each case in such code.

Sharing computations across languages is another reason to expose OCaml from Python. Minsky says the team can implement a core computation once, benefit from efficiency and reuse it in both environments, rather than maintain two versions. Mazare adds that shared code must withstand uses its author did not anticipate.

For a one-off script, the investment in handling every case may not be worthwhile. For a library called by many people in many contexts, the type system's pressure to consider different cases can justify the effort, though tests still matter.

Minsky argues that types and tests reinforce each other rather than substitute for one another. Static constraints can make behavior more rigid, so tests of key cases may provide broader confidence than they would in code whose behavior is constrained only by tests.

He asks whether Jane Street should improve Python testing and tooling. Mazare says the firm is learning, adding type annotations and generated documentation, and rebuilding practices long refined for OCaml. He acknowledges that its Python practices were not yet as mature as those of dedicated Python organizations.

Mazare suggests that when Python mainly glues together well-tested OCaml components, the shipped Python layer may be small, though the bridge itself needs testing. Minsky cautions that even very short programs can contain surprising bugs.

Python's numerical ecosystem can also hide tricky corner cases. Mazare cites NumPy and pandas: their power makes analysis fast, but unexpected library behavior can quietly produce the wrong simulation result. A large error may lead to a known gotcha; a smaller error may never be noticed.

Minsky compares spreadsheet analysis: a grid shows intermediate computations and enables quick visualization, yet automatic type inference can have dangerous consequences. He cites names interpreted as dates and the possibility of ticker symbols being misread, illustrating how plausible numerical errors can remain unnoticed.

Mazare jokes about Excel from his Linux-oriented perspective but admires expert traders who use shortcuts to pull data from Bloomberg, plot it and share results quickly. Recreating such an analysis in OCaml could take hours. Excel's reactive recalculation is another strength.

Python sits between spreadsheets and production code for him: it handles larger datasets and more computation but loses the ability to inspect all data at a glance. Analysts often rely on summaries and extremes once the data becomes too large to view directly.

Minsky highlights a correctness difference. Excel tracks a dependency graph and recalculates affected cells when values change. A Python notebook lets users rerun individual cells, so displayed results can reflect different execution states and need not be mutually consistent.

Mazare says a notebook's mutable state and out-of-order execution can make it hard to reconstruct how a result was reached. Rerunning everything may be expensive, but if a conclusion cannot be reproduced from the beginning, that is a real problem.

They discuss alternatives. Mazare says Polynote makes state more explicit and tries to restore the context of a cell, though deep mutation and aliasing can escape that model. Julia's experimental Pluto tracks dependencies and reruns dependent cells, with tradeoffs when computations are long.

Minsky suggests an interface could at least indicate which results are stale when a cell changes, even if the user chooses not to rerun expensive work immediately. Making the reliability of displayed results visible would be a useful improvement.

The conversation moves to Swift and differentiable programming. Mazare explains gradient descent: a model has parameters and a loss function, and gradients guide updates toward a local minimum. Minsky likens the process to a ball rolling downhill, extended to many dimensions.

Numerically estimating gradients is costly for models with millions of parameters. TensorFlow and PyTorch build computation graphs and differentiate them; a compiler could instead generate gradients from a program's operations, provided it understands how to differentiate them.

Minsky explains automatic differentiation as applying derivative rules to programs, including control flow. Mazare notes that branches and recursion create challenges, but many modern machine-learning models use operations that can be differentiated and composed.

Swift originated at Apple as an Objective-C successor. Mazare says Chris Lattner explored several languages for machine learning and selected Swift for a prototype; Swift's compiled nature, types and functional features made it an interesting environment for the work.

The compiler project aimed to identify computations suitable for CPUs, GPUs or TPUs and manage data movement between their memories. A function could be annotated so the compiler generated a backward pass, provided the operations it used supported differentiation.

Mazare contrasts Swift's more functional style with Python's dynamic, mutable state: derivatives are easier to reason about for pure functions. He notes that Python frameworks can also compile annotated functions and their backward passes, so the distinction is not an absolute prohibition.

Another consideration is memory management. Large tensors can occupy substantial GPU memory, making prompt release valuable. Mazare says Swift's reference counting, like Python's, can help release resources quickly, while tracing garbage collection may not run at the moment scarce GPU memory is needed.

Minsky notes that garbage collection can be poorly suited to scarce resources outside the ordinary heap, such as file descriptors or GPU memory. He discusses exploratory work toward type-checked resource management in OCaml, with explicit tracking where needed and garbage collection as the default elsewhere.

Mazare contrasts his Rust PyTorch bindings, where ownership machinery helps release tensors, with OCaml code in which he triggers garbage collection on each training loop to reclaim resources. More precise control over selected values would be useful.

Minsky stresses that the proposed OCaml approach is future work, not a finished capability. Both view resource management as an active design space: a system that combines convenient defaults with precise tracking for scarce resources could bring benefits without imposing the same burden everywhere.