Jane Street on GPU Infrastructure, Trading and Hiring

Dwarkesh Patel visits a Jane Street training data center in Texas with Ron Minsky, co-head of technology, and Dan Pontecorvo, head of physical engineering. Patel asks how GPU-based models fit into a business where some trading decisions must happen in nanoseconds.

Minsky explains that trading has many time horizons. In one extreme, a system must turn around a packet in under 100 nanoseconds. At that speed a CPU is not enough: an FPGA attached directly to the network can begin sending a response before it has finished receiving the incoming packet.

Such a fast system can perform only simple computations. The broader strategy combines decisions made at different speeds, from hundreds of nanoseconds through microseconds and milliseconds to an hour or a day. More time can allow more sophisticated models or decision processes.

Asked what the models predict, Minsky names fair value as one important target: estimating what an instrument is worth can feed into many trading processes. It is not the only prediction target, and he avoids discussing more sensitive details.

Inference may run on CPUs, FPGAs or GPUs depending on model size, compute needs and latency. At the fastest horizons, physical distance is so consequential that even fiber length matters. Larger, slower models allow more freedom to locate compute farther from an exchange.

Pontecorvo says that exchange colocation facilities impose their own constraints on power and cooling. A GPU may consume enough power that it must be spread across racks instead of being concentrated in a liquid-cooled rack designed for the purpose.

Patel asks about Jane Street’s $6 billion compute agreement with CoreWeave. Minsky says the firm wants more capacity for training and experimentation. Its researchers explore a diversity of architectures, and faster iteration gives them more chances to test ideas.

Minsky says specialized models are useful because applications consume different sources and rates of data. Financial data is noisy and less informative per byte; compared with some other settings, there can be more data and smaller models. Storage, loading, model architecture, inference throughput and latency all affect the design.

Asked how inference differs from a large chatbot workload, Minsky says latency is particularly important, though batching still matters. A market feed can deliver a very high sequential data rate in a single causally ordered domain. Similar engineering questions arise in both settings, but different constraints lead to different choices.

Data loading performance receives particular attention. Minsky describes work on an internal object store for research at large scale and across multiple data centers. Training compute cannot all be placed in one location, so moving the large datasets and scheduling storage alongside computation are important problems.

For years the firm simplified its environment by relying on x86-64 and one major research data center and storage cluster. Both assumptions have changed: power constraints require distributed facilities, and newer NVIDIA products require support for ARM. A useful organizational skill, Minsky says, is knowing which simplifying assumptions one can safely make.

Patel asks whether sufficiently capable AGI could simply do Jane Street’s work. Minsky does not dismiss a future in which systems surpass humans at all cognitive tasks and automate much of the business, but says he does not think that world is especially close. He argues that complex company-scale work is easy to underestimate.

Minsky calls trading “AGI complete” by analogy with hard computational problems: valuing assets requires predictions about the future, and many different aspects of the world affect those predictions. As some tasks become automated, competitive advantage can shift to the hard problems that remain. He says he wants more engineers and traders, not fewer.

Jane Street’s business includes many kinds of software and trading, some of which is not highly electronic. Minsky describes transactions negotiated over chat or phone, where people assess the other side and the risk of adverse selection. Bond trading, he says, remains less automated than equities despite decades of expectations that more markets would follow the same path.

Patel asks about human involvement between models and decisions. Minsky says unusually turbulent days can create both opportunities and difficult liquidity decisions. Humans can assess how a situation differs from previous days and supervise automated systems, even when individual transactions occur too quickly for a person to intervene one by one.

Pontecorvo says data centers and cooling now attract much more attention than they once did. Operators are also reconsidering old assumptions: instead of backing up an entire facility with long-lead-time generators, a team might protect only its critical systems and bring GPUs online six months earlier. A choice that is not ideal on one engineering dimension may be the better business decision.

Beyond compute and memory, Pontecorvo identifies generators, transformers and liquid-cooling equipment as current bottlenecks, while cautioning that shortages change quickly. His team works with procurement to stock interchangeable components, though large generators and turbines require more deliberate sourcing and staging.

As racks become denser, Pontecorvo hopes buildings can be smaller and built faster. Modular infrastructure can be designed and constructed offsite, then delivered closer to plug-and-play. Dense racks still require substantial power and cooling infrastructure; a one-megawatt rack is useless if the facility cannot support it.

Planning has to precede chip orders because infrastructure lead times can exceed a year. Pontecorvo says different accelerators have different cooling and density requirements, so the team works with suppliers to prepare facilities for future generations of equipment.

Patel asks what Jane Street does with spare capacity, comparing it with a large platform improving its advertising models when GPUs are otherwise idle. Minsky says the firm is compute constrained: it evaluates research runs but already turns away valuable experiments. More frequent retraining and bulk inference can also use additional capacity.

Minsky acknowledges the investment involves risk: model and trading value might not improve as expected, and competitors may erode an advantage. Still, given the current workload, he says the firm is far from having too much compute. Pontecorvo adds that a company can commit to power and data center capacity before buying expensive chips, and potentially transfer excess facility capacity to someone else.

Patel suggests hiring researchers creates competition for scarce compute. Minsky sees it differently: valuable researchers are a reason to buy more compute. He says the firm currently has tens of thousands of GPUs and expects hundreds of thousands before long, a forward-looking estimate rather than a present count.

Minsky says hardware is not the main obstacle to hiring. Finding strong people and having enough mentorship capacity to train them and integrate them into the culture are more limiting. The firm wants to grow while remaining a good place to work.

Pontecorvo lists physical engineering roles spanning site selection, design, construction and operations: mechanical, electrical and structural engineers, architects and project managers. Minsky adds that machine learning and trading require people with diverse backgrounds, including model architecture, LLM training, mathematics, computer science, physics and engineering.

Jane Street also seeks generalist software engineers and specialists. Minsky says its increased compute spending makes fleet-wide optimization more valuable than when the firm concentrated primarily on its most latency-sensitive code. Experience at hyperscalers can be relevant, even though the firm’s problems have distinctive features.

Minsky mentions custom ASIC work and an exploratory effort to build a formal methods team. He describes the latter as speculative but newly interesting in light of AI. He also emphasizes frontend engineers who can build technically sound applications that work well for people.

Minsky argues that enthusiasm for AI tools should not obscure the human element. The aim is to build systems, including AI tools, that improve people’s understanding, agency and efficiency. The organization’s ability to find and develop excellent people remains central to what it can accomplish.

Patel points listeners to Jane Street’s puzzles, including a challenge involving hidden trigger phrases in language models that neither entrants nor Jane Street had solved. Minsky says puzzles are part of the company’s culture and a way to reach potential candidates. The conversation closes with a pointer to the firm’s page for the podcast audience.