Andy Ray on FPGAs, Hardware Design and Hardcaml

Ron Minsky welcomes Andy Ray, who leads Jane Street’s hardware design team and previously built modem and video-codec designs. Ray created Hardcaml, a hardware description system embedded in OCaml. Their conversation asks how chip-design tools can be applied to a wider range of performance-critical problems.

Ray says hardware allows a custom architecture tuned to the particular problem, including fine-grained tradeoffs among performance, cost and power. A CPU is itself one possible hardware design; a product can instead combine processors, digital signal processors and custom blocks in the configuration the workload requires.

Minsky asks about the disadvantages. Ray says the designer must implement the architecture and deal with enormous amounts of detail. Software developers can rely on extensive support libraries; hardware has suppliers of reusable designs, often called IP, but the ecosystem is smaller and generally more expensive.

Some open-source hardware blocks are available, but Ray describes that ecosystem as tiny compared with software. Individual pieces may be logically simple, Minsky observes, yet the static graph of gates and wires becomes hard to understand when a complete system contains very many of them.

Ray explains that NAND is a universal Boolean function: circuits built from NAND gates can represent any Boolean function. Designers generally do not draw every gate by hand, however. A synthesizer translates a somewhat higher-level description of adders, multiplexers and registers into the low-level elements used by the target device.

An ASIC is a custom-fabricated chip designed for a specific function; an FPGA can be reprogrammed to implement different functions. Minsky asks how large the performance improvement can be when a task moves from software to hardware. Ray stresses that it depends on the workload.

Ray describes H.264 video encoding. A highly optimized software encoder could achieve real-time 1080p on contemporary processors only by disabling many features; enabling more compression features made it slower. Hardware implementations, including systems spanning multiple FPGAs, could support more of those features in real time for professional encoding.

For that example, Ray describes roughly an order of magnitude more computation from the FPGA designs. Packet processing is another candidate: a large hardware design can ingest traffic from many network ports and inspect or route packets on one chip. Minsky notes that throughput is not the only benefit; latency and predictability also matter.

A hardware design can be built to consume everything presented on a network link at line rate. Once the design meets its constraints, its timing can be far more predictable than a software path that needs buffering to handle variability. Ray says nondeterminism in software processing is a practical concern at Jane Street.

Minsky contrasts a hardware system with a modern CPU, which must turn an apparently sequential instruction stream into parallel execution. Speculation and prefetching rely on complex heuristics, making exact timing difficult to reason about ahead of time. A synchronous circuit generally does the designed work on each clock cycle.

Ray qualifies that comparison: hardware is not absolutely free of nondeterminism. Clock interactions and low-level network packing can introduce small variations. One useful mental model is a huge number of simple parallel operations that read old signals, compute new values and advance together on a clock tick, although multiple clock domains complicate reality.

External DDR memory introduces further timing variation. Access depends on bank and address-range rules, ordering, and periodic refresh when memory cannot serve requests. Ray says access patterns can dramatically affect usable bandwidth, but he expects this variability to be more manageable than a software system with many additional sources of nondeterminism.

Minsky summarizes the distinction as one of defaults: a hardware design starts from a largely deterministic core but interacts with external components and may intentionally add nondeterminism. Software running alongside other work on a CPU has timing variability that is difficult to eliminate completely.

A concern remains: must a custom chip be fabricated every time the design changes? Ray explains that reconfigurable hardware addresses that problem. An FPGA contains a grid of lookup tables that implement Boolean functions, connected by programmable routing, along with other specialized elements.

An ASIC design goes through complex tooling and fabrication at a foundry, a process that can take months for a first example. With an FPGA, a designer can reset the device and load a new configuration. A bug fix or new version can be deployed to hardware already in the field.

The flexibility has a price. Ray says an ASIC generally wins on power, area and clock frequency for the same design; he gives a rough power range of three to ten times less. The economics depend on fabrication expense, volume and whether the design needs to change, so his figures are illustrative rather than universal.

Minsky finds it remarkable that a programmable chip can approach a custom circuit at all. Even though an FPGA can cost more per unit, it can make hardware practical for small quantities and evolving designs. Without that flexibility, many specialized technology problems would not justify custom silicon.

Ray gives a contemporary purchasing example: an FPGA quoted at around $8,000 individually might cost roughly $500 under a substantial multiyear volume agreement. He links the difference to advance manufacturing commitments and inventory economics; the numbers reflect a particular example at the time of the conversation.

Turning to his background, Ray recalls studying computer science and physics. A short hardware-design course introduced VHDL projects, including a multiplier and a small digital signal processor. The project work appealed to him even more than the course itself.

He first worked in game programming, then joined a tiny embedded software and hardware IP company. After he mentioned an interest in hardware, his boss obtained a contract for a JPEG encoder and decoder on an early Xilinx FPGA. Ray took it on and found that he enjoyed the work.

The first time the design ran on a real card was memorable. It did not work perfectly, but it performed recognizable parts of the intended job after months of design. Ray says that bringing an FPGA design up in a real system remains one of the most satisfying parts of his work.

Minsky asks why hardware is so engaging. Ray enjoys software too, but says the long effort before a complete hardware design runs makes the payoff especially satisfying for him. Unlike a small software library that can be tested and shared early, a hardware project often requires much more of the system before it delivers a visible result.

Ray also likes building mental models at several scales. At one level he thinks about individual components; at another he considers bus widths, clock rates, memory and PCIe constraints. The system-level challenge is to move data from input to output without stalls while using the right amount of resources.

His interest in alternative hardware languages began with frustration at tooling, especially tests. Verilog and VHDL are used to describe hardware, but designers also write test benches at every level of a design. Testing is a software task, and Ray found Verilog an uncomfortable language for building those test frameworks.

Around 2003 Ray found Confluence, a functional hardware language implemented in OCaml, and later Hdcaml, an OCaml package related to those ideas. The prospect of expressing hardware in a full programming language clicked for him. He added libraries and wires, rewrote the system, and also built an F# version for a Windows-based workplace.

After returning to Linux, Ray developed what became Hardcaml, his third iteration. He values the design abstractions, but says the biggest attraction is being able to write test benches in OCaml. Its small set of composable concepts makes him much more productive at writing and testing hardware than he had been in Verilog.

Hardcaml is an OCaml library, technically an embedded domain-specific language. It provides functions for circuit components such as adders, multiplexers and registers; OCaml code wires them into a graph. Tools can then operate on that graph, including a simulator that produces a model during development.

A waveform viewer presents signals across clock cycles so designers can inspect their transitions and relationships. A signal may be a bit or a wider bus. Such a view helps show what a circuit does over time without requiring a physical FPGA for every test.

Jane Street integrated this workflow with its build system and editor. Expect tests print results that are pasted back into test code and checked into the repository. Continuous integration rebuilds the code and identifies changes that break a test, giving hardware designers a familiar software-development loop.

Minsky sees a broader cultural difference: software teams commonly combine build systems, integrated tests, code review and continuous integration, while traditional hardware tools often center on graphical workflows. He compares the challenge with network configuration, where software-defined approaches bring similar automation and quality control to another domain.

Ray acknowledges that expensive commercial tools offer valuable features such as coverage and constrained-random input generation. They can generate inputs matching specified constraints. He nevertheless prefers having a good general-purpose programming language for the test logic itself.

Minsky asks why Ray emphasizes testing more than a fundamentally higher level of circuit design. Ray says Hardcaml and Verilog work with broadly the same underlying kinds of components. Hardcaml’s host language makes generation and configuration more expressive, but the resulting design is still close to the circuit level.

Ray discusses decades of efforts to raise hardware abstraction. He points to high-level synthesis and Bluespec as notable approaches, while questioning how broadly successful the former is. Minsky counters that useful software ideas can also take decades to become mainstream, citing garbage collection as an example.

High-level synthesis translates C into parallel hardware. Ray is skeptical of extracting parallelism from an inherently sequential instruction stream, though familiarity with C helps adoption. He acknowledges that it can be very effective for streaming DSP-style workloads and can produce different architectures from one source using compiler guidance.

Bluespec takes another approach based on guarded atomic actions. A rule has a predicate and an operation that reads and updates state. Its conceptual model chooses an enabled rule; a compiler analyzes dependencies and schedules compatible rules together to obtain parallel hardware.

Ray says Bluespec also presents module interactions in a more composable, object-like style. Instead of manually aligning related data and validity signals, a designer can call a method. For a FIFO operation, the system can delay a rule until the queue is ready, while the compiler generates the underlying circuit.

Minsky contrasts this with Hardcaml’s core circuit representation. OCaml can generate it using modular, generic code—for example, a shared protocol specification can produce interfaces for software and hardware. Bluespec offers a more composable representation of the hardware itself, which must still eventually be lowered to wires and logic.

Minsky asks whether Bluespec’s nondeterministic rule model weakens guarantees. Ray qualifies his answer: he has read more about Bluespec than he has used it. He says the compiled circuit schedules enabled rules, and source pragmas can guide priorities; the practical system includes compromises beyond the elegant core model.

Ray still sees a need for lower-level tools such as Verilog or Hardcaml when exact signal behavior must be controlled, for example at a DDR interface. More abstraction means giving the compiler some choices. He nevertheless expects Jane Street to pursue Bluespec-like abstractions for internal logic, without committing to a particular implementation.

They compare host languages and related tools. Ray mentions Python frameworks that let a system start as a high-level model and refine individual parts down to gates. Hardcaml has hooks for a similar approach, though he says its API had not formalized that workflow at the time.

Chisel uses Scala for a similar embedded approach and benefits from teaching and publicly available IP, including RISC-V designs. Ray argues that functional programming fits combinational logic naturally: functions transform inputs into outputs and can be composed to build larger circuits.

Minsky asks about Hardcaml as an open-source project. Ray wants more people to use it but says adoption requires more than the core library. At the time, only a few outside users had contributed, and the project needed more libraries and realistic public designs that newcomers could study.

Ray says he had opened a number of additional libraries, but releasing full internal designs is harder because the team must separate what can be published from what cannot. Chisel’s public RISC-V ecosystem gives learners a much larger body of example code. Jane Street hoped to release more where practical.

In trading, Ray says Jane Street’s hardware work is strongly connected to network packet processing. The platform he describes is a network interface card with an FPGA: packets can travel to the host through normal drivers, or custom logic on the card can inspect and transform them first.

A generic network card understands standard protocols such as Ethernet, IP and TCP, but it cannot generally interpret arbitrary application payloads. Because Jane Street knows a particular exchange’s packet format, its FPGA design can parse deeper into market data and perform exchange-specific operations.

For example, hardware can split incoming messages into groups for different downstream interfaces, so each software process sees less data. It can also selectively send information across PCIe to the host, reducing both bus traffic and the volume software must inspect.

Minsky notes that US markets can peak at several million messages per second and that firms consume data from multiple exchanges. Ray adds that traffic from several high-speed sources can overwhelm CPUs precisely when markets are busiest. Separate hardware processing paths can handle those streams in parallel.

The goal is to design for the known maximum rate of a link and test with back-to-back packets at line rate. If the hardware meets that bound, the team can be more confident it will not fall behind merely because traffic is busy. Minsky suggests that software systems could also benefit from stronger stress testing of their limits.