Matt Godbolt: Inside Skylake’s Out-of-Order Pipeline

Matt Godbolt is introduced as a C++ developer who investigates compilers, operating systems and silicon, and maintains Compiler Explorer. He says he enjoys reverse-engineering processors, including the branch target buffer, and will focus on Skylake because he could not readily test the newest machines.

An older textbook pipeline fetches, decodes, executes and writes back instructions in sequence, overlapping stages to complete useful work each cycle. A modern processor instead has a front end, an out-of-order back end and retirement. Independent operations can execute concurrently when their inputs and execution units are ready; retirement restores program order.

Skylake’s front end contains fetch and decode paths, a micro-op cache, a queue and a renamer. Complex x86 instructions are broken into simpler internal micro-operations. Godbolt cautions that much of the diagram is inferred by a community of researchers rather than directly documented by Intel.

The physical register file contains many more slots than the architectural registers exposed to a program. A register alias table maps names to current physical values; a reorder buffer tracks unretired operations, while reservation stations hold work waiting to execute. Branch and memory-order buffers help manage speculation and memory dependencies.

To make the stages concrete, I use a C++ loop that squares integers in an array and adds them to a running total. It compiles into six instructions occupying exactly 16 bytes. The assembly is a textual representation; the machine actually reads the bytes alongside it.

The x86 instructions in that loop have different lengths. Across the instruction set they can range from one to 15 bytes, with prefixes and decades of compatibility making boundaries difficult to identify. That complexity is one reason the front end needs substantial decoding machinery.

A modified version of Andreas Abel’s uiCA lets me illustrate an instruction’s progress through the pipeline over time. In the example, pre-decode, queueing, issue, dispatch, execution and retirement occupy different cycles. The diagrams are a model built from measurements, not a direct view into Intel’s implementation.

The fetcher follows a predicted instruction pointer rather than waiting for certainty. It picks up 16 bytes at aligned boundaries, even before decoding has established whether those bytes contain a branch. Branch prediction must keep the fetcher moving, and a jump into the middle of a block can waste fetch bandwidth.

Pre-decode tries to find instruction boundaries so several instructions can be decoded in parallel. Godbolt describes the hypothesis that hardware speculatively considers possible offsets and filters overlapping candidates. He emphasizes that the representation shown on his slide is an educated guess, not a canonical internal format.

The pre-decoder can also recognize a common compare-and-branch pair as one internal unit, called macro-fusion. It can process up to five instructions per cycle, subject to restrictions. Length-changing prefixes can impose a penalty, and instructions identified as complex must use the appropriate decoder.

Four decoders accept partially decoded instructions in order. Macro-fusion combines two architectural instructions; micro-fusion is different: a memory read and an arithmetic operation can travel together as one packed micro-op before later separating for execution. The distinction affects how much queue and buffer space the work consumes.

A microcode sequencer supplies more elaborate sequences for instructions that require many operations, including locked operations, repeated stores and integer division. It can emit a sequence or contain control logic. Godbolt notes that his individual-operation diagrams are empirically inferred, so the exact internal work is not always known.

The first decoder handles complex instructions and up to four micro-ops; the other three handle simple single micro-ops, including fused ones. A sequence of two-micro-op instructions can be particularly unfavorable because each needs the first decoder while the others sit idle. The micro-op cache helps avoid repeating this work.

The micro-op cache sits alongside the legacy decode path. As decoded operations pass through, they can populate it; a jump to cached code can switch delivery to that cache. This is not described as an ordinary lookup on every instruction: the front end streams from one source or the other.

Skylake’s cache has 32 sets, eight ways and up to six micro-ops per line, with some operations occupying two slots. At most three ways can represent a 32-byte block, and a branch ends a line. Multiple entry points into a small code block can therefore reduce cache effectiveness; newer designs have different limits.

Aligning loops to 16-byte boundaries can help fetching. Godbolt reports that literature describes delivery of up to six micro-ops per cycle from the cache, although his own measurements reached four. He treats the difference as an observation rather than pretending to have resolved it.

The loop stream detector can reuse a loop already present in the micro-op queue, allowing the front end to stop fetching and decoding it repeatedly. It can deliver up to four micro-ops per cycle and may unroll a short loop internally to use that bandwidth more efficiently.

On Skylake, however, Intel disabled the loop stream detector after a severe bug involving short loops and high-byte registers. Godbolt says members of the OCaml community found the issue; a microcode update turned the feature off. It is a reminder that even sophisticated hardware can have surprising defects.

The renamer records speculative work in the reorder buffer, maps architectural registers onto physical registers and sends operations toward execution units. An operation waits for its dependencies and a suitable unit. Skylake can perform this complicated bookkeeping for four micro-ops per cycle.

Some operations undergo “unlamination” at this stage, producing additional micro-ops and reorder-buffer entries. The Skylake reorder buffer has 224 entries, limiting the work in flight; Godbolt says newer processors have expanded that capacity substantially. The buffer tracks completion and exceptions so retirement can proceed in order.

Issuing also selects an execution port. Even if several units can perform an add, the choice is made before the operation becomes ready, so the chosen port may not later be the least busy. Researchers have reverse-engineered an approximate balancing algorithm from observed behavior.

A micro-fused memory-and-arithmetic operation separates into a load and a dependent arithmetic operation for the execution units, while retaining one reorder-buffer slot. Godbolt distinguishes this from unlamination, which creates additional buffer entries. The scheduler holds pending work; the reorder buffer is the ledger for completing it in order.

Godbolt is uncertain whether ready operand values are stored directly in scheduler entries or referenced elsewhere. Results appear to be broadcast to dependents, but the number of apparent register reads raises further questions. He presents the diagrams as a useful model rather than a claim to know the exact circuitry.

Renaming removes false dependencies on reused architectural names. A register alias table identifies the current physical register, and a free list supplies a new one when an instruction fully overwrites a register. Reads use the old mapping; writes update the mapping, so unrelated uses of the same name can proceed independently.

In the example, a load writes EAX into a fresh physical register, and a subsequent multiply reads that value and writes another physical register. The resulting micro-ops express real dependencies through physical names. The next loop iteration can start without waiting for an unrelated earlier use of EAX to finish.

In Godbolt’s illustrative model without renaming, the loop takes about ten cycles per iteration because false dependencies serialize work. With renaming, the load and pointer increment can overlap and the modeled throughput is about one and a half cycles per iteration. He notes that real code might use SIMD instead.

The familiar `xor eax, eax` sets a register to zero and has a compact encoding. The processor also recognizes it as a zeroing idiom: the renamer can map the result to a known zero without sending work to an execution unit, although the operation remains tracked for retirement.

A register-to-register move can likewise be eliminated by mapping two architectural names to the same physical value. Aliasing complicates knowing when that physical slot can be freed. Godbolt describes a limited number of alias slots on older designs and notes that later designs appear to handle this more flexibly.

On newer processors, he says, small increments can sometimes be represented as an offset in the rename machinery instead of immediately executing an add. The offset has a limited range and eventually must be materialized. A shift can expose an extra cycle when it consumes such a value; his proposed explanation is a hypothesis about timing, not confirmed circuitry.

The front end has now turned predicted instruction addresses into decoded, renamed operations. The reorder buffer holds their program-order ledger, while pending operations live in schedulers. In the back end, ready operands and an available assigned execution port determine when work can actually run.

Execution units are themselves pipelined, with different ports supporting different operations and latencies. When a result completes, it is written to the physical register file, marked complete in the reorder buffer and made available to dependent work. Choosing a less versatile port can leave more flexible ports available for other instructions.

Memory ordering is more complicated. The store buffer holds speculative stores until retirement makes it safe to commit them, while later loads may need to see those values before they reach memory. The machine therefore tracks both addresses and data rather than simply delaying every memory operation.

A store’s address and data can become ready at different times. If the address is known early, a later load to a different address may proceed even while the store’s data is still being calculated. Separating these pieces creates more opportunities for parallel work.

The load buffer tracks outstanding loads and the processor predicts whether a load depends on an earlier store. A wrong prediction can require expensive recovery. If the matching earlier store is known and no unresolved store intervenes, the load can receive its value through store-to-load forwarding without going to L1.

If no earlier store can affect a load, it can go to cache; if the situation remains ambiguous, it may have to wait. Fences and locked operations impose additional ordering constraints. Godbolt stresses that parts of his account are simplified and that some details are not established by the available measurements.

Execution can also encounter exceptional cases. Godbolt describes floating-point assists for very small denormal values: in his understanding, the hardware may have to flush and use a microcode sequence when the ordinary unit cannot handle the value. This can create a severe performance cliff.

Write-back places results in physical registers and marks operations complete. Retirement then advances through completed work in program order, handles machine exceptions and frees an older physical register mapping once it is safe. A retired store remains buffered until it can commit to memory.

Godbolt is less certain about the precise point at which read-for-ownership and cache interactions occur as stores commit. His practical advice is to keep work simple, be wary of integer division, use small aligned loops and let renaming do its job. He adds that discussions with the audience have complicated his earlier assumptions about the need for more registers.

Asked about physical register widths, Godbolt says the register file is divided by type: flags, general-purpose and vector registers have different pools. He describes literature suggesting that full-width AVX registers are allocated selectively, while acknowledging that he has not personally verified all those details.

He also discusses partial-register writes. In earlier designs, a write to a narrow portion could require combining it with the rest of the old register value. His understanding is that newer processors handle this differently, though the details are not fully visible.

On differences between Intel and AMD, Godbolt says he has concentrated on Intel and cannot confidently compare all aspects of the back end; he does know their branch predictors differ. Asked whether instruction-set design explains ARM performance, he suggests modern decoding and implementation details make a simple answer difficult.

How can anyone discover these internal mechanisms? Godbolt credits Agner Fog and other researchers for experiments using performance counters and precise timing. His own branch work compared forward and backward branches and early pipeline redirects to infer behavior that is not directly described in public documentation.

Repeated operations and long dependency chains can reveal which ports execute an instruction or when a resource becomes exhausted. He points to Travis Downs’s uarch-bench and work by Andreas Abel among the tools and research in this area. A virtual machine may expose too few hardware counters, so many experiments require physical hardware.

Asked why Intel does not disclose everything, Godbolt cannot give a definitive answer. He suggests that published behavior can become a compatibility promise, while intellectual property and the cost of documentation may also matter. Some large customers may receive more information than is publicly available.

He describes research into Skylake’s branch predictor that found an address bit effectively separated prediction resources. Researchers used that observation to construct separate prediction domains for generated code and its supervisor, suggesting a way to address a particular speculative-execution concern. Godbolt notes the tension: once such a detail is public and relied upon, changing the design becomes harder.