Why File Systems Slow Down Coding Agent Workflows

I've been running more coding agents in parallel, and some old frustrations with my MacBook have become much harder to ignore. Its hardware is excellent, but a workflow that repeatedly creates worktrees, installs packages and deletes directories exposes a performance problem I wanted to investigate myself.

In one cleanup run, deleting project folders took more than 35 seconds on my Mac; comparable runs on my Linux machine took roughly 7 to 12 seconds. In another demonstration, a cached pnpm install finished on Linux in under 10 seconds while the Mac was still running. These are different machines and configurations, so the comparison illustrates my experience rather than isolating one cause.

The Mac install eventually finished in about 35 seconds even though packages were already cached locally. Much of the time involved post-install scripts. I suspect the way the operating systems and filesystems handle lots of small files is particularly important for the agent workflows I now use.

Apple has achieved impressive things with filesystems: the transition from HFS+ to APFS was delivered to iPhones with little visible disruption. My complaint is about a specific set of tradeoffs, not a lack of engineering skill. Repeated worktree creation and large Node dependency trees put pressure on operations that might be less noticeable in ordinary use.

NullVoxPopuli's disk performance benchmark uses a monorepo of web-framework projects, measures a cached pnpm installation and times a Git cleanup. In the results I show, an AMD Linux machine using ext4 cleaned in about six seconds and installed in about 4.5, while an M1 Max Mac took about 31 and 44 seconds. Hardware and setup differ, so this is not a controlled filesystem-only comparison.

A filesystem is the system that maps the files and directories we see to data and metadata on a storage device. The drive itself holds blocks of data; the filesystem tracks where things belong and what happens when we create, move, copy or delete them. Different implementations make different choices about those operations.

Windows commonly uses NTFS, and Apple uses APFS after HFS+. Linux offers many choices, including ext4, XFS, ZFS and Btrfs. They all provide ways to organize stored data, but their features, platform support and performance characteristics differ. Some also offer features useful for managing multiple drives or redundant storage.

Those choices affect the tasks a developer actually waits for: cleaning up a huge dependency tree, creating a checkout, searching through files and managing metadata. My conclusion from daily work is that the combination I use on macOS is poorly suited to the volume of small-file operations my agent setup performs.

The benchmark also includes a Linux machine throttled to 550 MHz whose installation time was close to that M1 Max result. That's a striking observation, but CPU frequency alone does not control for every difference between the machines. My interest is practical: which setup makes this particular workflow faster and less wasteful?

I care about several kinds of speed: deletion, creating worktrees, indexing and lookup, especially when a project contains many small files. I also care about metadata, operating-system support and the ability to avoid duplicating data unnecessarily. A fast sequential transfer benchmark alone wouldn't answer these questions.

Copy-on-write is one way to avoid immediately duplicating data: copies can initially share storage and diverge when modified. Hard links are another mechanism, but they are not the same thing. The exact result depends on what the application requests and what the filesystem and configuration support; ext4 does support hard links.

Compatibility matters too. A format that's practical on one operating system may be awkward to use on another; for a portable drive, broad support can matter more than the features I want on a development machine. My hope of simply using a different filesystem for part of the Mac's internal storage was frustrated by the support available to me.

Compression introduces another tradeoff: it can save space, but it also consumes processing time when data is written or read. Space is surprisingly difficult to measure when copies share blocks or data is compressed. The apparent size of files does not necessarily equal the physical space used on the drive.

The ideal for my workflow is simple to describe: quick worktree creation, quick pnpm installs and cleanup, and less space wasted on duplicate project data. I asked people who run home labs and development machines what they were using, including Wendell from Level1Techs.

I described the pain of many worktrees and node_modules directories, along with the benchmark results and my concerns about copy-on-write behavior. Wendell suggested XFS plus LZ4. That sent me toward a setup with XFS on a VDO layer and prompted a much more detailed comparison.

Think of the storage device as blocks beneath a layer that maps names and file metadata to data. A compression and deduplication layer can change how much physical storage repeated or small pieces of data consume. That extra layer also needs computation, so it is worth testing the performance cost rather than assuming it is free.

VDO sits below the filesystem and can compress and deduplicate blocks. It is a different mechanism from filesystem copy-on-write or hard links, even though all can reduce redundant storage in the right workload. I was concerned that the extra CPU work would cancel the benefit, so I measured the setup against other options.

The candidates I focused on were ext4, XFS, ZFS and Btrfs, alongside APFS on the Mac. Each has different features and a different performance profile. ZFS and Btrfs have useful capabilities for storage management, but their suitability still depends on the actual workload and configuration being tested.

For one T3 Code worktree in my test, APFS took about 1.26 seconds; ext4 and XFS were around 0.64 seconds, Btrfs around 0.65 and ZFS around 0.78. When creating eight worktrees in parallel, the Mac result rose substantially, while the ext4 and XFS results shown stayed much closer to their single-worktree times.

Storage also differed across the clean and installed worktrees. The chart shows about 0.239 GiB per clean APFS worktree and 0.128 GiB for XFS with VDO; results for other configurations varied. These are measurements in this test, and interpreting shared or compressed storage requires care about what each measurement counts.

Installing dependencies changed the picture again. In the displayed test the installed worktree was about 0.349 GiB on APFS, 0.604 GiB on ext4 and 0.195 GiB for XFS plus VDO. Package-manager linking, filesystem features and measurement methods all affect the result; the numbers aren't universal properties of the formats.

For the separate pnpm monorepo benchmark, the displayed APFS cleanup and install were approximately 39 and 37 seconds. Ext4 was about 2.5 and 8.8 seconds, and XFS about 7.2 and 9.4 seconds. XFS with VDO was close to plain XFS in this particular run, though the systems being compared were not identical.

Changing pnpm to hard-link mode changed the results. In the run I show, the APFS install rose to roughly 100 seconds, while the XFS plus VDO cleanup fell to under three seconds. This is why it's useful to measure the package manager's actual mode instead of treating a filesystem name as the whole setup.

I use Btrfs for other storage, including video files headed to my editor, so I am not saying it has no value. Here I was trying to minimize the delays that happen when an agent gets a new worktree and its dependencies. In my day-to-day T3 Code experience, the difference was roughly 10–12 seconds versus under two.

For transparency, I did most of the later scaling work after narrowing my attention to APFS, ext4, plain XFS and XFS with VDO. My Linux drive had filled up with logs, and the experience of finding and deleting them pushed me to reconsider the storage setup instead of simply keeping the default.

On my own data, I report saving around 44% of storage after switching to the XFS and VDO partition. That is a result for my files and configuration, not a guaranteed saving. The important question is whether the reduction in space comes with acceptable performance for the work I do.

In the scaling chart, 125 installed T3 Code worktrees used about 75.5 GiB on ext4, 43.68 GiB on Mac APFS and 24.4 GiB of physical storage on VDO. The Linux and Mac checkouts were nearby but not identical commits. As the number of worktrees grows, those differences add up, though they depend on the contents and how space is measured.

The hard-link test also narrowed the cleanup gap between ext4 and XFS plus VDO: the latter dropped from around seven seconds to 2.85 seconds in the run shown. After all this testing, Wendell's suggestion has worked well for me. That result reflects the particular combination of filesystem, storage layer and package-manager settings.

In T3 Code I demonstrate starting an agent session that first creates a worktree and installs dependencies. On my Linux box, the prompt gets underway almost immediately; repeating the action on the Mac takes noticeably longer. It is an illustration of the end-to-end experience, not a controlled comparison of identical hardware.

The default ext4 Ubuntu installation had already felt much faster than my Mac, so I had not planned to change it. I expected compression overhead might erase the benefit of VDO, but in my tests it remained fast while reducing space use. I now run this work on a separate Linux drive using XFS and VDO.

I don't think we necessarily need a brand-new filesystem just for agents. Existing systems have different strengths, and the best choice depends on the workload. For me, moving development to a Linux machine on my home network and tuning its storage made the frequent setup steps much less painful.

For people who want to stay on a Mac, I also mention Rift by Anomaly, an experimental worktree alternative that virtualizes files and uses copy-on-write ideas. It can make creating a workspace feel immediate. At recording time I noticed its latest commit was two months old, so I was unsure how actively it would be developed.

I'm hopeful that approaches like Rift or a successor will improve the experience, but for now I'm using the filesystem arrangement that works for me. I spent a lot of time benchmarking because I couldn't find this particular information in the form I needed. I hope the measurements help others test their own agent workflows and make better-informed choices.