Ayush Bhardwaj: Agent workarounds are feedback for your CLI

I built a command-line tool that lets coding agents retrieve context from previous sessions. Watching agents use it taught me that their workarounds are feedback: they can silently write a program around a tool rather than report a bug.

I extracted about 20,000 calls involving 14 command-line tools from my transcripts. Around a quarter piped output into filters such as head, tail, grep or Python; the proportion was higher for read-heavy commands. These are observations from my own sessions.

Pipelines can also hide failures. Without pipefail, a shell pipeline normally reports the final command’s exit status, so a successful head or tail can mask a failed test. Trimming output can remove the actual error too; I recommend a concise failure summary and a path to the full log.

My chat-history CLI searches past sessions from several coding tools. In a smaller audit of my own usage, agents often opened a session and immediately filtered it again. That repeated cut suggested the interface should offer better selection rather than making the agent rebuild it each time.

One full session was around 1.2 million characters, while the messages around a match were about 3,000. A generic size cap limits volume but does not identify the relevant section, so I added ranges, matching messages and recent-message options.

Repeated Python processing of JSON suggested a compact search format with fewer fields; repeated loops over sessions suggested a batch inspection command. The commands agents wrote around my command became a product roadmap.

Help text and skill files are part of the interface. I made a flag do nothing but left it in the skill instructions; all 95 observed searches still used it. After two sessions loaded revised instructions, neither used the obsolete flag and both tried new options.

Transcripts can provide replay tests: run the same commands against old and new versions, then inspect whether a result the agent used remains available. What the agent did next is only a rough clue about its intention, so hold back cases for evaluation.

I would first add useful ranges and limits, teach agents the options, and keep default output concise while explaining how to retrieve more. A flag only helps if agents know when to use it.

An agent-specific mode can present concise output and clear errors. But one example reduced tool-output tokens without materially improving overall cost or runtime. Measure correct answers and turns as well as the number of characters returned.

I do not expect an agent to design the best agent-facing interface in one shot. People should decide what the tool needs to do, then use real transcripts and tests to help the agent find and fix friction.

Back up your transcripts, search for a tool followed by a pipe or repeated retries, and inspect what happened next. Those workarounds are the product analytics for an agent-facing tool.