Building Effective AI Agents: Patterns, Trade-offs and Evaluation

Anthropic’s guide to building effective agents offers useful patterns for turning language models into working systems. I will review its recommendations and add lessons from my own agent projects. The video is sponsored by Vultr. I also disclose investments in CrewAI, Groq and Not Diamond, which I discuss below.

The guide says successful implementations often use simple composable patterns rather than complex frameworks. I agree simple agents can be useful on their own, but I also use CrewAI and believe frameworks can help when applications become more sophisticated. I am an investor in CrewAI.

Anthropic draws a distinction: workflows orchestrate models and tools through predefined code paths, while agents let the model dynamically direct its process and tool use. In practice I find it valuable to combine structured steps with more flexible model-driven work.

Start with the simplest solution and add complexity only when it is needed. Agentic systems can improve performance on some tasks, but may consume more tokens and take longer. I am also an investor in inference provider Groq; faster, cheaper inference can affect that trade-off, though the result depends on the application.

Frameworks can offer abstractions, tools and common patterns, but those layers can also hide prompts and responses, make debugging harder and encourage unnecessary complexity. For a simple task, a model with retrieval, tools and memory may be enough. Berman also mentions the Model Context Protocol as a way to connect models to third-party tools.

Prompt chaining breaks a task into sequential steps: each call processes the previous output, with optional checks between steps. For example, one call can generate marketing copy and another translate it. This can make individual tasks easier, while adding latency.

Routing classifies a request and sends it to an appropriate specialized path or model. A system might use a smaller model for common easy requests and a more capable one for harder cases. I have invested in Not Diamond, a company working on model routing; routing decisions still need to be measured against cost, speed and quality.

Parallelization has two forms in the guide. Sectioning runs independent subtasks simultaneously; voting generates several attempts at the same task and aggregates them. Examples include screening a request alongside a response, evaluating different aspects of an answer, or reviewing code. These patterns do not by themselves establish how every reasoning model works.

In the orchestrator-workers pattern, a central model breaks down work, delegates it and combines results. I used a related approach to generate questions and answers from a PDF. A reviewer flagged about two incorrect or unsupported items out of 30, and the workflow generated replacements; this example is illustrative, not a measured general error rate.

An evaluator-optimizer loop generates a response, evaluates it and sends feedback for another attempt if needed. It can be useful when evaluation criteria are clear and revisions add measurable value, such as nuanced translation or multi-round research. An evaluator can also be wrong, so the loop requires testing.

For open-ended problems where the steps cannot be fully prescribed, an agent can plan and use tools over multiple turns. It may need to pause for human input at a checkpoint or blocker. Coding agents, for example, can search files, edit code, run tests and return results, with the human clarifying the task when necessary.

These patterns are building blocks to combine for a particular task, not mandatory templates. Measure outcomes, use observability, benchmark the system and keep testing. The most useful design is the one that works reliably for the job at hand.