How Brainbase Improved a Competitor-Pricing Agent from 41% to 97%

I am Milind Sunil Shah, a member of technical staff at Brainbase. We help enterprises deploy and manage internal agents. I will explain how we improved a back-office agent for Domino’s through repeated measurement and iteration.

The customer wanted to know the lowest available prices at nearby competitors: on their own sites and delivery apps, with and without promotions. Analysts previously gathered these figures manually, selecting the right store, fulfillment method and deal before filling in a spreadsheet. A plausible-looking cell can still be wrong if any step fails.

The agent needed to perform the same workflow, record sources and flag uncertainty for an analyst. According to our project measurements, version one was 41% accurate; by version eleven it reached 97%. Cost per source fell from $2 to $1, and a run fell from 129 to 71 minutes. We changed instructions, skills and playbooks rather than the model.

Our six-step framework is kickoff, ontology, benchmarks and evaluations, a narrow first agent, iteration, and handoff. It is a way to make the work measurable and improve the entire operating process rather than treating the first agent as the finished product.

At kickoff, agree with the end user on the job, what a human signs off, current time and cost, required accuracy, who judges the output, and what the agent must never assume or do alone. For Domino’s, the aim was repeatable local price collection without losing judgment on edge cases.

An analyst had spent about two hours a day actively assembling the spreadsheet, and every cell was checked. Our target was high accuracy at low source cost and runtime, with an analyst signing off and examining flagged cases. At the outset it was not even clear whether the discount search could be automated.

The ontology defines the terms and rules the agent needs: what to collect, where each answer comes from, where it goes and what to do when it cannot verify a result. If a term is not defined, the agent may guess. Each definition should reflect how the actual analyst performs the work.

We supplied a target table identifying each item, competitor, fulfillment method and source. In one example a product displayed $24 crossed out and $19 on sale. A rule says to record the price the customer pays at checkout. The observation table stores the selected price, full price and source, so a bad cell can be traced to the page and rule.

Other rules came from real failure traces: define what qualifies as a large pizza or bundle, keep delivery distinct from takeaway, calculate percentage discounts, and record an unverifiable value as a gap rather than guessing. These definitions became part of the working system.

For benchmarks, ask for past cases with known answers and decide what can be evaluated offline or online. We started with analyst spreadsheets and attempted to recreate historical competitor sites using archive snapshots. Delivery apps did not work the same way, so from version one we stored browser screenshots to build our internal benchmark environment.

We used deterministic code to check that IDs were replaced, discount math was correct and a price changing by more than 20% was flagged. A language-model judge checked traces for item and promotion selection. An agent judge investigated flagged prices with multiple tool calls, and an analyst reviewed the final exceptions.

Ship the smallest useful slice that can be scored from day one. Version one’s job is to be measured, not to be good. We began with one item in one city on one app, then added burgers, takeaway, another app and further sites as the baseline improved.

Iteration means score, fix and ship: improve the ontology, benchmarks and evaluations, prioritize frequent or expensive failures, and consider whether a fix adds time or cost. Change one thing at a time where possible, and fix instructions before replacing the model.

One agent followed a link straight to a large pizza and stopped at the displayed price. A human found a $21 deal lower on the same page. The fix was simple: do not stop at the linked section; inspect the entire page for applicable offers.

In another run, promotion prices appeared inside 26 images. The agent found the right card at minute 13 but kept zooming and had to be stopped at minute 15. Engineers inspected the trace and page, then instructed it to examine the HTML DOM, image titles and text first and to stop beyond a zoom limit.

That change reduced image reads to one third and cut cost by more than half in the example, while producing a completed report. Automated evaluations catch many problems, but engineers still need to inspect traces and browser tool calls to understand some failures.

Handoff means handing over the whole improvement loop: who runs the agent, reviews outputs, turns corrections into rules and benchmark cases, and decides when a new version is ready. At Domino’s, the agent runs on a schedule or by email trigger, analysts review exceptions, and corrections can be added to the benchmark. The team is expanding the approach to other markets and work.