How Ramp Applies Smaller Models, KV-Cache Cartridges and Agent Swarms

I work on applied AI at Ramp, bringing language models and machine learning into different products. Although our use of LLMs has grown substantially, newer frontier models have reduced the unit cost of some features while maintaining their accuracy.

That raises a question: why should an application company invest in research instead of waiting for the frontier to improve? We face customer problems today, and product-specific work can produce better experiences before a general model makes them easy.

In 2023, building a GPT-4 agent demanded tight guardrails, carefully tuned wording and limited context. A wrong step could send the system in the wrong direction, yet we still needed to ship agents for finance professionals.

By 2026, stronger models make it much easier to connect tools and context. I still think teams can go a layer deeper to improve the user experience, and I will give three examples from Ramp Labs.

Our first example grew out of Ramp Sheets, a spreadsheet agent launched in November. Finance and accounting work is full of spreadsheets, so we wanted to reuse its agent framework across more Ramp workflows.

When given many sheets and a task such as building a discounted cash flow model, the frontier agent spent time planning and examining its inputs. Opus 4.6 performed well, but its reasoning was relatively slow and costly for this specialized retrieval and question-answering work.

We kept a frontier model as orchestrator and partnered with Prime Intellect to post-train a Qwen 3.5 35B model for financial questions. On our specific task, it exceeded Opus 4.6’s accuracy while serving at latency comparable to Haiku; this is a task-specific result, not a general model ranking.

The second area is KV-cache cartridges, a family of techniques we are exploring to compress large amounts of context offline. I describe compression on the order of 20 to 30 times, allowing context to be prepared before an agent needs it.

Month-end close is predictable: finance professionals will need ledgers, aging reports and supporting documents. Rather than having an agent retrieve and interpret everything at the moment work starts, a cartridge could be built overnight with relevant context.

Cartridges can involve less post-training effort, but model providers may eventually expose similar capabilities through APIs. Even then, hands-on experimentation gives our team practical knowledge we can apply when those interfaces arrive.

Our third example is agentic map-reduce. Ramp used a cloud-agent system called Ramp Inspect to run 10,000 parallel agents against its own environment, distributing penetration-testing work and collecting possible findings.

We used models to deduplicate the resulting tickets and attempt to reproduce the suspected issues adversarially. According to our results, the process found seven high-severity vulnerabilities, which we patched.

One example involved a Lua injection delivered by email. No individual component was unsafe on its own; finding the issue required connecting behavior across several components. The example illustrates the value of end-to-end investigation rather than a promise of complete coverage.

We also studied model choice for swarms. In the reported experiment, open-weight models could cost up to five times less than frontier reasoning models, while the swarm’s detection rate did not fall by the same factor.

Across these projects, the shared goal is reducing latency and token costs. Those themes also inform two Ramp offerings: tools for understanding a company’s model spending and a router that applies different strategies to cost and latency.

I also point to other Ramp Labs research, including work on transferring post-training learnings between models, agentic map-reduce for usage taxonomies and self-improving service monitoring. These are additional research directions rather than results established by this talk.