How Upstart Evaluates Voice Agents for Regulated Lending Calls

Brooke Hopkins introduces herself as the founder of Coval, which builds evaluations for voice agents. Previously she led evaluation infrastructure at Waymo. She compares voice agents with autonomous vehicles: both must move reliably from a starting point to an outcome while responding to many possible paths and coordinating multiple models.

Hopkins says Coval works with enterprises on voice agents used in areas including customer support and logistics. Shiv Indap introduces himself as the lead for voice AI at Upstart, a financial technology company focused on lending.

Indap says Upstart handles thousands of calls each day. Lending is regulated, so a voice agent’s mistake can create a regulatory problem as well as a poor customer experience. The team’s first requirement is to operate within the applicable rules.

The use case he describes is a verification call. Indap says about 90% of applicants receive an instant decision, while the remainder call to verify information. The agent checks whether income is expected to continue, how funds will be used, and whether there are outstanding obligations.

Before those questions, the system must verify the caller’s identity so it does not disclose information about someone else’s application. Indap emphasizes that even a short script can be complicated once rules about follow-up questions are included.

He gives an example from the team’s policy: if an applicant mentions medical use of funds, the agent must not ask for the amount. A different answer, such as travel to a graduation, can require clarification about whose graduation it is. These are the examples he gives for this workflow, not a general statement of lending law.

The compliance team also wanted the agent to stay within the verification task. If an applicant asks about application status or when money will arrive, Indap says the agent should politely avoid making a commitment it is not authorized to make.

When the team launched, human review was required for each call. The operations team began with a Google Sheet, but as people added columns and entered information inconsistently, the process became hard to manage. Upstart needed structured review data that could improve both the agent and its performance measures.

Indap describes an LLM judge that runs after a call. For example, it can answer whether compliance requirements were met and provide an explanation. A human then reviews the judge’s output so the team can test whether the automated assessment aligns with expert judgment.

The team started with a broad set of measures, then narrowed its focus to metrics such as accuracy, clarity, completeness, policy alignment, escalation and customer satisfaction. Indap advises teams not to become attached to a metric: start with a reasonable set and change it when the use case reveals what matters.

For compliance, calls are scored, placed in a queue and reviewed by people who provide feedback. Indap reports that agreement between the LLM judge and human reviewers was 33% when the process began and later rose to 92%. He is describing agreement in this evaluation process, not a 92% compliance or accuracy rate for every call.

The team found that some operational knowledge had never been written into the agent or review criteria. Reviewers also disagreed with one another because of unwritten practices. Encoding that knowledge and improving the judge as more cases appeared helped close the gap, Indap says.

Hopkins takes over to explain the continuous-improvement loop. A release gate alone can permit one launch but make later model or prompt changes difficult. She argues that the system needs an ongoing process for monitoring conversations, defining measures, flagging calls and learning from review.

She groups evaluations into three questions: did the agent reach the outcome, did it follow the required steps, and what was the quality of the conversation? Human labels should clarify what reviewers actually care about before teams turn those criteria into automated evaluations.

Hopkins says teams can combine LLM judges, heuristics and audio processing to monitor calls. Known patterns might include missing a required verification or using incorrect wording. Clustering and anomaly detection can also help surface unexpected patterns that were not specified in advance.

Flagged calls can be routed to human review queues, allowing reviewers to focus on the cases most likely to matter. Hopkins stresses that detecting failures is not enough: organizations need an operational process for assigning each confirmed problem to someone who can act.

For illustration, she says a wrong answer might go to the agent team, while a compliance concern goes to compliance staff for an appropriate response. She explicitly says the percentages shown in that example are made up, not measured results.

Once a failure is understood and a fix proposed, the team can simulate the scenario again before exposing users to the change. Coval’s voice-to-voice simulations can vary background noise, accents, voice quality and interruptions, Hopkins says, so teams can check whether the fix addresses the problem.

She calls this shifting detection left: find as many failures as possible in simulation rather than relaunching a known problem to production and risking another user impact. The loop runs from product-defined metrics through monitoring, review, prioritization and simulated testing.

Hopkins says evaluation data can also support automated suggestions, reinforcement-learning pipelines or fine-tuned models that propose improvements. She describes this as a possible self-healing loop, but says the changes are still gated by human review.

Indap concludes that evaluations help score calls and identify which ones need attention when people cannot review thousands of calls a day. Reviewer annotations improve the LLM judge, while the collected data can also help improve the voice agent.

Changes remain behind feature flags and require sign-off. Indap says engineers normally approve judge changes, while compliance staff participate when a change affects compliance behavior. For both speakers, robust evaluations and human oversight are necessary to improve performance in a regulated workflow.