Human seeded Evals — Samuel Colvin, Pydantic
The article isn't ready yetWe have this video's details, but its article hasn't been written. You can watch it on YouTube in the meantime.Open on YouTube
More from this channel
17:56EN
AI Engineer· Oct 25, 2026How Bridgewater Keeps Its AI Analyst Inside Permission BoundariesNikhil Benesch, Aaron Linsky · turbopufferAaron Linsky and Nikhil Benesch explain why search over an enterprise vault must honor document permissions without sacrificing recall or speed. Their solution combines vector indexing with native filtering to search hundreds of millions of vectors in roughly 120 milliseconds.
35:19ENAI Engineer· Oct 25, 2026SallyAnn DeLucia: How to Write Your First AI EvalSallyAnn DeLucia · Arize AITurn an agent’s real traces into concrete pass/fail criteria, then build an LLM judge you can test, align with people, and run in production.
20:58ENAI Engineer· Oct 25, 2026Vinoo Ganesh on Making Financial AI Cite Every NumberVinoo Ganesh · KeplerKepler CEO Vinoo Ganesh explains why financial AI should retrieve and cite figures while deterministic software handles the math. His design makes traceability, retrieval quality, and tenant security part of the answer itself.
18:31ENAI Engineer· Oct 25, 2026Janie Gu and Suyeol Yun: Why Frontier Models Fail as Buy-Side AnalystsJanie Gu, Suyeol Yun · Google DeepMindJanie Gu explains why benchmark scores can mislead, while Suyeol Yun presents MandateBench, which tests whether a model can turn a fund mandate into a defensible investment memo. The two talks show why real workflows, reproducible grading, and evidence-backed reasoning matter.
15:27ENAI Engineer· Oct 25, 2026What AI Engineers Do Now: Build the Loop Around AgentsAparna Dhinakaran · Arize AIAparna Dhinakaran argues that AI engineers are moving from shipping individual features to building systems that let agent teams improve them. At Arize, that means tracing agent behavior, evaluating its work against human judgment, and feeding those signals back into an always-on agent.
19:37ENAI Engineer· Oct 25, 2026Jay Sahnan and Arik Bird on Building Financial Agents for the Whole WebJay Sahnan, Arik Bird · BrowserbaseBrowserbase’s Jay Sahnan and Arik Bird show how browser agents reach financial data hidden behind logins and bot defenses. Their Ramp, Bill, and Stripe examples connect reliable cloud browsers with Stagehand’s more adaptive controls.