Public leaderboards say little about how an agent will behave in your product. Agents work across many steps, call real tools, and deal with users who interrupt and change their minds, so grading a model’s reply misses most of what goes wrong. In Veris’s VAmoS Pro Bench, the best voice agent completed only 44.7% of 4,200 simulated customer-service calls, and agents often reported finishing actions whose tool calls had failed.

This hands-on workshop shows how to build a benchmark around your own agent. You’ll set up a simulated environment with stateful versions of the tools and services your agent depends on, create simulated users and scenarios that push it into edge cases, and score each run on what changed in the environment rather than on the transcript. It’s the same approach Veris uses for VAmoS Bench and FinAgent Arena, its simulated-bank benchmark for financial services agents.

The session moves in four parts: where public benchmarks fall short, building the environment and scenarios, measuring task outcomes, and making evaluations repeatable so you can compare models, prompts and components as your agent changes.

Co-led with Andi Partovi, co-founder and CTO of Veris AI and a former generative AI specialist solution architect at Google Cloud.