New
EXPERIMENTS
Research
6 min read
Testing agent changes on live traffic, safely
How Kestrel compares a new agent version against a control without putting customers at risk.
By
Amara Holt
A change to an agent can look great in testing and fall flat with real customers. The only reliable way to know is to measure it on live conversations, without making anyone’s day worse.
Start small, stop early
Every experiment starts on a small slice of traffic and grows only when results hold. Guardrails such as handoff rate and repeat contacts can stop an experiment on their own.
Decide on evidence
When an experiment ends, Kestrel shows the measured difference with sample conversations from both versions. A person approves the rollout, and the old version stays one click away.