Voice Agent Behavior Lab

ProblemNo rigorous instrument for studying voice agent trust dynamics — researchers were using ad-hoc observation notes that couldn't be compared across sessions.

ApproachFour-instrument session kit: disclosure observation, pacing protocol, post-task interview, and handoff rubric — all designed for cross-reference mid-session.

Outcome42 sessions, 3 researchers, 91% inter-rater agreement. Observed and self-reported trust diverged by 1.8 points — the headline finding.

Context

Designing voice agents is harder than designing screens because the artifact you're designing — a conversation — disappears the moment it ends. There is no persistent visual state to inspect. The failure mode happened in real time, and unless someone was actively documenting it, it's gone. The result is that most voice AI design is done reactively: something breaks, someone notices, a fix is applied. There is no systematic method for observing, capturing, and reasoning about conversational behavior at the interaction level.

The Voice Agent Behavior Lab is the instrument I built to close that gap. It is a structured research environment for observing and documenting voice-agent behavior across the dimensions that matter most for design: how conversations journey from intent to outcome, where memory fails, how silence and hesitation are handled, how interruptions and misunderstandings are recovered from, and how trust forms and breaks across an interaction.

It is explicitly not a product, not a dashboard, not a monitoring tool. It is a behavioral science instrument — designed for rigorous qualitative research into how voice agents actually behave, so that design decisions can be grounded in observation rather than assumption.

The Instrument

Conversation Journey Map — the primary research instrument, designed in Pencil

The Lab is structured around six behavioral dimensions where voice agent design most commonly succeeds or fails:

Conversation journeys — the full arc from what the user meant to what actually happened

Memory & forgetting behavior — what the agent retains, what it loses, and what that does to the caller

Silence & hesitation — how the agent handles pauses, delays, and non-verbal signals

Interruption & clarification — what happens when the caller redirects, backtracks, or asks the agent to repeat

Recovery from misunderstanding — how the agent detects and repairs conversational breakdowns

Trust formation & breakdown — the moment-by-moment accumulation and erosion of the caller's confidence in the agent

The primary research instrument is the Conversation Journey Map: a structured documentation frame for tracing the full arc of a conversational interaction. Five stages, each with its own capture area:

User Intent → Agent Interpretation → Response → User Reaction → Outcome

The map surfaces the gap between what the user meant and what the agent understood — and the cascading effects of that gap through the rest of the interaction. Each stage captures the actual utterances, the agent's observable behavior, the caller's reaction, and the eventual outcome. This is not a transcript; it's an annotated behavioral map.

Below the Journey Map, four capture areas complete the research record: Transcript (verbatim record), Screenshots (visual evidence of UI state during the call), Field Notes (researcher observations in real time), and Observations (synthesized behavioral findings after the session).

The Lab is designed in Pencil (.pen format) — which, beyond being the tool I use for interaction design, makes the research instrument itself a design artifact: a document that thinks about its own structure, not just a template dropped into a note-taking app. Pencil's ability to compose structured visual environments made it the right tool for a research instrument that needs to be both rigorous and legible at a glance.

Instrument Suite 01–04 — four instruments, one question: does the caller stay in charge?

Rationale

Research instrument, not product

The most important decision in the Lab's design is what it refuses to be. A product optimizes for an end user's needs. A dashboard optimizes for monitoring. A chat interface simulates the thing you're studying. A research instrument optimizes for rigorous observation — which requires a completely different design stance. Everything in the Lab exists to make behavior visible and documentable, not to make the researcher feel productive or the data feel organized.

Intent vs. outcome as the structural spine

The five-stage Journey Map is structured around the gap between user intent and actual outcome — because this gap is where almost all meaningful voice agent design problems live. An agent that correctly executes what the user said but missed what they meant is a failure. An agent that recovers from a misunderstanding and restores the caller's confidence is a design success. The instrument makes this gap structurally visible rather than something to be inferred after the fact.

Six behavioral dimensions, not interaction metrics

Standard voice AI measurement is metric-driven: call duration, resolution rate, CSAT scores. These measure outcomes, not behaviors. The six focus areas of the Lab measure the behaviors that produce outcomes — including behaviors that metrics can't capture, like how trust forms and breaks across a conversation, or how an agent's handling of hesitation changes the caller's willingness to proceed. Behavioral observation produces design-actionable findings. Metrics don't.

What NuvAI applies, the Lab discovers

The Lab and NuvAI are related but distinct. NuvAI is a product application — real estate, specific stack, specific use case. The Lab is the research practice that generates the behavioral findings NuvAI's design is built on. The disclosure-first design and pace-to-the-caller philosophy in NuvAI are not intuitions; they are findings. The Lab is where those findings come from.

Three assumptions carried into the first protocol cycle that the research corrected:

Disclosure timing matters more than disclosure content. We assumed users cared most about whether they were told. It turned out the when was the primary driver — disclosure in the first 10 seconds produced different trust dynamics than disclosure at 30 seconds, even with identical wording.

Pacing controls don't confuse users — their absence does. The original protocol treated speed controls as a confound. In practice, users who wanted to slow down developed workaround behaviors (repetition requests, extended silences) that contaminated the handoff measurements.

Observed behavior and self-reported trust diverge. The 91% inter-rater agreement was expected. The divergence — observed and self-reported trust differing by an average of 1.8 points on a 5-point scale — was not. That gap became the study's headline finding.

Status

The Voice Agent Behavior Lab is an active research instrument in use alongside the NuvAI product design process. The Pencil design file holds the full instrument structure — available for review in conversations with potential collaborators or employers.

For any team hiring a voice AI designer or conversational UX researcher: the Lab is the clearest evidence that I approach voice agent design as a behavioral science problem, not a UI problem. The instrument exists because the design questions I care about cannot be answered without systematic observation. That's what it's for.