I led five studies that took an AI assistant for employees’ HR questions (the Assistant) from concept to global launch. I ran generative discovery and evaluative usability, then studied the product pilot in the live environment, directing a contract researcher throughout. The research shaped the core experience decisions (when to hand off to a human, how to earn trust with sensitive HR data, and how accurate is accurate enough) and fed a formal launch decision.
01 — Stakes HR is a bad place for an AI to be wrong
The product team, working with the People organization, was building the Assistant to answer employees’ HR questions, guide them through workflows, and hand off to a human specialist when needed.
Three things raised the stakes:
The research question
Where is the line between what employees will trust AI to do and what they won’t, and how do we design so the Assistant never crosses it?
02 — Studies A different method for each stage of the product
I matched the method to how far along the product was and what the team had to decide next.
I owned study design and strategy, did the analysis and synthesis, and delivered readouts to the product, engineering, design, and People partners. I directed a contract researcher throughout, set the quality bar, and coached her along the way. She helped moderate the sessions.
Generative discovery
11 interviews · early 2025
Before the experience was built, we ran exploratory interviews (deliberately not usability testing) to map what employees expected from AI in the People space. I recruited for range: corporate and retail, new hires to people with fifteen years or more, tech and non-tech roles, across AMR, EMEIA, and APAC. The work also built on two prior AI studies.
Evaluative usability testing
6 participants against prototype · mid 2025
We tested the virtual chat, live agent, and case management journeys across three regions. I held answer accuracy out of scope on purpose. Engineering was still tuning the model, and I wanted the study to judge the experience without getting derailed by answer quality. I documented that reasoning, including the NN/G sample size rationale.
A pilot in the live environment
Log analysis, retail & corporate · late 2025
Once the Assistant reached the real environment it ran as a product pilot, and I studied it in three phases: log analysis with the product team, then usability with retail and corporate cohorts in the live environment. Together they fed a formal launch decision. The conditions weren’t ideal. The environment changed during testing, samples were small, and timelines were tight, and I said so in every readout.
03 — Findings Where employees drew the line
The same pattern held across all five studies. Employees welcomed AI that saved them time, and wouldn’t tolerate it overstepping: getting answers wrong, making calls a person should make, or taking control of their data out of their hands.
AI assistant
Live specialistEmployees forgave the Assistant’s limits far more readily when it could hand off to a live specialist.
The wariness dropped when a human was reachable
Participants had been wary in earlier rounds. Adding the live specialist option visibly changed that.
I recommended a live specialist reachable at any point, with the AI conversation passed along so people don’t have to repeat themselves.
“You’re just more forgiving with a live person.” Knowing a real person had tried made them forgiving even of a failure.
Almost no tolerance for inaccuracy
In the People space, expectations approached 100% accuracy because, as participants put it, “it’s people’s lives” and “there’s no margin for error.”
I recommended leaving any topic with poor accuracy out of scope until it improved, always citing sources, explaining why the Assistant can’t answer when it can’t, and considering a human co-pilot in the early phases.
The realistic bar: match or exceed the accuracy of the current human People Support experience.
“It’s people’s lives. There’s no margin for error.”
Some topics should go to a person
Employees were clear that some topics need a person. On those, the Assistant shouldn’t attempt an answer at all: it should route people to a specialist, and do it unprompted. The boundary was consistent across regions and segments.
I recommended guardrails on the topics an AI should avoid, with the Assistant suggesting a specialist for them.
Opting in, one type of data at a time
Employees wanted to choose which kinds of data the Assistant could use. They also wanted to know what it accessed and what it stored, and treated those as separate questions.
I recommended letting people opt in one type of data at a time, leaving family details, home address, and medical data out by default, and following regional norms.
Retail and corporate weren’t the same audience
Corporate employees preferred a professional tone. Retail employees wanted warmth, needed a chat that wouldn’t time out on the shared devices they use on the floor, and valued the live agent even more.
I recommended starting with a professional tone and adding warmth once the AI could carry it, and minimizing or removing the chat timeout.
They wanted it to take actions in other systems
Some raised it unprompted, and others were keen once the idea came up. I recommended building toward it, as a roadmap item beyond simple Q&A.
04 — Launch The handoff finding held up once it shipped
Each readout ended with recommendations, addressed to the teams that would act on them: engineering, design, and comms and change management. In the pilot I sorted them by effort, separating quick fixes from work engineering needed to size, and tracked their status.
The findings and recommendations went into the Assistant’s requirements and the formal launch decision. In the experience itself, they became five choices:
Confirmed in production
The handoff gets real use: 58% of interactions resolve with the virtual agent alone, and the rest hand off to a live specialist or ask for one. That’s the path the research said would matter most.
05 — Artifacts A plan, a script, and two readouts
Working documents from the program, with names, tools, and identifiers generalized for confidentiality.
06 — Reflection Trust as a design constraint
Capability matters, and it helps earn trust, but what decided adoption was whether employees believed the Assistant, acted on what it said, and came back. The research showed that belief rests on things the model alone doesn’t provide: an obvious path to a human, honesty about limits through explainability, and control over personal data.
If I did it again, I’d push for larger samples. We moved fast under executive deadlines, and the signal would have been clearer with more people. I’d also measure the AI’s answers directly, seeding known errors and checking them against what subject matter experts know, and I’d track ticketing metrics over time to see whether the Assistant actually resolves issues or just defers them.
As people get more familiar with generative AI, I think they’ll trust it with more, as long as the safeguards keep pace with what it can do. If I were continuing this project, I’d keep running generative discovery to track the cultural changes, and I’d put newer, more advanced prototypes in front of users.