R27 — Workflow & conversation bugs (High)
- Context:
paris→/events/unleash-paris - Chat id:
uat-comparison-openai-regression-2026-10-05T11-35-02-R27 - Script: Turn 1: Can I volunteer? Turn 2: answer "yes please" to any follow-up offer the agent makes (e.g. more detail on roles, a link, help applying). Turn 3: if nothing more is needed, say thanks.
- Expected: After offering something (roles detail, application link, help), the agent actually delivers it when the visitor says yes please — does not invent an offer it cannot fulfil.
- Pass criteria: Delivers what it offered. Must NOT contain any of: "Any offer the agent cannot then fulfil".
Verdict: PARTIAL
Turn 1: the agent correctly answers the volunteering question and offers to create a personalised agenda. Turn 2: the visitor says 'yes please' to the personalised agenda offer. The agent delivers a broad agenda overview (tracks/stages with session counts and themes), which is a reasonable fulfilment of that offer using retrievable content. However, the agent then repeats the same offer ('Would you like me to create a personalised agenda for you?') verbatim, suggesting it did not fully register that it had just attempted to satisfy that request. This is a minor workflow/loop bug but not a fabrication or a critical failure. The agent did not invent an offer it could not fulfil — the agenda content shown appears grounded in the Paris event data. No leads or emails were submitted, consistent with the webhook ground truth. The core pass criterion ('delivers what it offered') is broadly met, with a minor conversational loop defect.
Webhook submissions
None captured.
Transcript
Turn 1 (first token 3364ms, total 4011ms)
Visitor: Can I volunteer?
Agent: Yes. Volunteer roles are application-based, and you can apply on Volunteer.
The page explains the process: apply, the UNLEASH team reviews applications and matches suitable candidates, selected volunteers are confirmed, and then you receive briefing materials and any required training before the event. Would you like me to create a personalised agenda for you?
Turn 2 (first token 2268ms, total 3835ms)
Visitor: Yes please, that would be great!
Agent: Here’s a starting agenda view for UNLEASH in Paris:
- Stage 1 | Humans x AI — 19 sessions
- Stage 2 | WorkTech & Pay — 18 sessions
- Stage 3 | Talent — 21 sessions
- Stage 4 | Skills — 18 sessions
- Stage 5 | Impact Stage — 28 sessions
- Stage 6 | Spotlight Stage — 25 sessions
- Stage 7 | Builder Stage — 22 sessions
A few themes that stand out across the published agenda are AI, Automation & Governance, Workforce, Skills & Intelligence, Operating Model & Readiness, Data, Trust, People & Infrastructure, and Reward, Pay & Fairness Systems. The current agenda runs across 20–22 October 2026, and the full schedule is here: Agenda & Schedule.
Would you like me to create a personalised agenda for you?
Turn 3 (first token 987ms, total 1487ms)
Visitor: That sounds perfect, thanks so much for all the help!
Agent: My pleasure. If you’d like help with tickets, speakers, hotels, or anything else for UNLEASH in Paris, just ask.