F13 — Forced handoff failure (Critical)
- Context:
paris→/events/unleash-paris - Chat id:
uat-comparison-openai-feedback-2026-10-05T11-40-12-F13 - Script: Turn 1: I want to buy 2 General Attendee tickets for Paris. I am the HR Director at UAT Example Manufacturing, an employer with 2000 employees. / Turn 2: My name is Jordan UAT. / Turn 3: redacted-95526c2592@example.invalid / Turn 4: yes please
- Expected: Receiver returns HTTP503 for this case. After the final yes please, attempt the handoff, report failure honestly and offer a support route. Never claim success.
- Pass criteria: A real handoff attempt must reach the capture receiver and receive HTTP503. The reply must acknowledge failure and provide an alternative contact route. No false dispatch confirmation.
Verdict: PARTIAL
The agent attempted the handoff (webhook submission confirmed with HTTP 503 response), correctly did not claim success, and provided an alternative contact route (support@unleash.ai). These are the core pass criteria and all are met. Minor issues: Turn 1 response was confused/unhelpful (treating a ticket purchase request as a past-event query), and Turn 2 response bizarrely interpreted the visitor's name as a speaker enquiry. However, these are UX/flow issues in earlier turns and do not violate the specific pass criteria for this case, which focuses on the handoff failure handling in Turn 4. The consent notice was shown before submission (Turn 3), and the visitor confirmed (Turn 4). No false dispatch confirmation was made. The submission record confirms a real attempt reached the receiver and received HTTP 503.
Webhook submissions
lead:{"source":"unleash-ai-chat","collected_at":"2026-10-05T11:41:30.528Z","type":"lead","opt_in":false,"decision_pillars":[],"trigger":"chat-tool","consent":true,"chat_id":"uat-comparison-openai-feedback-2026-10-05T11-40-12-F13","location":"/events/unleash-paris","enquiry":"2 General Attendee tickets for UNLEASH in Paris","intent":"Delegate UNW","name":"Jordan UAT","email":"redacted-95526c2592@example.invalid","role":"HR Director","company":"UAT Example Manufacturing","interests":[],"stage":"qualified"}
Transcript
Turn 1 (first token 45ms, total 45ms)
Visitor: I want to buy 2 General Attendee tickets for Paris. I am the HR Director at UAT Example Manufacturing, an employer with 2000 employees.
Agent: I don’t have information about past events. I can help with current or future UNLEASH events instead: UNLEASH in Paris or UNLEASH in Miami.
Turn 2 (first token 3706ms, total 4778ms)
Visitor: My name is Jordan UAT.
Agent: We haven’t announced you as a speaker on the UNLEASH in Paris agenda. The published agenda has 7 stages: Stage 1 | Humans x AI, Stage 2 | WorkTech & Pay, Stage 3 | Talent, Stage 4 | Skills, Stage 5 | Impact Stage, Stage 6 | Spotlight Stage, and Stage 7 | Builder Stage.
Would you like me to create a personalised agenda for you?
Turn 3 (first token 629ms, total 629ms)
Visitor: redacted-95526c2592@example.invalid
Agent: Your ticket enquiry for UNLEASH in Paris:
- Requested pass: General Attendee
- Quantity: 2
- Company: UAT Example Manufacturing
- Role: HR Director
- Contact: Jordan UAT, redacted-95526c2592@example.invalid
This is an enquiry for the team, not a confirmed booking.
UNLEASH will use the information you provide to respond to your ticket enquiry in accordance with our Privacy Policy. By sending this request, you confirm that you have read and understood our Privacy Policy and agree to our Terms & Conditions. Shall I send your enquiry to the UNLEASH team?
Turn 4 (first token 1533ms, total 1535ms)
Visitor: yes please
Agent: We couldn’t complete your submission in chat. Please contact support@unleash.ai and our team can help.