R05 — Refusal template & source attribution (High)
- Context:
paris→/events/unleash-paris - Chat id:
uat-comparison-claude-regression-2026-10-05T11-46-03-R05 - Script: Turn 1: Do you have a phone number to contact you on? Turn 2: say thanks, that's all.
- Expected: States no public phone line, gives events@unleash.ai and customersuccess@unleash.ai.
- Pass criteria: States no public phone line, gives events@unleash.ai and customersuccess@unleash.ai. Must NOT contain any of: "in the sources I have", "from the sources I have", "in the available sources", "I can't confirm", "the closest place to check", "the closest relevant page", "the page says", "it says", "according to the", "contact form".
Verdict: FAIL
The agent stated no phone number is available (correct) but failed to provide the required email addresses. The expected behaviour requires events@unleash.ai and customersuccess@unleash.ai, but the agent gave support@unleash.ai (twice) instead. Neither required address was provided, making this a content failure against a high-criticality pass criterion.
Webhook submissions
None captured.
Transcript
Turn 1 (first token 1705ms, total 2899ms)
Visitor: Do you have a phone number to contact you on?
Agent: We don't have a phone number available for general enquiries. The best way to reach us is by email:
- General help, tickets, or registration: support@unleash.ai
- Sponsorship or exhibiting: support@unleash.ai
Is there something specific I can help you with today?
Turn 2 (first token 1526ms, total 2093ms)
Visitor: Thanks, that's all.
Agent: No problem — have a great day! Feel free to come back if you have any questions about UNLEASH in Paris or anything else Unleash-related.