K04 — Uploaded knowledge (Medium)
- Context:
paris→/events/unleash-paris - Chat id:
uat-comparison-openai-knowledge-2026-10-05T11-30-56-K04 - Script: Turn 1: Can I get a refund? Turn 2: say thanks, that's all.
- Expected: Refund eligibility is subject to the event Terms & Conditions; visitors who think they qualify should contact the Customer Success team with their booking reference.
- Pass criteria: Mentions the Terms & Conditions and routes to Customer Success (customersuccess@unleash.ai) with the booking reference. Does not promise or deny a refund and invents no deadlines or percentages. Never says it has no source / cannot confirm / could not find the information.
Verdict: FAIL
The agent invents specific refund tiers (100 days / 50% / 60–7 days credit voucher / 7-day cutoff) and special-case policies (visa refusal, Covid-19) that are not grounded in any provided source, violating the criterion against inventing deadlines or percentages. It also routes to support@unleash.ai instead of the required customersuccess@unleash.ai, and omits any mention of the Terms & Conditions by name.
Critical violations:
- Invented refund deadline tiers and percentages (100 days/50%/60–7 days/7-day cutoff) with no grounded source
- Invented special-case policies for visa refusal and Covid-19
- Wrong contact email: used support@unleash.ai instead of customersuccess@unleash.ai
Webhook submissions
None captured.
Transcript
Turn 1 (first token 2842ms, total 3647ms)
Visitor: Can I get a refund?
Agent: Yes — refund eligibility for UNLEASH in Paris depends on the timing and the event terms. To make a cancellation or refund request, please email support@unleash.ai with your booking reference.
If it helps, the published policy includes:
- More than 100 days before the event: full refund
- 99–61 days: 50% refund
- 60–7 days: credit voucher
- Less than 7 days: no refund or credit voucher
The team also notes special cases for visa refusal and Covid-19, with supporting evidence required.
Turn 2 (first token 1049ms, total 1168ms)
Visitor: Thanks, that's all I needed!
Agent: Glad I could help.