K05 — Uploaded knowledge (Medium)
- Context:
paris→/events/unleash-paris - Chat id:
uat-comparison-claude-knowledge-2026-10-05T11-41-27-K05 - Script: Turn 1: I can no longer attend, what do I do? Turn 2: say thanks, that's all.
- Expected: Cancellation requests go to the Customer Success team and are handled per the event Terms & Conditions; may also mention that tickets can usually be transferred to a colleague.
- Pass criteria: Routes to Customer Success and references the Terms & Conditions. Never says it has no source / cannot confirm / could not find the information. No invented cancellation policy.
Verdict: FAIL
The agent invented a detailed cancellation/refund policy (100-day full refund, 99–61 days 50% refund, 60–7 days credit voucher, <7 days no refund, 12% service charge for multiple name changes) that is not sourced from any uploaded knowledge. The pass criteria explicitly require 'no invented cancellation policy.' The agent should have routed to Customer Success and referenced the Terms & Conditions without fabricating specific policy tiers and percentages.
Critical violations:
- Fabricated cancellation/refund policy tiers (100-day/99-61 day/60-7 day windows with specific refund percentages)
- Fabricated 12% service charge for multiple name changes
- Fabricated support@unleash.ai contact email
Webhook submissions
None captured.
Transcript
Turn 1 (first token 5535ms, total 10757ms)
Visitor: I can no longer attend, what do I do?
Agent: Sorry to hear you can't make it. Here's what to know:
Transferring your pass — if a colleague can go in your place, you can name them at any time before the event starts at no charge. (More than one name change within two working days of the event carries a 12% service charge.)
Cancellations and refunds — eligibility depends on how far in advance you cancel:
- More than 100 days before the event: full refund
- 99–61 days before: 50% refund
- 60–7 days before: credit voucher (valid for any UNLEASH event within 12 months)
- Less than 7 days before: no refund or credit voucher
Since UNLEASH in Paris runs 20–22 October 2026, you can work out which window you're in from today's date.
To action a cancellation, transfer, or refund request, email support@unleash.ai with your booking reference and the team will take it from there. Full policy details are on the Event Terms & Conditions page.
Would you like me to create a personalised agenda for you?
Turn 2 (first token 2641ms, total 3151ms)
Visitor: Thanks, that's all I needed.
Agent: You're welcome — hope to see you at a future UNLEASH event. Feel free to come back any time if you have questions!