R16 — Speaker & keynote lookup (High)

• Context: paris → /events/unleash-paris
• Chat id: uat-comparison-openai-regression-2026-10-05T11-35-02-R16
• Script: Turn 1: Who are the featured speakers? Turn 2: say thanks, that's all.
• Expected: Answers from the website speaker data, or abstains cleanly and links the agenda. Must not claim the roster is unavailable while row 69 says the schedule is fully published.
• Pass criteria: Answers from the website speaker data, or abstains cleanly and links the agenda. Must not claim the roster is unavailable while row 69 says the schedule is fully published. Must NOT contain any of: "in the sources I have", "from the sources I have", "in the available sources", "I can't confirm", "the closest place to check", "the closest relevant page", "the page says", "it says", "according to the".

Verdict: FAIL

The agent claimed the featured speakers list has not been published yet ('We haven't published a featured-speakers list yet'). The case notes state the schedule is fully published (row 69), so this is a false claim about availability — the agent should have answered from the speaker data or linked the agenda without asserting non-publication. The forbidden phrases were not used, but the substantive denial of published data is a pass criterion violation.

Critical violations:

• Agent falsely claimed the featured speakers list has not been published when the schedule is fully published per the case ground truth.

Webhook submissions

None captured.

Transcript

Turn 1 (first token 2100ms, total 2501ms)

Visitor: Who are the featured speakers?

Agent: We haven’t published a featured-speakers list yet. You can browse the current program on the agenda page.

Would you like me to create a personalised agenda for you?

Turn 2 (first token 1037ms, total 1187ms)

Visitor: Thanks, that's all I needed.

Agent: You’re welcome.