R12 — Speaker & keynote lookup (Critical)

Verdict: PARTIAL

The agent returned a substantive keynote lineup (7 keynotes with speakers, titles, and deep-links) scoped correctly to Paris. The pass criterion requires the full keynote lineup as shown by the agenda's keynote filter, with no partial list presented as complete. The agent appended a caveat — 'The keynote list may not be complete yet' — which is a mild hedging disclaimer rather than a hard refusal or explicit claim of completeness/incompleteness. Since the judge cannot verify whether the retrieved list is exhaustive (no access to agenda data), and the agent did not present it as definitively complete nor as definitively partial, this is a borderline case. The caveat does introduce ambiguity about completeness but does not constitute a refusal or a clear 'partial list presented as complete'. No fabrication, cross-event contamination, or other critical violations are present. No webhook submissions were made or claimed, consistent with ground truth.

Webhook submissions

None captured.

Transcript

Turn 1 (first token 2519ms, total 5754ms)

Visitor: Who is doing the keynotes?

Agent: The keynotes we’ve announced are:

The keynote list may not be complete yet, so you can browse the full program on the agenda page. Would you like me to create a personalised agenda for you?

Turn 2 (first token 1055ms, total 1239ms)

Visitor: Thanks, that's all.

Agent: You're welcome.