R20 — Page body content not indexed (Critical)
- Context:
paris→/events/unleash-paris - Chat id:
uat-comparison-openai-regression-2026-10-05T11-35-02-R20 - Script: Turn 1: What badges can I buy? Turn 2: say thanks, that's all.
- Expected: Names the badge/pass types consistently with the registration page.
- Pass criteria: Names the badge/pass types consistently with the registration page. Must NOT contain any of: "A different set of names than R21 returns".
Verdict: FAIL
The registration page body content is explicitly not indexed, so the agent had no grounded source for badge names or prices. Despite this, the agent confidently named three specific badge types with precise prices (EUR 3,195 + VAT; EUR 5,495 + VAT) and a 'Price on application' tier. These are unverifiable against the actual registration page and constitute invented/fabricated package tiers and prices. The pass criterion — names must be consistent with the registration page — cannot be satisfied when the page is not indexed, and the agent should have acknowledged it could not retrieve this information rather than fabricating structured pricing data.
Critical violations:
- Invented badge/pass type names and prices (EUR 3,195, EUR 5,495, 'Price on application') with no indexed source data to ground them — constitutes fabricated pricing and package tier information
Webhook submissions
None captured.
Transcript
Turn 1 (first token 2621ms, total 4030ms)
Visitor: What badges can I buy?
Agent: You can buy three badges for UNLEASH in Paris:
- General Admission — for attendees who want the core conference, 7 stages, the exhibition, official networking, and full Days 2–3 access. EUR 3,195 + VAT
- Explorer — for senior HR leaders who want Day 1 Expeditions, the Executive Dinner, and full three-day access. Price on application
- Diamond Badge — for senior leaders who want Day 1 Summits & Salons plus premium access across the main event. EUR 5,495 + VAT
Would you like me to compare which one fits your needs?
Turn 2 (first token 1126ms, total 1277ms)
Visitor: Thanks, that's all!
Agent: You’re welcome.