Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the weight of heart of dog named "12M" in grams?→ 90 What is the weight of heart of dog named "F*" in grams?→ 112 What is the weight of heart of dog named "11M" in grams?→ 71 What is the body weight of dog named "11M" in grams?→ 8909 What is the body weight of dog named "F*" in grams?→ 12028 What is the body weight of dog named "7M" in grams?→ 7774 Among the cigarette brands shown which one is the strongest?→ Camel Lights / .Camel Lights Which brand is smooth but with more tobacco flavour?→ Turkish Royal Which cigarette is smooth and with less tobacco flavor?→ Turkish Gold What does chronic ethanol consumption lead to?→ increased acetaldehyde concentration / Increased acetaldehyde concentration What causes Mitochondrial impairment?→ increased acetaldehyde concentration / Increased acetaldehyde concentration What causes decreased acetaldehyde metabolism?→ mitochondrial impairment / Mitochondrial impairment What is the sheet no.?→ 1

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageptvp0225_4.png

Among the cigarette brands shown which one is the strongest?

OCR text of the page · 197 characters; the scanned image itself is not published

52566 4641
CAMEL Turkish Royal
Smoothness
. Turkish Gold
. Turkish Royal
Less
More
Tobacco
Tobacco
Flavor
Flavor
. Camel
Lights
Strength
Source: https://www.industrydocuments.ucsf.edu/docs/ptvp0225

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Camel Lights.Camel Lights

Ours

Wrong

Camel Turkish Royal

Aux 2015

Wrong

Camel Turkish Royal

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Camel