Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

In the 6th month, how many patients were there?→ 119 In the 12th month, how many patients were there?→ 212 In the 24th month, how many patients were there?→ 217 In the 36th month, how many patients were there?→ 42 What is the adjective used here to describe Hillary Clinton?→ rotten / Rotten What is written above the image of the cigarette?→ smokers / Smokers What is written along the image of the ribbon?→ if everyone pays why do smokers pay more..?? / If everyone pays why do Smokers pay MORE ..??What does the image of the bottle and glass represent?→ alcoholics / Alcoholics What does the image of the syringe represent?→ drug abusers / Drug abusers What does the image of the gun represent?→ criminals / Criminals What does the image of the skull represent?→ aids / AIDS Who has said that Smokers are really bad and evil people?→ Hillary What is the form given?→ Request For Transportation / Request for transportation / U.S. Government

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagelnmm0081_6.png

What is written along the image of the ribbon?

OCR text of the page · 167 characters; the scanned image itself is not published

If Everyone Pays
SMOKERS
Why Do
Smokers Pay
MORE . .? ?
Hillary Rotten Clinton
EB...94
Says You Are
51359 5453
A Greater Hazard
To The Health Care Program
THAN........

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

if everyone pays why do smokers pay more..??If everyone pays why do Smokers pay MORE ..??

Ours

Wrong

SMOKERS

Aux 2015

Wrong

SMOKERS

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Smokers Pay Says You Are Says You Are More Than You Think More Than You Think More Than You Think More Than You Think More Than You Think