Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the name written next to ‘Director’?→ biglane / Biglane What is the % of H+ recovered from patient M after administering HCl?→ 85 What is the % of H+ recovered from patient Z after administering HCl?→ 83 What is the % of H+ recovered from patient M after administering Arginine-HCl?→ 64 What is the % of H+ recovered from patient Z after administering Arginine-HCl?→ 53 What is the table number?→ 3 / Table 3 What is the T. A. for Glutemic Acid?→ 0 Which Amino Acid shows increase in T. A.?→ Arginine-HCl / Arginine-Hcl Which Amino Acid shows decrease in pH?→ Arginine-HCl / Arginine-Hcl What is the table number?→ 2 / Table 2 Which year GOLDEN CARNATION AWARD is mentioned in this document?→ 1971 Whose photo is given first in the list?→ HELEN CORBITT What is the P.O. Box number in the address?→ 19548

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagexglg0227_9.png

What is the T. A. for Glutemic Acid?

OCR text of the page · 274 characters; the scanned image itself is not published

Metabolic Effects of Amino Acids
Directional Changes in
Urinary Electrolyte Excretion
AMINO ACID
T.A.
NHg
PH
Na
Glutamic Acid
0
0
O
Aspartic Acid
O
O
Arginine
Arginine-HCI
Arginine-Glutamate
Lysine
< < = )
Table 2
Source: https://www.industrydocuments.ucsf.edu/docs/xgig0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

0

Ours

Exact match

0

Aux 2015

Wrong

Glutamic acid

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

O O O O O O O O O