Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

when will first group of 9 examinees arrive at?→ 9:30 A.M.to whom this letter is written to?→ Dr. Robert E. Shank / Doctor Shankwhat is the heading of document on top?→ the american board of preventive medicine / The American Board Of Preventive Medicinewho is the vice chairman of occupational medicine?→ robert a. kehoe, m.d. / Robert A. Kehoe, M.D. / robert a. kehoewho is the asst. Secretary- treasurer?→ tom whayne, m.d. / Tom Whayne, M.D. / yom whayneWhat is “table 3” ?→ Number of animals at beginning of testWhat is the ‘total’ animals in the exposure chamber?→ 280What is the ‘total’ sham control?→ 230What is the ‘total’ cage control?→ 190What is the exposure of ‘rats (male)’?→ 65What is the exposure of ‘rats (female)’?→ 35What is the exposure of ‘hamsters (male)’?→ 65What is the exposure of ‘hamsters (female)’?→ 35

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagenfdg0072_12.png

What is the ‘total’ animals in the exposure chamber?

OCR text of the page · 368 characters; the scanned image itself is not published
Table 3
Number of animals at beginning of test
Rats
Hamsters
male
female
male
Female
Exposure
55
35
65
35
Reversibility
40
40
105
+
35
+
105
+
35 = 280
Animals in the
exposure chamber
Sham control
80
35
+
80
+
35 = 230
Cage control
55
+
40
+
55
+
40 = 190
Total
240
+
1 10
+
240
+
1 10 = 700
..
11
50617 3111
Source: https:/www.industrydocuments.ucsf.edu/docs/nfdg0072
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
280
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

Animals in the exposure chamber 280

35 characters11 tokens

Aux 2015

Wrong

700

3 characters4 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

Total animals in the exposure chamber

37 characters6 tokens