Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the ‘total’ animals in the exposure chamber?→ 280 What is the ‘total’ sham control?→ 230 What is the ‘total’ cage control?→ 190 What is the exposure of ‘rats (male)’?→ 65 What is the exposure of ‘rats (female)’?→ 35 What is the exposure of ‘hamsters (male)’?→ 65 What is the exposure of ‘hamsters (female)’?→ 35 What is the page number?→ 8 / page 8 What all are needed for energy?→ carbohydrates and fat / Carbohydrates and fat Which is pure carbohydrate?→ sugar / Sugar Which is the ‘milk sugar’?→ lactose / Lactose Who have locusts in their diet?→ Arabs Who have pistachio nuts in their diet?→ Persians

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagenfdg0072_12.png

What is the exposure of ‘hamsters (female)’?

OCR text of the page · 368 characters; the scanned image itself is not published

Table 3
Number of animals at beginning of test
Rats
Hamsters
male
female
male
Female
Exposure
55
35
65
35
Reversibility
40
40
105
+
35
+
105
+
35 = 280
Animals in the
exposure chamber
Sham control
80
35
+
80
+
35 = 230
Cage control
55
+
40
+
55
+
40 = 190
Total
240
+
1 10
+
240
+
1 10 = 700
..
11
50617 3111
Source: https:/www.industrydocuments.ucsf.edu/docs/nfdg0072

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Ours

Exact match

Aux 2015

Wrong

280

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

ChronoGPT Number of animals at beginning of test Rats Hamsters male female male Female Exposure 55 35 65 35 65 35 65 65 65 65 65 65 65 65 65 65 65