Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Among dieters and weight watchers, how many persons used ‘21-30 times’ ?→ 97 Among dieters and weight watchers, how many persons used ‘11-20 times’ ?→ 225 Among dieters and weight watchers, how many persons used ‘6-10 times’ ?→ 128 What is the year of review of pinnacle ultamet experience?→ 2011 What are the ‘dates implanted’?→ 2001~2008 / 2001 2008 How many ‘total implanted’?→ 1100 How many’ no reported revisions’?→ 1087 (98.8%) / 1087 How many ‘ no reported metal reaction revisions’?→ 1094 (99.5%) / 1094 What is the percentage shown under ‘no reported revisions’?→ 98.8%What is the percentage shown under ‘no reported metal reaction revisions’?→ 99.5%What is the first month plotted along the x axis?→ Oct What is the last month plotted along the x axis?→ Mar What is the share of high market in dec?→ 1.5

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagefhxn0226_2.png

How many’ no reported revisions’?

OCR text of the page · 277 characters; the scanned image itself is not published

2011 Review of Pinnacle Ultamet Experience for
Drs. Griffin and Mason
Dates
Total
No Reported
No Reported
Implanted
Implanted
Revisions
Metal Reaction
Revisions
2001-2008
1 100
1087
1094
(98.8%)
(99.5%)
P-00594-00002
Source: https://www.industrydocuments.ucsf.edu/docs/fhxn0226

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

1087 (98.8%)1087

Ours

Partial · ANLS 0.50

1094

Aux 2015

Wrong

2011 Review of Pinnacle Ultamet Experience for Drs. Griffin and Mason
Total
No Reported
No Reported
Implanted
Implanted
Revisions
Metal Reaction
Revisions
2001-2008
1 100
1087
10

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

No reported revisions’

Question:

What is the most common type of language model?

OCR text: OCR text: OCR text: OCR text: OCR text: OCR text: OCR text: OCR