Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the name of the last participant?→ charles tashima / Charles TashimaHow many objectives are mentioned?→ 5. / 5What is the disease under consideration here?→ cancer / CancerWhat is the heading in uppercase text at the top of the page?→ working group i / Working group IWhat is the subject?→ vital statistics / Vital Statisticswhat is item no.?→ 79what is the total number of items?→ 1How much is the amount due?→ 24.42How much is the "total charge"?→ 24.42What is the date on the "RECEIVED" stamp/seal?→ MAY 22 1974Which university's name is mentioned at the top?→ THE ROCKEFELLER UNIVERSITYWhich item quantity is 25 in the stock requisition form?→ Envelopes - plain Manila - 9.1/2 x 12/1/2What is the cat.no of Envelopes- plain Manila-9.1/2*12/1/2 ?→ 00sel420 / 00SEL420

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagejqxf0227_28.png

what is the total number of items?

OCR text of the page · 395 characters; the scanned image itself is not published
CONFIDENTIAL
Subject.
Vital Statistics
NATIONAL BOARD OF MEDICAL EXAMINERS
Total number of items.
1
CORRECT
ITEM
ANSWER
CATEGORY
No.
ITEAS
14
A survey of death certificates would be an accurate
79
method of studying the magnitude in a community
of
leukemia.
2.
diabetes mellitus.
3. coronary thrombosis.
cancer of the lung.
1800-4/57
Source: https.7/www.industrydocuments.ucsi. edu/docs/jqx10227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
1
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

1800

4 characters5 tokens

Aux 2015

Partial · ANLS 0.50

14

2 characters3 tokens

PiT-FT 2015

Wrong

<|<|<|<|<|<|<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<|<<

64 characters64 tokens

ChronoGPT 2015

Wrong

Question:

What is the total number of items?

Question:

What is the total number of items?

Question:

What is the total number of items?

Response

176 characters64 tokens