Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagelpdl0226_11.png

What is the total number of respondents?

OCR text of the page · 1,029 characters; the scanned image itself is not published
Question 1: What does the word calories mean to you?
Total*
Men*
Women*
Total Respondents
(506=100% ) (249=100%) (257=100%)
Fattening food, fat in food, fat
building components, fattening
qualities in foods, concerns
weight
36%
28%
Energy, heat, unit of heat, unit
of energy, heat burned in body,
heat energy, amount of heat needed
to raise 1 gram of water 1 0
25
28
22
Body requirement, need them for
balanced diet, beneficial to body,
something vital to body, nourish-
ment in food, food value, unit of
nutrition
16
16
15
Measurement of food content, method
of counting food value, method of
measuring food consumption, food
weight, food is measured in calories,
different foods have different
amounts .
11
12
11
Vitamins in food, vitamin content,
sort of a vitamin, mineral
w
Just food
N
Something in food, content in food
N
Dieting, on a diet
NN N F
Protein in food, food value such as
high or low protein
All other
N
N H
Nothing
Don't know-
NN N H
*Adds to more than 100 % because some respondents gave more
than one answer.
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
506
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

506

3 characters4 tokens

Aux 2015

Wrong

11

2 characters3 tokens

PiT-FT 2015

Wrong

<|<|<|<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

64 characters64 tokens

ChronoGPT 2015

Wrong

Total* Men* Women* Total Respondents (506=100% ) (249=100%) (257=100%) Fattening food, fat in food, fat building components, fattening qualities in foods, concerns weight 36% 28% Energy

185 characters64 tokens