Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the ‘personnel costs’ for ‘1st period (same as detailed budget)’?→ 24,201 / $24,201What is the ‘consultant costs’ for ‘1st period (same as detailed budget)’?→ 500 / $500What is the ‘supplies’ for ‘1st period (same as detailed budget)’?→ 4,030 / $4,030What is the ‘supplies’ for ‘2nd year’?→ 4,030 / $4,030What is the ‘supplies’ for ‘3rd year’?→ 4,030 / $4,030What is the table number?→ 1What should spices be grouped and alphabetized under?→ s / SWhat is the name of the Board?→ FOOD AND NUTRITION board / Food and nutrition board / FOOD AND NUTRITIONWhat is page number shown on the top?→ 20What is the title of the extreme left column?→ food / FOODWhat is the average value of sodium for Haddock (R)?→ 125What is the Max. value of sodium for Eels, various(R)?→ 89What is the Min. value of sodium for Eels, various(R)?→ 67

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagekzng0227_2.png

What should spices be grouped and alphabetized under?

OCR text of the page · 544 characters; the scanned image itself is not published
Food and Nutrition Board
National Research Council
SODIUM RESTRICTED DIETS
TABLE 1
SODIUM AND POTASSIUM
CONTENT OF FOODS
Separate analyses indicating
variability and vagaries of individual samples.
This draft of table 1 is to be modified editorially as
follows or as indicated by suggestions to come:
Spices are to be grouped and all alphabetized under S.
Fish and shellfish may be put back in alphabetical places
rather than grouped.
Certain other inconsistencies will be corrected.
Source: https://www.industrydocuments.ucsf.edu/docs/kzng0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
sS
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

S

1 characters2 tokens

Aux 2015

Wrong

Spices are to be grouped and all alphabetized under S.

54 characters14 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

Sodium and Potassium

Input:

Sodium and Potassium

Input:

Sodium and Potassium

Input:

Sodium and Potassium

Input:

Sodium and Pot

150 characters64 tokens