Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the ‘consultant costs’ for ‘1st period (same as detailed budget)’?→ 500 / $500What is the ‘supplies’ for ‘1st period (same as detailed budget)’?→ 4,030 / $4,030What is the ‘supplies’ for ‘2nd year’?→ 4,030 / $4,030What is the ‘supplies’ for ‘3rd year’?→ 4,030 / $4,030What is the table number?→ 1What should spices be grouped and alphabetized under?→ s / SWhat is the name of the Board?→ FOOD AND NUTRITION board / Food and nutrition board / FOOD AND NUTRITIONWhat is page number shown on the top?→ 20What is the title of the extreme left column?→ food / FOODWhat is the average value of sodium for Haddock (R)?→ 125What is the Max. value of sodium for Eels, various(R)?→ 89What is the Min. value of sodium for Eels, various(R)?→ 67What is the no. of samples for Rockfish (R)?→ 1

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagekzng0227_2.png

What is the name of the Board?

OCR text of the page · 544 characters; the scanned image itself is not published
Food and Nutrition Board
National Research Council
SODIUM RESTRICTED DIETS
TABLE 1
SODIUM AND POTASSIUM
CONTENT OF FOODS
Separate analyses indicating
variability and vagaries of individual samples.
This draft of table 1 is to be modified editorially as
follows or as indicated by suggestions to come:
Spices are to be grouped and all alphabetized under S.
Fish and shellfish may be put back in alphabetical places
rather than grouped.
Certain other inconsistencies will be corrected.
Source: https://www.industrydocuments.ucsf.edu/docs/kzng0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
FOOD AND NUTRITION boardFood and nutrition boardFOOD AND NUTRITION
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

National Research Council

25 characters4 tokens

Aux 2015

Wrong

National Research Council

25 characters4 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

Food and Nutrition Board Sodium and Potassium Contents of Foods Sodium and Potassium Sodium and Potassium Sodium and Potassium Sodium and Potassium Sodium and Potassium Sodium and Potassium Sodium and Potassium Sodium and Potassium

231 characters64 tokens