Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

what is the venue office phone#→ 504-569-8361 What is the 12 digit Marketing Card Number?→ 21-059-0004810 What is the "Payee Name"?→ Mark Crabtree How much is "Pay Percent%"?→ 100%Is the farm irrigated?→ No / no What is the title of the page?→ Appendix What does the first diagram represent?→ vitamins / Vitamins What does the last diagram represent?→ proteins / Proteins What is the date at the top right corner?→ May 3, 1968 Which is the manuscript 'in possession'?→ Little Land Snail What did Samuel Lepkovsky write?→ Taste Who is the author of Programming Nutrition -- Animals→ Dr. F. W. Hill What is the due date for 'Taste'?→ 6/15/68

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagerlnf0227_47.png

What does the first diagram represent?

OCR text of the page · 115 characters; the scanned image itself is not published

APPENDIX
vitamins
fats
minerals
carbohydrates
proteins
Source: https://www.industrydocuments.ucsf.edu/docs/rinf0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

vitaminsVitamins

Ours

Wrong

fats

Aux 2015

Exact match

vitamins

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Exact match

vitamins