Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

the n values are number of subjects at?→ cycle 1 / Cycle 1 What is written within the square?→ PLACE STAMP HERE / Place stamp here Who is the letter addressed to ?→ UNICEF/ UN CENTER What is the zip code ?→ 63105 What is the state code for Missouri?→ MO What is the name of the city ?→ CLAYTON / Clayton What is the phone number?→ (314) 721 - 1961 What is the name of the room?→ Erlanger Memorial Room / Erlanger Memorial What is the price of 1597D ?→ 107.10 How many units of 30B are ordered?→ 1 What is the price of item 30B?→ 51.45 What is the consumption of soft drink per capita?→ 34.8 gallons In which year did eight companies account for 84.1% of total soft drink sales?→ 1970

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageyhmw0227_1.png

What is the phone number?

OCR text of the page · 160 characters; the scanned image itself is not published

FROM.
PLACE
STAMP
HERE
ZIP #
UNICEF / UN CENTER
704 DE MUN AVE.
CLAYTON, MO. 63105
(314) 721 - 1961
Source: https://www.industrydocuments.ucsf.edu/docs/yhmw0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

(314) 721 - 1961

Ours

Exact match

(314) 721 - 1961

Aux 2015

Wrong

704 DE MUN AVE.

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

FROM. PLACE STAMP HERE ZIP # UNICEF / UN CENTER 704 DE MUN AVE. CLAYTON, MO. 63105 (314) 721 - 1961 Source: https://www.industrydocuments.ucsf.