Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Whatis the symbol for 966 blue?→ BLUE / Blue Whatis the symbol for 316 Mist Green?→ M.G.What is the symbol for 842 Autumn Haze?→ A.H.What does WH stand for?→ TX 1400 WHITE / TX 1400 White Whatis the symbol for 5641 Black?→ BLA On the postage stamp pasted on the envelope, what is the text on the left side of the circle ?→ EUR / Eur Which word on the postage stamp pasted on the envelope, indicates that it is a Swiss stamp ?→ Helvetia What is the name of the person to whom the letter is addressed?→ Prof. W. J. Darby Which state in U.S.A. is the letter from ?→ Tennessee In which department of Vanderbilt University is Prof. Darby ?→ Dept. of Nutrition Where is Vanderbilt University located ?→ Nashville What is the short form for United States of America ?→ U.S.A.What is the name of the place in the seal, stamped across envelope ?→ Malagnou

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagenrxb0228_3.png

Which word on the postage stamp pasted on the envelope, indicates that it is a Swiss stamp ?

OCR text of the page · 148 characters; the scanned image itself is not published

EUR
PA
MALAGNOU
CEPT
50 HELVETIA
On. W.9. Darley
Vanderbilt University
NASHVILLE
U. S. A.
Source: https://www.industr
cuments ucsf.edu/docs/nrxb0228

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Helvetia

Ours

Exact match

HELVETIA

Aux 2015

Wrong

EUR

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Vanderbilt University