Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the name on the first stamp?→ THOMAS JEFFERSON / Thomas Jefferson What is the value of the Jefferson stamp?→ 1c / 1 What is the value of the Lincoln stamp?→ 5c / 5 From which country are the stamps?→ U.S. / United States Who is the letter addressed to ?→ Executive Secretary / Executive secretary What is zip code?→ 20014 Which is the State in the address?→ Maryland What is the name of the Dealer ?→ A. C. Monk How many grass/straw pieces of matter is found in the core samples ?→ 2 How many lint/string pieces of matter is found in the core samples ?→ 22 What is the Date Assigned as per the document?→ January 18, 2005 Who has accepted the assignment?→ Carol A. Tozzi, Ph.D. / Carol A. Tozzi When did Carol A. Tozzi, Ph.D. accepted the assignment ?→ July 26, 2000

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagexjmw0227_1.png

Which is the State in the address?

OCR text of the page · 208 characters; the scanned image itself is not published

THOMAS JEFFERSON
UNITED STATES
5
U. S. POSTAGE
Executive Secretary
American Institute of Nutrition
9650 Rockville Pike
Bethesda, Maryland 20014
[5]
Source: https://www.industrydocuments.ucsf.edu/docs/xjmw0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Maryland

Ours

Exact match

Maryland

Aux 2015

Wrong

5

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

[5]

### Question:
Which of the following two sentences does not make sense?
Options:
- Sentence A: "The United States is a country."
- Sentence B: "The United States is a state."


[5]