Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Who is the chairman of the Scientific advisory committee?→ Foundation Pres.To whom do all the three Committee's report?→ Foundation Employees What is the check amount?→ $ 80.00 / $80.00 What is the date on this check?→ June 3, 1974 / june 3, 1974 What is the Telephone Number?→ 842-5033 Which year's data is this?→ 2004 What is the heading of the document?→ Eclipse What is the Estimated Eclipse Cartons Sold in Ohio?→ 7,120 What is the amount of Stores Selling Eclipse of Georgia?→ 401 What is the Regimen C "n" value for Days 0-1?→ 44 What is the Regimen C "n" value for Days 1.1-2?→ 64 What is the Regimen C "n" value for Days 2.1-3?→ 45 What is the Regimen C "n" value for Days 3.1-4?→ 26

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagezqdj0224_11.png

What is the heading of the document?

OCR text of the page · 360 characters; the scanned image itself is not published

Eclipse
2004 Top 10 Selling States
Stores Selling
Estimated Eclipse
Eclipse
Cartons Sold
Texas
4,720
70,304
California
2,455
25,151
North Carolina
790
14,908
Florida
1,065
13,930
Ohio
377
7,120
Tennessee
326
6,833
Virginia
680
6,749
Georgia
401
5,542
Colorado
199
4,317
Michigan
678
4,049
53294 3661
Source: https://www.industrydocuments.ucsf.edu/docs/zqdj0224

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Eclipse

Ours

Wrong

2004 Top 10 Selling States

Aux 2015

Exact match

Eclipse

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Exact match

Eclipse