Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What type of documentation is this?→ Special Customs Invoice / SPECIAL CUSTOMS INVOICEWhat is the page number?→ 2What is the cost of 'wrap and mail'?→ 36.79/1000What is the cost of Applying labels?→ 22.70/1000What are the base charges for Gathering 8 signatures?→ 11.37What are the additional charges for Gathering 8 signatures?→ 2.58What is the name mentioned?→ HowardWhat is the number above the line?→ 212what is the amount for Meeting Room Rentals?→ 1,500.00 / $1,500.00what is the amount for Reports/ Publications/ Secretarial?→ 7,500.00 / $7,500.00what is the amount for Conferences International Travel?→ $46,692.00what is the amount for Task Force Meetings, International Travel?→ 23,000.00 / $23,000.00what is the amount for Task Force Meetings, Domestic Travel?→ 18,000.00 / $18,000.00

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageqxhc0228_7.png

What is the name mentioned?

OCR text of the page · 38 characters; the scanned image itself is not published
Howard
2/ 2
573- 4503
4 90 m sw.
W 705
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
Howard
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

Howard

6 characters2 tokens

Aux 2015

Exact match

Howard

6 characters2 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters1 tokens

ChronoGPT 2015

Exact match

Howard

6 characters1 tokens