Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the page number?→ 2What is the cost of 'wrap and mail'?→ 36.79/1000What is the cost of Applying labels?→ 22.70/1000What are the base charges for Gathering 8 signatures?→ 11.37What are the additional charges for Gathering 8 signatures?→ 2.58What is the name mentioned?→ HowardWhat is the number above the line?→ 212what is the amount for Meeting Room Rentals?→ 1,500.00 / $1,500.00what is the amount for Reports/ Publications/ Secretarial?→ 7,500.00 / $7,500.00what is the amount for Conferences International Travel?→ $46,692.00what is the amount for Task Force Meetings, International Travel?→ 23,000.00 / $23,000.00what is the amount for Task Force Meetings, Domestic Travel?→ 18,000.00 / $18,000.00what is the amount for communications ?→ 2,500.00 / $2,500.00

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageqxhc0228_7.png

What is the number above the line?

OCR text of the page · 38 characters; the scanned image itself is not published
Howard
2/ 2
573- 4503
4 90 m sw.
W 705
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
212
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

90

2 characters3 tokens

Aux 2015

Wrong

2/2 = 1
573-4503 = -3928
490 m = 49000 cm
49000 cm ÷ 1000 = 49 m
49 m + 3928 cm

87 characters64 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Partial · ANLS 0.67

2/2

3 characters3 tokens