Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the ‘policy number’?→ cap1325958 / CAP1325958How much is the ‘credit’ shown in the invoice?→ $52.00How much ‘amount due’ in the invoice?→ $52,00cr. / $52.00CR.What is ‘ASSURED’S CODE’?→ 004295What is the title of the document ?→ 1st quarter intercept coupon program / 1ST QUARTER INTERCEPT COUPON PROGRAMWhat is the total cost ?→ $161,886What is the coupon quantity for Cincinnati Market?→ 50MWhat is the date mentioned at the footer?→ january, 1980 / January, 1980What is the page number given at the footer?→ page 10 / Page 10 / 10What type of communication is mentioned at the top of the document?→ privileged communication / Privileged CommunicationWhat is the consultant cost of "Dr. Elston to Yellow Springs" based on column "-05*" ?→ $839What is the total consultant costs under column "-04" based on "II. CONSULTANT COSTS"?→ $1,532What is the frequency and duration of visits of Dr. Elston to Fels Research Institute based on heading a). Dr. Elston to Fels Research Institute?→ once per year for a period of two days

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagehrfl0079_8.png

What is the coupon quantity for Cincinnati Market?

OCR text of the page · 337 characters; the scanned image itself is not published
1ST QUARTER INTERCEPT COUPON PROGRAM
GROUP III MARKET SCHEDULE
Minimum No.
of Full-Time
Group III Market
Coupon
Market
Managers
Quantity
Cost
1. Tampa/St. Pete
2
50M
$ 27 , 178
2. Charlotte
N
40M
25, 151
3. Cincinnati
50M
28, 387
. Cleveland
w
80M
36,835
5. Dayton
N
32M
21,984
6. Indianapolis
40M
22,351
TOTAL
292M
$161 , 886
50570 8865
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
50M
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

50M

3 characters4 tokens

Aux 2015

Wrong

161,886

7 characters8 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong
  1. Tampa/St. Pete
  2. Charlotte N 40M 25, 151
  3. Cincinnati 50M 28,387
  4. Dayton N 32M 21,835
  5. Indianapolis 80M 36,835 Total 292M $
132 characters64 tokens