Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the page no mentioned in this document?→ Page 3 of 12 / Page 3 / 3What is the title mentioned in the header of this document?→ Trimegestone - Publication and Abstract Tracking Report / Trimegestone - Publication and Abstract Tracking ReportWhat is the title of the document ?→ Trimegestone - publication and Abstract tracking report / Trimegestone - Publication and Abstract Tracking ReportWhat is the date mentioned on the top right hand side of the document?→ 1/12/04What is the ‘date of invoice’?→ 6/3/75What is the ‘renewal date’?→ 12/1/75What is the ‘policy number’?→ cap1325958 / CAP1325958How much is the ‘credit’ shown in the invoice?→ $52.00How much ‘amount due’ in the invoice?→ $52,00cr. / $52.00CR.What is ‘ASSURED’S CODE’?→ 004295What is the title of the document ?→ 1st quarter intercept coupon program / 1ST QUARTER INTERCEPT COUPON PROGRAMWhat is the total cost ?→ $161,886What is the coupon quantity for Cincinnati Market?→ 50M

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagehrcy0227_4.png

What is the ‘policy number’?

OCR text of the page · 670 characters; the scanned image itself is not published
DATE OF
RENEWAL
12/1/75
Premiums are due and payable
INVOICE
6/3/75
DATE
on effective date of coverage
-EFFECTIVE
DATE
POLICY NUMBER
COMPANY
COVERAGE
PREMIUM
CREDIT
AMOUNT DUE
6/ 14/75
CAP1 325958
Cont1. CAs.
Cancelling Contents, liability
$52. 00
$52. 00CR.
insurance policy 6/14/75
Chick included
At 1615
THORNTON AGENCY, INC.
Payable tofill
PLEASE RETURN WITH PAYMENT
ASSURED'S CODE 004295
THORNTON AGENCY, INC.
F
Citizens' Commission on Science
TO
Law and The Food Supply
Insurance
1230 York Avenue at 66th Street175 MAIN STREET, WHITE PLAINS, N. Y. 10601
L
New York, New York 10021
TELEPHONE: 914 WH 9-8180
https://www.
Source: http
locuments.ucsf.edu/docs/hrcy0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
cap1325958CAP1325958
Models
4 of 4 columns · click a model to add or remove it

Ours

Partial · ANLS 0.91

CAP1 325958

11 characters10 tokens

Aux 2015

Wrong

Policy number

13 characters3 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

ChronoGPT

Question:

What is the 'policy number' of the 'ChronoGPT' language model?

ChronoGPT

Question:

What is the 'policy number' of the 'ChronoGPT' language

170 characters64 tokens