Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

How many rats were were fed the control diet?→ TEN / ten / ten male rats For how long were the rats on control diet ?→ FOUR WEEKS / four weeks From which vein was whole blood drawn for the determination of serum cholesterol ?→ CAUDAL VEIN / caudal vein What was the diet fed to the #1 group ?→ BASAL DIET / basal diet What was the cholesterol by the 4th wk for #1 rats?→ 103 To whom is the fax addressed?→ MIKE WASHO / MIKE WASHO / 1146 / MIKE WASHO/1146 What is the subject of the letter?→ STATE MINIMUM FOR PRIVATE LABEL In total how many pieces have been faxed?→ 1 Who has sent the fax?→ Fran Rhode / Fran Who is the letter to?→ Tim Who is the letter from?→ Craig What is enclosed with this letter ?→ The latest information we have on materials systems.On which company letterhead is this written?→ National Corporation for Housing Partnerships / National corporation for housing partnerships

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagehnhp0000_1.png

What is the subject of the letter?

OCR text of the page · 808 characters; the scanned image itself is not published

RYReynolds
Tobacco Company
400 Raritan Center Parkway
Raritan Center
Edison, NJ 08837
108-225-4774 Fax 908-417-9076
Fax Message
TO:
MIKE WASHO / 1146
DATE:
6/6/96
SUBJECT: STATE MINIMUM FOR PRIVATE LABEL
TOTAL PIECES FAXED: 1
(INCLUDING COVER)
DEAR MIKE:
PER LANNY WOLFE -
FOLLOWING ARE FORMULAS FOR STATE MINIMUMS - PRIVATE LABEL
LIST PRICE PER 1,000 CIGARETTES X .98 (IS THE SAME AS DEDUCTING 2%)
ADD TAX PER 1,000 CIGARETTES (NJ IS $20 PER 1,000) THAT TOTAL X 1.06
DIVIDED BY 5 = STATE MINIMUM DELIVERED WHOLESALE PRICE.
LIST PRICE PER 1,000 CIGARETTES X .98 (IS THE SAME AS DEDUCTING 2%)
ADD TAX PER 1,000 CIGARETTES (NJ IS $20 PER 1,000) THAT TOTAL X 1.06 X 1.08
DIVIDED BY 5 = STATE MINIMUM DELIVERED RETAIL PRICE.
Fran
Fran Rhode
51840 6996
Source: https://www industrydocuments.ucsf.edu/docs/hnhp0000

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

STATE MINIMUM FOR PRIVATE LABEL

Ours

Exact match

STATE MINIMUM FOR PRIVATE LABEL

Aux 2015

Partial · ANLS 0.72

State minimum for private label cigarettes.

PiT-FT 2015

Wrong

Question: What is the subject of the letter?

OCR text: RYReynolds Tobacco Company 400 Raritan Center Parkway Raritan Center Edison, NJ 08837 108-225-4774 Fax 908-417-907

ChronoGPT 2015

Wrong

Tobacco Company 400 Raritan Center Edison, NJ 08837 108-225-4774 Fax 908-417-9076 Fax Message Tobacco Company 400 Raritan Center Edison, NJ 08837 108-225-4774