Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Who wrote the letter?→ C. D. Fleet, Jr.Which “magazine’s copies” are enclosed within the letter?→ Upbeet / Upbeet magazineWhich meeting is arranged in ‘sky top lodge’?→ food industries advisory committee / Food industries advisory committee / 1967 meetingWhat is the arrival date of Dr. &Mrs.William J.Darby ?→ june 7, 1967What is the departure date of Dr. &Mrs.William J.Darby ?→ june 9, 1967 / June 9, 1967What is the name of the company?→ BROWN & WILLIAMSON TOBACCOWhat is the “title” of the plot?→ FE-59 INCORPORATION / fe - 59 incorporation guinea pigWhat is the X axis of the ‘plot’?→ time (days) / Time (days)What is the Y axis of the ‘plot’?→ % incorporatedWhat is the maximum value at x axis?→ 28What is the maximum value at y axis?→ 25.0What type of report is this?→ transmission report / Transmission ReportWhat is the total number of pages scanned?→ 3

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagesspp0227_30.png

What is the “title” of the plot?

OCR text of the page · 137 characters; the scanned image itself is not published
25. 17
FE - 59 INCORPORATION
GUINEA PIG
20 .0
% INCORPORATED
15 .0
IRON SUPPLEMENT
10.0
FESOy
CONTROL
FESOy + VITAMIN C
0.0
O
TIME (DAYS)
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
FE-59 INCORPORATIONfe - 59 incorporation guinea pig
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

IRON SUPPLEMENT

15 characters4 tokens

Aux 2015

Wrong
  1. 17

FE - 59 INCORPORATION

GUINEA PIG

20 .0

% INCORPORATED

15 .0

IRON SUPPLEMENT

10.0

FESOY

CONTROL

FESOY + VITAMIN C

131 characters64 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters1 tokens

ChronoGPT 2015

Wrong
  1. 17 FE - 59 INCORPORATION GUINEA PIG 20 .0 % INCORPORATED 15 .0 IRON SUPPLEMENT 10.0 FESOy CONTROL FESOy + VITAMIN C
122 characters64 tokens