Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the ‘date’ on the letter?→ august 18, 1976 / August 18, 1976 Who wrote the letter?→ C. D. Fleet, Jr.Which “magazine’s copies” are enclosed within the letter?→ Upbeet / Upbeet magazine Which meeting is arranged in ‘sky top lodge’?→ food industries advisory committee / Food industries advisory committee / 1967 meeting What is the arrival date of Dr. &Mrs.William J.Darby ?→ june 7, 1967 What is the departure date of Dr. &Mrs.William J.Darby ?→ june 9, 1967 / June 9, 1967 What is the name of the company?→ BROWN & WILLIAMSON TOBACCO What is the “title” of the plot?→ FE-59 INCORPORATION / fe - 59 incorporation guinea pig What is the X axis of the ‘plot’?→ time (days) / Time (days)What is the Y axis of the ‘plot’?→ % incorporated What is the maximum value at x axis?→ 28 What is the maximum value at y axis?→ 25.0 What type of report is this?→ transmission report / Transmission Report

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagessmy0046_1.png

What is the name of the company?

OCR text of the page · 333 characters; the scanned image itself is not published

BW physmethods.doc
BEST
COPY
BROWN & WILLIAMSON
TOBACCO
The Physical Methodology for the Determination of
Filter Type, Ventilation, Circumference, Length, Paper
Permeability, and Tobacco Weight
Research & Development
Brown & Williamson Tobacco Company
Macon, GA
52495 2006
Source: https://www.industrydocuments.ucsf.edu/docs/ssmy0046

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

BROWN & WILLIAMSON TOBACCO

Ours

Partial · ANLS 0.76

Brown & Williamson Tobacco Company

Aux 2015

Partial · ANLS 0.69

BROWN & WILLIAMSON

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

BEST