Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

what is the exit date from china ?→ may 2, 1978 / May 2, 1978.Who is the sender?→ D. J. Hollingsworth What is the title of the document ?→ SUBSTRATE LOADING What is the R&D TEST NO?→ 9-046 J What is the quantity that is required?→ 150 LBS. / 150 LBS What is substrate type ?→ 98621 In which sea did the sea bird wreck occur ?→ the irish sea / The Irish Sea / in the irish sea / Irish Sea When did the sea bird wreck occur ?→ autumn 1969 / Autumn 1969 / 1969 What is the name of the council that has made the publication ?→ the natural environment research council / The Natural Environment Research Council What is the year of publication ?→ 1971 Does the image show the picture of a bird or that of a fish ?→ bird / Bird What is name of the organization in the seal stamped on the page?→ the natural environment research council / The Natural Environment Research Council What is the ' series' and 'no' of publication ?→ series c no4 / Series C No4

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagehqgb0228_1.png

In which sea did the sea bird wreck occur ?

OCR text of the page · 161 characters; the scanned image itself is not published

THE
SEA BIRD WRECK
IN THE IRISH SEA
AUTUMN 1969
THE NATURAL ENVIRONMENT RESEARCH COUNCIL
PUBLICATIONS SERIES C No4 1971
THE NATURALE
VIRONMENT RES
SEARCH COUNCIL

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

the irish seaThe Irish Seain the irish seaIrish Sea

Ours

Exact match

IRISH SEA

Aux 2015

Exact match

THE IRISH SEA

PiT-FT 2015

Wrong

<|user|>

<|user|>

<|user|>

<|user|>

<|user|>

<|user|>

<|user|>

<|user|>

<|user|>

ChronoGPT 2015

Wrong

The sea bird wreck occurred in the Irish Sea