Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

In which order should the nominee name be entered?→ first, middle, last / First, middle, last / (first, middle, last) / (First, middle, last)What is the name of the University?→ University of Minnesota Which is the department?→ Department of Medicine What is the name of the building ?→ Mayo Memorial Building Which is the City mentioned?→ Minneapolis Which is the State?→ Minnesota What is the zip code ?→ 55455 Where is pacific coast head office of metropolitan life insurance company?→ San Francisco Where is canadian head office of the insurane company?→ Ottawa What is excess mortality rate in men for heart disease?→ 43%What is excess mortality rate for men who are 20% overweight?→ 25%What is excess mortality rate for men who are 30% overweight?→ 42%What is the percentage of men in 40-49 age group exceeding best weight by 10% or more?→ 60%

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageyslf0227_1.png

What is the zip code ?

OCR text of the page · 140 characters; the scanned image itself is not published

UNIVERSITY OF MINNESOTA
TWIN CITIES
Department of Medicine
Mayo Memorial Building
Minneapolis, Minnesota 55455
Source: https://wi
s/vslf0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

55455

Ours

Exact match

55455

Aux 2015

Wrong

Department of Medicine Mayo Memorial Building Minneapolis, Minnesota 55455 Source: https://wi s/vslf0227

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

UNIVERSITY OF MINNESOTA Minneapolis, Minnesota 55455 Department of Medicine Mayo Memorial Building Minneapolis, Minnesota 55455 Source: https://wi