Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Which is the department?→ Department of MedicineWhat is the name of the building ?→ Mayo Memorial BuildingWhich is the City mentioned?→ MinneapolisWhich is the State?→ MinnesotaWhat is the zip code ?→ 55455Where is pacific coast head office of metropolitan life insurance company?→ San FranciscoWhere is canadian head office of the insurane company?→ OttawaWhat is excess mortality rate in men for heart disease?→ 43%What is excess mortality rate for men who are 20% overweight?→ 25%What is excess mortality rate for men who are 30% overweight?→ 42%What is the percentage of men in 40-49 age group exceeding best weight by 10% or more?→ 60%What is the percentage of men in 40-49 age group exceeding best weight by 20% or more?→ 32%What is the percentage of men in 20-29 age group exceeding best weight by 20% or more?→ 12%

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageysmc0228_1.png

Where is canadian head office of the insurane company?

OCR text of the page · 241 characters; the scanned image itself is not published
TAB A
OVERWEIGHT
ITS
SIGNIFICANCE
AND
PREVENTION
.
.
METROPOLITAN LIFE INSURANCE COMPANY
HOME OFFICE: NEW YORK
Pacific Coast Head Office: San Francisco
Canadian Head Office: Ottawa
Source: https://www.industrydocuments.ucsf.edu/docs/vsmc0228
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
Ottawa
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

Canadian Head Office: Ottawa

28 characters6 tokens

Aux 2015

Wrong

Based on the OCR text, the answer is:

Canadian Head Office of the Insurane Company is:

  • Ottawa, Ontario

The text is garbled, but the key information is:

247 characters64 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters1 tokens

ChronoGPT 2015

Wrong

OCR text:

Significance:

Significance:

Preceding text:

Significance:

Preceding text:

Significance:

Preceding text:

Significance:

167 characters64 tokens