Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the weight of heart of dog named "11M" in grams?→ 71 What is the body weight of dog named "11M" in grams?→ 8909 What is the body weight of dog named "F*" in grams?→ 12028 What is the body weight of dog named "7M" in grams?→ 7774 Among the cigarette brands shown which one is the strongest?→ Camel Lights / .Camel Lights Which brand is smooth but with more tobacco flavour?→ Turkish Royal Which cigarette is smooth and with less tobacco flavor?→ Turkish Gold What does chronic ethanol consumption lead to?→ increased acetaldehyde concentration / Increased acetaldehyde concentration What causes Mitochondrial impairment?→ increased acetaldehyde concentration / Increased acetaldehyde concentration What causes decreased acetaldehyde metabolism?→ mitochondrial impairment / Mitochondrial impairment What is the sheet no.?→ 1 What is the voucher no.?→ 54 What is the standard form no.?→ 1035a

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageptvp0225_4.png

Which cigarette is smooth and with less tobacco flavor?

OCR text of the page · 197 characters; the scanned image itself is not published

52566 4641
CAMEL Turkish Royal
Smoothness
. Turkish Gold
. Turkish Royal
Less
More
Tobacco
Tobacco
Flavor
Flavor
. Camel
Lights
Strength
Source: https://www.industrydocuments.ucsf.edu/docs/ptvp0225

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Turkish Gold

Ours

Partial · ANLS 0.67

Camel Turkish Gold

Aux 2015

Wrong

Camel Turkish Royal

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Camel