Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

From the 100, how many of them had baseline angiogram?→ 81 / n=81 Why were 3 rejected?→ Protocol Violation From the 103, How many were enrolled?→ 100 / N=100 How many Subjects signed the consent form?→ 103 / N=103 How many were given IVUS?→ 63 How many refused to continue in the study?→ 19 From the 100, how many of them had baseline angiogram?→ 81 From the 103 who signed the consent form, how many were enrolled?→ 100 From the 81, how many had IVUS?→ 63 / N=63 From the 100, how many of them had baseline angiogram?→ 81 / N=81 How many subjects were enrolled?→ 100 / N=100 Why were 3 rejected?→ Protocol violation / Protocol Violation How many Subjects signed out the consent form?→ 103 / N=103

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagenpvw0217_5.png

From the 100, how many of them had baseline angiogram?

OCR text of the page · 332 characters; the scanned image itself is not published

Subjects Signed Consent
(signed consent form)
N=103
study Subjects Enrolled drug
Protocol violation
N=100
N=3
Table 1)
Subject with baseline angiogram
Subject refused to continue in study
n=81
N=19
(Table 2)
IVUS
No IVUS
N=63
18
All SVG occluded or diseased
DWRITE 045517
Source: https://www.industrydocuments.ucsf.edu/docs/npvw0217

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

81

Ours

Partial · ANLS 0.50

n=81

Aux 2015

Wrong

N=103

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

N=103 N=3 N=3 N=3 N=3 N=3 N=3