Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the quantity of the order?→ 630 What is the code for Unacceptable?→ 1 What is the code for Average?→ 3 What is code 2 for?→ Minimally Acceptable / Minimally acceptable What is code 5 for?→ Comprehensive What is code 4 for?→ Above Average / Above average What is title of the flowchart?→ EAGAR subject flow / Eagar Subject Flow How many were given IVUS?→ 63 / N=63 From the 100, how many of them had baseline angiogram?→ 81 / n=81 Why were 3 rejected?→ Protocol Violation From the 103, How many were enrolled?→ 100 / N=100 How many Subjects signed the consent form?→ 103 / N=103 How many were given IVUS?→ 63

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagenpvw0217_4.png

What is title of the flowchart?

OCR text of the page · 351 characters; the scanned image itself is not published

EAGAR subject flow
Subjects Signed Consent
(signed consent form)
N=103
study Subjects Enrolled drug
Protocol violation
N=100
N=3
(Table 1)
Subject with baseline angiogram
Subject refused to continue in study
n=81
N=19
Table 2)
IVUS
No IVUS
N=63
18
All SVG occluded or diseased
DWRITE 045516
Source: https://www.industrydocuments.ucsf.edu/docs/npvw0217

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

EAGAR subject flowEagar Subject Flow

Ours

Exact match

EAGAR subject flow

Aux 2015

Wrong

EAGAR subject flow

Subjects Signed Consent (signed consent form) N=103 study Subjects Enrolled drug Protocol violation N=100 N=3 (Table 1) Subject with baseline angiogram Subject refused to continue in study n=81

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

N=103 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3 N=3