Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagemfnf0004_2.png

What is the agenda item at 10:05 on july 17?

OCR text of the page · 349 characters; the scanned image itself is not published
Agenda
Wednesday, July 17
10:00
Welcome Speech (Mr. TADA)
10:05
Report on Europe and US launch (Mr. Bass)
11:00
Launch Plan for NSP Product (Mr. Fujimoto)
12:00
Launch
13:00
License Negotiation
16:30
Adjourn
18:30
Dinner
Tuesday, July 18
10:00
License Negotiation
12:00
Adjourn
51722 0577
Source: https://www.industrydocuments.ucsf.edu/docs/minf0004
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
Report on Europe and US launch (Mr. Bass)Report on Europe and US launch
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

Report on Europe and US launch (Mr. Bass)

41 characters12 tokens

Aux 2015

Wrong

10:00 Welcome Speech (Mr. TADA) 10:05 Report on Europe and US launch (Mr. Bass) 11:00 Launch Plan for NSP Product (Mr. Fujimoto) 12:00 Launch 13:0

146 characters64 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

10:05

Question:

What is the agenda item at 10:05 on july 17?

OCR text: Agenda Wednesday, July 17 10:00 Report on Europe and US launch (Mr. TADA) 11:00 Launch 13:00

169 characters64 tokens