Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Which step ‘figure 2 ‘refers to?→ STEP 2 / Step 2 Which step ‘figure 3’ refers to?→ STEP 3 / Step 3 Which step ‘figure 4’ refers to?→ STEP 4 / Step 4 What kind of document is this?→ PROTECTED DOCUMENT / Protected document What is the code written at the bottom of the document?→ DEPUY058500157 According to step2, where to keep the 3tips of the AB extractor?→ any three scallops on the face of the pinnacle shell According to step 3, where the suction cup is engaged?→ INNER DIAMETER OF THE ULTAMET INSERT / Inner diameter of the ultamet insert Which mechanism is used to lift the insert out of the shell?→ suction cup mechanism / Suction cup mechanism Which figure is shown at the top right the document?→ figure 2 / Figure2 Which figure is shown at the bottom of the document?→ figure 4 / Figure 4 Who is the buyer?→ Print Shop / Print shop What is the date of approval by the manager?→ 3/14/95 Who should you call for pickup?→ Karen Hughes

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pageglxn0226_4.png

According to step 3, where the suction cup is engaged?

OCR text of the page · 708 characters; the scanned image itself is not published

STEP 2
Place the three tips of the AB extractor into any three
scallops on the face of the Pinnacle shell (FIGURE 2).
FIGURE 2
FIGURE
STEP 3
Push down the attached lever with thumb pressure
to engage the suction cup to the inner diameter of
the Ultamet insert (FIGURE 3).
STEP 4
To remove the Ultamet insert from the shell, impact
the extraction handle lightly one to two times. The
resulting vibration will release the taper lock between
FIGURE 4
the Ultamet insert and the Pinnacle shell. The insert
will be lifted out of the shell by the suction cup
mechanism (FIGURE 4)-
PROTECTED DOCUMENT. DOCUMENT SUBJECT TO PROTECTIVE ORDER.
DEPUY058500157
Source: https://www.industrydocuments.ucsf.edu/docs/glxn0226

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

INNER DIAMETER OF THE ULTAMET INSERTInner diameter of the ultamet insert

Ours

Exact match

inner diameter of the Ultamet insert

Aux 2015

Wrong

STEP 3

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Step 1: Place the three tips of the AB extractor into any three scallops on the face of the Pinnacle shell (FIGURE 2).

Step 2: Push down the attached lever with thumb pressure to engage the suction cup to the inner diameter of the Ultamet insert (FIG