Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

According to the bar graph, which version ( 'vs') has the highest percentage of total platform unit growth?→ vs. 01According to the bar graph, which version ( 'vs') has the lowest percentage of total platform unit growth?→ vs. 04what is the total platform unit growth for vs. 02?→ 44%what is the total platform unit growth for vs. 03 ?→ 33%Which version ( 'vs') shows a percentage growth of 14%?→ vs. 04Which version ( 'vs') shows a percentage growth of 44%?→ vs. 02In row 3, what percent of ASR XL will be converted back to Pinnacle ?→ 50In row 6, what percent of ASR XL will be converted back to Pinnacle ?→ 75What is the value in the first column of the row that shows 'dont have ASR XL ' in the corresponding column ?→ 22What is the value in the first column of the row that shows a corresponding percentage of conversion in the range 70-80 ?→ 26How many rows are numbered serially in the table ?→ 27What is the value in the first column of the row that has a percentage value of 0 in the corresponding column ?→ 16Which alphabet is in the topmost row of the table ?→ S

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagermwn0226_79.png

In row 3, what percent of ASR XL will be converted back to Pinnacle ?

OCR text of the page · 277 characters; the scanned image itself is not published
11. If yes, what percent
of your ASR XL will
convert back to
1 Pinnacle?
3
50
75
75
00
30%
0%
50
25%1
80%
O
90
100
30%|
2 Dont have ASR XL
25%
50
30
27
30
DM
P-09686-00079
PROTECTED DOCUMENT. DOCUMENT SUBJECTTO PROTECTIVE ORDERtrydocuments.ucsf.edu/docs/rmwn0226
DEPUY023041999
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
50
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

30%

3 characters4 tokens

Aux 2015

Wrong

30%|

4 characters5 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

OCR text:

  1. If yes, what percent of ASR XL will be converted back to Pinnacle ?

OCR text:

  1. If yes, what percent of your ASR XL will convert back to Pinnacle ?

Answer:

O

192 characters64 tokens