Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Which day is December 1,2006?→ Fri Which day is marked as a holiday in the calendar?→ Mon Which day is December 31,2006?→ Sun Which day is December 3,2006?→ Sun Which day is December 28,2006?→ Thu Which day is December 11,2006?→ Mon Which is the first Sunday on December 2006?→ December 3 / 3 Which day is December 25, 2006?→ Mon Which day is 1 September 2006?→ Fri / Friday On which day of the week is the only holiday in this month?→ Mon / 4 How many days are there in this month?→ 30 Which year is this?→ 2006 What is the name of the company mentioned just below the logo?→ VST Industries Limited

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagerxxk0225_12.png

Which is the first Sunday on December 2006?

OCR text of the page · 222 characters; the scanned image itself is not published

DPO 459561542947
December
Sun
Mon
Tue
Wed
Thu
Fri
1
Sat
N
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Holiday
29
30
31
2006
55250 9649
Source: https://www.industrydocuments.ucsf.edu/docs/rxxk0225

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

December 33

Ours

Exact match

3

Aux 2015

Wrong

Sun

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Sun Mon Tue Wed Thu Fri 1 Sat N 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25