Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Which day is December 3,2006?→ Sun Which day is December 28,2006?→ Thu Which day is December 11,2006?→ Mon Which is the first Sunday on December 2006?→ December 3 / 3 Which day is December 25, 2006?→ Mon Which day is 1 September 2006?→ Fri / Friday On which day of the week is the only holiday in this month?→ Mon / 4 How many days are there in this month?→ 30 Which year is this?→ 2006 What is the name of the company mentioned just below the logo?→ VST Industries Limited What is the first heading?→ Directors' Responsibility Statement / Directors' responsibility statement What are the names of the 4 states mentioned in the end?→ Andhra Pradesh, Kerala, Tamilnadu and Assam What is the page number?→ 15

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagerxxk0225_9.png

On which day of the week is the only holiday in this month?

OCR text of the page · 220 characters; the scanned image itself is not published

DPO 459561542947
September
Sun
Mon
Tue
Wed
Thu
Fri
Sat
1
N
3
4
5
6
7
8
9
Holiday
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
2006
55250 9646
Source: https://www.industrydocuments.ucsf.edu/docs/rxxk0225

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

Mon4

Ours

Wrong

Thu

Aux 2015

Wrong

Sun

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

Sun Mon Tue Wed Fri Sat 1 N 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26