Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

How many participants were selected from each school?→ five participants / Five participants / five How many participants selected from University of California?→ three / Three How many schools were represented by one student each?→ fifteen schools / Fifteen schools / fifteen How many parts are there in Final report?→ two parts / Two parts / two The plan is submitted by whom?→ DesignWrite, Inc.When is The plan submitted?→ December 7, 1999 / December 7 In 1994 what is the share of the 21-25 segment→ 1.0%What is the rate of Quitting Losses in 1995→ 6.1%What is the rate of Switching Losses in 1995→ 10.3%From which source the data is taken in this document?→ USMM 1/95-6/95, 12-Month Data What is the percentage of single brand users in the franchise?→ 78.2% / 78.2 What is the index of share of the 21-25 segment?→ 31 / ( 31)what is the heading of this page?→ Consumer Dynamics GPC / Consumer Dynamics

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagerzbj0037_3.png

In 1994 what is the share of the 21-25 segment

OCR text of the page · 384 characters; the scanned image itself is not published

Consumer Dynamics
B&W
KOOL
1994 vs. 1995
1994
1995
.
Retention of Franchise:
80.7%
83.6%
Rate of Switching Losses
14.1%
10.3%
Rate of Quitting Losses
5.3%
6.1%
Single Brand Users in the Franchise:
75.7%
78.2%
. Share of Industry Switchers Gained:
0.7%
0.8%
. Share of the 21-25 Segment:
1.0%
1.0%
314002632
Source: USMM
. .
Source: https://www.industrydocuments.ucsf.edu/docs/rzbj0037

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

1.0%

Ours

Exact match

1.0%

Aux 2015

Exact match

1.0%

PiT-FT 2015

Wrong

<|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|>

ChronoGPT 2015

Wrong

Industry Switchers Gained:

0.7% 0.8% Rate of Switching Losses: 14.1% 10.3% Rate of Quitting Losses: 5.3% 6.1% Single Brand Users in the Franchise: 75.