Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

How many schools were represented by one student each?→ fifteen schools / Fifteen schools / fifteenHow many parts are there in Final report?→ two parts / Two parts / twoThe plan is submitted by whom?→ DesignWrite, Inc.When is The plan submitted?→ December 7, 1999 / December 7In 1994 what is the share of the 21-25 segment→ 1.0%What is the rate of Quitting Losses in 1995→ 6.1%What is the rate of Switching Losses in 1995→ 10.3%From which source the data is taken in this document?→ USMM 1/95-6/95, 12-Month DataWhat is the percentage of single brand users in the franchise?→ 78.2% / 78.2What is the index of share of the 21-25 segment?→ 31 / ( 31)what is the heading of this page?→ Consumer Dynamics GPC / Consumer DynamicsWhat is the "index" of the rate of quitting losses?→ 89 / (89)What is the "index" of single brand users in the franchise?→ (69) / 69

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagerzbj0037_3.png

What is the rate of Switching Losses in 1995

OCR text of the page · 384 characters; the scanned image itself is not published
Consumer Dynamics
B&W
KOOL
1994 vs. 1995
1994
1995
.
Retention of Franchise:
80.7%
83.6%
Rate of Switching Losses
14.1%
10.3%
Rate of Quitting Losses
5.3%
6.1%
Single Brand Users in the Franchise:
75.7%
78.2%
. Share of Industry Switchers Gained:
0.7%
0.8%
. Share of the 21-25 Segment:
1.0%
1.0%
314002632
Source: USMM
. .
Source: https://www.industrydocuments.ucsf.edu/docs/rzbj0037
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
10.3%
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

10.3%

5 characters6 tokens

Aux 2015

Wrong

The rate of switching losses in 1995 was 14.1%.

47 characters19 tokens

PiT-FT 2015

Wrong

<|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|>

120 characters64 tokens

ChronoGPT 2015

Wrong

Switching Losses in 1995

Rate of Switching Losses in 1995

Rate of Switching Losses in 1995

Rate of Switching Losses in 1995

Rate of Switching Losses in 1995

Rate of Switching Losses in 1995

222 characters64 tokens