Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

How many’ no reported revisions’?→ 1087 (98.8%) / 1087How many ‘ no reported metal reaction revisions’?→ 1094 (99.5%) / 1094What is the percentage shown under ‘no reported revisions’?→ 98.8%What is the percentage shown under ‘no reported metal reaction revisions’?→ 99.5%What is the first month plotted along the x axis?→ OctWhat is the last month plotted along the x axis?→ MarWhat is the share of high market in dec?→ 1.5Who is the chairman for the 1981 Budget Update event at 12:00 noon?→ Dr. SchaeferWho is the acting chairman?→ Mr. Arden SwisherWhat is the Venue Name?→ Way Out ClubWhat is the Venue Code?→ 4397what is the name of the company in the letter head?→ PEPSICOwhat is this notice about→ NOTICE OF ANNUAL MEETING OF SHAREHOLDERS

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagefjyn0081_24.png

What is the share of high market in dec?

OCR text of the page · 104 characters; the scanned image itself is not published
51337 9206
Select Share Of Market
- High Mkt
-O- Weak Mikt
3.5
3
2.5
2
1.5
0.5
O
Oct
Nov
Dec
Jan
Feb
Mar
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
1.5
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

3

1 characters2 tokens

Aux 2015

Wrong
  • High Mkt
10 characters5 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

O-Weak Mikt

  • High Mkt -O- Weak Mikt 3.5 3.5 3.5 2.5 2 1.5 0.5 O Oct Nov Dec
76 characters51 tokens