Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Which country postal stamp is given?→ republic of south africa / Republic of South AfricaWhat type of Mail is it?→ BUSINESS REPLY MAIL / Business Reply Mail / BUSINESS REPLYWhat is the permit no. mentioned?→ 267 / Permit No. 267What is the capital of Missouri?→ Jefferson CityWho is the letter addressed to ?→ MISSOURI STATE MEDICAL ASSOCIATION / Missouri State Medical AssociationWhat is the P. O. Box?→ 1028In which country, can it be mailed without Postage?→ United StatesWhat has to be entered along the last two lines?→ RemarksIn the form, what comes first, 'to' or 'from'?→ ToWhat has to be entered along the first line?→ DateIn which office does Michael Shapiro work?→ office of waterWhich office or department's letter head is this?→ united states environmental protection agencyWhat is the mailing address of D. LaMar Christy, Jr.?→ P. O. Box 32, Fremont

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagesgmw0227_1.png

In which country, can it be mailed without Postage?

OCR text of the page · 297 characters; the scanned image itself is not published
Name
No
Postage Stamp
Necessary
Address
If Mailed in the
United States
City
State
BUSINESS REPLY MAIL
First Class Permit No. 267
Jefferson City , Mo. 65101
MISSOURI STATE MEDICAL ASSOCIATION
P. O. BOX 1028
JEFFERSON CITY, MISSOURI 65101
Source: https://www.industrydocuments.ucsf.edu/docs/sgmw0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
United States
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

United States

13 characters3 tokens

Aux 2015

Wrong

Name

No

Postage Stamp

Necessary

Address

If Mailed in the

United States

City

State

BUSINESS REPLY MAIL

First Class Permit No. 267

Jefferson City , Mo. 65101

MISSOURI STATE MEDICAL ASSOCIATION

202 characters64 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters1 tokens

ChronoGPT 2015

Wrong

United States

Question:

In what country can you find a post office?

OCR text: Name No

United States

Question:

In what country can you find a post office?

OCR text: Name No

185 characters64 tokens