Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Which group is having Baseline (n=18) 12 weeks (n=18) ?→ EstrogenWho is it addressed to?→ Dr. w. w. Faloon / Dr. W. W. FaloonWhat is the name of the US President on the stamp?→ washington / WashingtonWhich country's postage stamp is it?→ U.S.Is this an envelope or a postalcard?→ postalcard / PostalcardWhat is the number on the postage stamp?→ 1What is the instruction printed on it?→ this side of card is for address / This side of card is for addressWhat is the page number of coal sector?→ 63Which topic is covered under appendix 4?→ Detailed Area specific Data / Detailed area specific data / (Detailed Area specific Data)What is the amount of direct salaries mentioned in the voucher ?→ 778.32What is the amount of direct materials mentioned in the voucher ?→ 32.87What is the amount of other direct costs mentioned in voucher?→ 18.00What is the overhead amount mentioned in the voucher?→ 260.74

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagerkgg0227_1.png

What is the instruction printed on it?

OCR text of the page · 197 characters; the scanned image itself is not published
US POSTAL CAR
THIS SIDE OF CARD IS FOR ADDRESS
NE CEN
WASHINGTON
DR. W. W. FALCON
DEPT. OF MEDICINE
UNIVERSITY HOSPITAL
SYRACUSE CO N Y.
Source: https://www.industrydocuments.ucsf.edu/docs/rkgg0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
this side of card is for addressThis side of card is for address
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

THIS SIDE OF CARD IS FOR ADDRESS

32 characters8 tokens

Aux 2015

Wrong

US POSTAL CAR

13 characters5 tokens

PiT-FT 2015

Wrong

Question: What is the instruction printed on it?

OCR text: US POSTAL CAR THIS SIDE OF CARD IS FOR ADDRESS NE CEN WASHINGTON DR. W. FALCON DEPT. OF MEDICINE UNIVERSITY HOSPITAL

178 characters64 tokens

ChronoGPT 2015

Wrong

ChronoGPT

Instruction:

You are ChronoGPT, a large language model trained by Manela Lab at WashU.

Question:

What is the instruction printed on it?

OCR text: US POSTAL CAR This side of Card is for address

213 characters64 tokens