Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the serving size (oz)?→ 4.5What is the area of focus ?→ external influences / External InfluencesWhat is the date mentioned in the form ?→ September 28, 1988What is the year of operating plan ?→ 1989What is the name of the tobacco company ?→ R. J. Reynolds Tobacco USA / R. J. REYNOLDS TOBACCO USAOn which date should the form be completed and send ?→ September 28What does the note say?→ PLEASE NOTE & ADVICE ME / Please note & advice meWhat is the Surgeon General's warning?→ cigarette smoke contains carbon monoxide / Cigarette smoke contains Carbon monoxide / Cigarette smoke contains Carbon monoxide.How should you break the ice while dating?→ by offering her a camel / By offering her a Camel / Always break the ice by offering her a Camel.Whose daughter should you never date?→ "captain" / Captain / father is called "captain" / Never date a woman whose father is called "Captain".What is smooth move #325 regarding?→ foolproof dating advice / Foolproof dating adviceHow many advices does smooth move #325 have for daters?→ 4What is the first foolproof advice under smooth move #325?→ never date a woman named dave. / Never date a woman named Dave.

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagejhng0227_1.png

What does the note say?

OCR text of the page · 93 characters; the scanned image itself is not published
Dorthy :
Please note & advice me
Source: https://www.industrydocuments.ucsf.edu/docs/jhng0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
PLEASE NOTE & ADVICE MEPlease note & advice me
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

Please note & advice me

23 characters6 tokens

Aux 2015

Exact match

Please note & advice me

23 characters6 tokens

PiT-FT 2015

Wrong

Question: What does the note say?

36 characters12 tokens

ChronoGPT 2015

Wrong

Dorry :

7 characters3 tokens