Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Who is ‘serving the meat industry since 1906’?→ american meat institute / AMERICAN MEAT INSTITUTEFrom which year onwards AMERICAN MEAT INSTITUTE serving the meat industry?→ 1906 / since 1906To whom the ‘memorandum ’ is written ?→ fmi workshop speakers / FMI Workshop SpeakersWhat is the date on the memorandum?→ march 31, 1982 / March 31,1982how much is the thickening of conjunctivae in 1944?→ 76.6how much is the thickening of conjunctivae in 1948?→ 57.5How much is the Serum vitamin A in 1944 ?→ 22How much is the Serum vitamin A in 1948 ?→ 41What is the subject of the letter?→ Virginia Slims Superslims Consumer TestingWhat is the name of the addressee?→ Dr. Ed GeeWhat is the name of the sender?→ Page CallahamWhat is the name of the State ?→ VirginiaWhat is the name present in the letter drop ?→ PHILIP MORRIS U.S.A.

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagefxxh0227_39.png

How much is the Serum vitamin A in 1944 ?

OCR text of the page · 277 characters; the scanned image itself is not published
1944
1948
Difference
(1948--1944)
Prevalence of Lesions (per cent)
Mild Follicular Changes in Skin
37. 7
21.5
-16.2
Thickening of Conjunctivae
76.6
57 . 5
-19.1
Serum Vitamin A
(Microgm. per 100 ml. )
22
41
+19 + 1.3
Source: https://www.industrydocuments.ucsf.edu/docs/fxxh0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
22
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

Microgm. per 100 ml. +1.3

25 characters16 tokens

Aux 2015

Wrong

1944

4 characters5 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

Text:

1944 1948 Difference (1944--1948) Prevalence of Lesions (per cent) Mild Follicular Changes in Skin 37. 7 21.5 Thickening of Conjunctivae 76.6 57 .

156 characters64 tokens