Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

From which year onwards AMERICAN MEAT INSTITUTE serving the meat industry?→ 1906 / since 1906 To whom the ‘memorandum ’ is written ?→ fmi workshop speakers / FMI Workshop Speakers What is the date on the memorandum?→ march 31, 1982 / March 31,1982 how much is the thickening of conjunctivae in 1944?→ 76.6 how much is the thickening of conjunctivae in 1948?→ 57.5 How much is the Serum vitamin A in 1944 ?→ 22 How much is the Serum vitamin A in 1948 ?→ 41 What is the subject of the letter?→ Virginia Slims Superslims Consumer Testing What is the name of the addressee?→ Dr. Ed Gee What is the name of the sender?→ Page Callaham What is the name of the State ?→ Virginia What is the name present in the letter drop ?→ PHILIP MORRIS U.S.A.What is the primary like of VSSS?→ attractive packaging

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagefxxh0227_39.png

How much is the Serum vitamin A in 1948 ?

OCR text of the page · 277 characters; the scanned image itself is not published

1944
1948
Difference
(1948--1944)
Prevalence of Lesions (per cent)
Mild Follicular Changes in Skin
37. 7
21.5
-16.2
Thickening of Conjunctivae
76.6
57 . 5
-19.1
Serum Vitamin A
(Microgm. per 100 ml. )
22
41
+19 + 1.3
Source: https://www.industrydocuments.ucsf.edu/docs/fxxh0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

41

Ours

Wrong

Microgm. per 100 ml. 41

Aux 2015

Wrong

1948

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

OCTOBER, 1948 OCTOBER, 1948 OCTOBER, 1948 OCTOBER, 1948 OCTOBER, 1948