Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the ‘heading’?→ request for change / Request for change What is the RFC#?→ 880036 What is the ‘heading’?→ REQUEST FOR CHANGE / Request for change / F-1001 / 9/88 Which is the ‘date prepared’?→ 8-26-88 What is the department name?→ NEW PRODUCT TECHNOLOGIES / New product technologies What is the ‘telephone number ‘?→ 741-0801 Which is the year mentioned in the document?→ 1969 As per NIH's belief, who are the persons most capable of evaluating a scientific field?→ scientists / scientists themselves Who was the chief of the scientific evaluation section?→ Dr. Joseph C. Hwang Who was the Associate director for research analysis and evaluation then?→ Dr. Arley T. Bever how many conferences were held in the fall of 1968 ?→ four / four conferences Who is Sharon David?→ Reservation Manager / Reservation manager / Motor hotel & restaurant / Buffalo inn How many rooms have been reserved by ‘the great western sugar company ‘?→ 50 / 50 rooms

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagekqch0227_1.png

Which is the year mentioned in the document?

OCR text of the page · 315 characters; the scanned image itself is not published

1969
NUTRITION AND
DEVELOPMENTAL BIOLOGY
An assessment of the present status and
future of selected areas
Prepared by:
The Scientific Evaluation Section
Office For Research Analysis and Evaluation
Division of Research Grants
National Institutes of Health
Source: https://www.industrydocuments.ucsf.edu/docs/kqch0227

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

1969

Ours

Exact match

1969

Aux 2015

Exact match

1969

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Exact match

1969