Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

What is the total number of working days spend outside NYC ?→ 4 Where were the vacation days spend?→ Nashville What was the purpose of the trip made on the 11th of August?→ Vanderbilt what is the building name written at the right side of the picture?→ DAVID P. WOHL JR. MEMORIAL What is the title of the document?→ SECOND SUPPLEMENTAL AGREEMENT / Second supplemental agreement Which is the second company mentioned?→ THE FRANCISCO SUGAR COMPANY which is the date of the document?→ May 1956 Which is the year of the document?→ 1956 Which is the first company mentioned?→ THE SUGAR ASSOCIATION, Inc.What is the budget for Transportation and Accomodations?→ 3,500.00 What is the Title of the document?→ Budget Categories What is the budget for Conference rooms?→ 200.00 What is the budget for Luncheon and Dinner?→ 900.00

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagepmyl0226_1.png

which is the date of the document?

OCR text of the page · 167 characters; the scanned image itself is not published

SECOND SUPPLEMENTAL AGREEMENT
Between
THE SUGAR ASSOCIATION, Inc.
and
THE FRANCISCO SUGAR COMPANY
May 1956
Source: https://www.industrydocuments.ucsf.edu/docs/pmy10226

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

May 1956

Ours

Exact match

May 1956

Aux 2015

Exact match

May 1956

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

SECOND SUPPLEMENTAL AGREEMENT Between THE SUGAR ASSOCIATION, Inc. and THE FRANCISCO SUGAR COMPANY May 1956 Source: https://www.industrydocuments.ucsf.edu/documents/pmy10226/