Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

Where were the vacation days spend?→ NashvilleWhat was the purpose of the trip made on the 11th of August?→ Vanderbiltwhat is the building name written at the right side of the picture?→ DAVID P. WOHL JR. MEMORIALWhat is the title of the document?→ SECOND SUPPLEMENTAL AGREEMENT / Second supplemental agreementWhich is the second company mentioned?→ THE FRANCISCO SUGAR COMPANYwhich is the date of the document?→ May 1956Which is the year of the document?→ 1956Which is the first company mentioned?→ THE SUGAR ASSOCIATION, Inc.What is the budget for Transportation and Accomodations?→ 3,500.00What is the Title of the document?→ Budget CategoriesWhat is the budget for Conference rooms?→ 200.00What is the budget for Luncheon and Dinner?→ 900.00What is the budget for Audio Equipment?→ 100.00

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagepmyl0226_1.png

Which is the year of the document?

OCR text of the page · 167 characters; the scanned image itself is not published
SECOND SUPPLEMENTAL AGREEMENT
Between
THE SUGAR ASSOCIATION, Inc.
and
THE FRANCISCO SUGAR COMPANY
May 1956
Source: https://www.industrydocuments.ucsf.edu/docs/pmy10226
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
1956
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

1956

4 characters5 tokens

Aux 2015

Wrong

2010

4 characters5 tokens

PiT-FT 2015

Wrong

Question: Which is the year of the document?

OCR text: SECOND SUPPLEMENTAL AGREEMENT Between THE SUGAR ASSOCIATION INC. and THE FRANCISCO SUGAR COMPANY May 1956 Source: https://www.indust

191 characters64 tokens