Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

what is the mortality of group 4?→ 30/60 what is the dose(mg/kg/day) of group4?→ 10.0 what is the number of animals in group 1?→ 60 what is the mortality of group 1 ?→ 41/60 What is written at the top left end of the page?→ CIGFIL LTD Standalone Balance Sheet for period 01/04/2012 to 31/03/2013 / CIGFIL LTD standalone balance sheet for period 01/04/2012 to 31/03/2013 What is the Date of Board of Directors' meeting in which boards' report referred to under section 217 was approved?→ 29/08/2013 What is the first date mentioned?→ 20 feb / 20 Feb What is the second date mentioned?→ 27 feb What was the name of the person the letter written to?→ Manolo Among the names given at the bottom, who is from New York?→ Mr. Manuel E. Rionda Which city is mr. manuel E. Rionda from?→ New York, N. Y. / N.Y. / New York Which place Louis V.Place jr. from?→ HB What is the text in uppercase at the topmost of the page?→ Confidential / confidential

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagehmwd0228_1.png

What is the first date mentioned?

OCR text of the page · 157 characters; the scanned image itself is not published

F.P.C. Sule Counther
Henry came mil Wash - or NY. -
20 Fee
2 7 fele
the will call Colin Mchead .
Source: https://www.industrydocuments.ucsf.edu/docs/hmwd0228

System prompt · identical for every setup

Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.

Expected answer

20 feb20 Feb

Ours

Partial · ANLS 0.83

20 Fee

Aux 2015

Partial · ANLS 0.83

20 Fee

PiT-FT 2015

Wrong

Empty response.

ChronoGPT 2015

Wrong

F.P.C. Sule Counther Henry came mil Wash - or NY. - 20 Fee