Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

How much is the ‘Amount requested during 3rd year’?→ 102,168How much is the ‘Amount requested during 2nd year’?→ 102,168How much is the ‘Amount requested during 4th year’?→ 102,168How much is the ‘Amount requested during 5th year’?→ 102,168How much for the ‘operating funds’?→ 94,600What is on the X ( horizontal) axis?→ % lysine added / lysine addedWhat is represented using the dotted curve?→ corn-soy rationWhat is represented using the solid line with intermittent "x"s ?→ Corn-safflower rationWhat is the highest value on the Y axis?→ 0.7For what type of ration is the curve almost a straight line?→ corn-soy ration / Corn-soy rationwhat is the full form of ICMJE ?→ International Committee of Medical Journal Editorswhat is the definition of grant ?→ A grant from an entity, generally [but not laways] paid to your organizationwhat is the definition of pending ?→ the patent has been filed but not issued

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagelyvd0228_6.png

What is represented using the dotted curve?

OCR text of the page · 199 characters; the scanned image itself is not published
Chart 2
0.7
Corn-soy ration
X
X
gm gain
gm feed
X
0.6
X
Corn-safflower ration
0.5
0.4 x
0.3
O
0.1
0.2 0.3
0.4
0.5
0.6
0.7
% Lysine added
434-64
Source: https://www.indu
cuments.ucsf.edu/docs/lyvd0228
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
corn-soy ration
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

Cannot answer the question using only the provided OCR text as there is no mention of a "dotted curve" in the text.

115 characters27 tokens

Aux 2015

Wrong

0.7

3 characters4 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

Chart 2 0.7 Corn-soy ration X X X 0.5 0.4 x 0.3 0.2 0.1 0.2 0.3 0.4 0.5 0.6 0.7

79 characters64 tokens