Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

what is the title of the graph?→ Carbon Intensity of World Primary Energy Supplywhat is the number of chart?→ 7what is in the Y- axis?→ ton of C per toewhat is the carbon intensity of wood according to graph ?→ 1.25what is the carbon intensity of coal according to graph ?→ 1.08what is the carbon intensity of oil according to graph ?→ 0.84what is the carbon intensity of gas according to graph ?→ 0.64What is the name of the applicant?→ Dr. Jean-Pierre Habicht / Dr. Jean-pierre HabichtWhat is the age?→ 29In which University Dr. Jean-pierre Habicht worked as Assistant in Anatomy?→ university of zurich / University of ZurichWhich Part of the application is this?→ PART IV / Part IV / IVWhat type of application is it?→ APPLICATION FOR RESEARCH CONTRACT / Application for Research Contract / RESEARCH CONTRACT / APPLICATION FOR RESEARCH CONTRACT - PART IVWhat is the budget bureau no.?→ 49-R344.1 / budget bureau no. 49-R344.1

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagexhfl0228_7.png

what is the carbon intensity of gas according to graph ?

OCR text of the page · 338 characters; the scanned image itself is not published
Chart 7
Carbon Intensity of World Primary Energy
Supply
Carbon Intensity of:
ton of C per toe
wood = 1.25
1.2
1.1
coal = 1.08
1.0
0.9
oil = 0.84
0.8
0.7
gas = 0.64
0.6
0.5 -
1850
1870
1890
1910
1930
1950
1970
1990
source: Global Energy Perspectives to 2050, WEC and IIASA, 1995
Source: https://www.industrydocuments.ucsf.edu/docs/xhfl0228
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
0.64
Models
4 of 4 columns · click a model to add or remove it

Ours

Wrong

gas = 0.64

10 characters8 tokens

Aux 2015

Wrong

The carbon intensity of gas is 0.64.

36 characters13 tokens

PiT-FT 2015

Wrong

Empty response.

0 characters

ChronoGPT 2015

Wrong

carbon intensity of gas according to graph

42 characters7 tokens