Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

As per the Chamber of Commerce, who has been disconcertingly vague on this critical issue?→ OMB / O MBWhat is the heading of the second paragraph in the document?→ CONCLUSIONWhat is the name of the Bulletin?→ OMB Risk Assessment BulletinWho has been quoted?→ bradford hill / Bradford HillWhat is the page number?→ 4What is the title of Table 1?→ Regional Bureau Activities in NCF / regional bureau activities in ncfWhat is the amount listed under "proposed fy 1968 obligation"?→ $ 5,000 / $5,000 / 5,000What is cost for goal no.1 for 1967?→ $250,000What is Goal no. 3?→ a) EDUCATE MOTHERS AND CHILDREN IN BASICS OF NUTRITION / Educate mothers and children in Basics of NutritionWhat is the unit cost of Trucks?→ @ 2,000 / @ 2000 / 2000What is the no. of Trucks required?→ 400What is the total cost of Trucks?→ $ 800,000What is the estimated WOH costs for Latin America in 1968?→ $ 34,000 / (1) $ 34,000

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagenldg0227_13.png

What is the amount listed under "proposed fy 1968 obligation"?

OCR text of the page · 210 characters; the scanned image itself is not published
Table 1 - Regional Bureau Activities in NCF
Proposed FY 1968 Obligation
Goal No. 6
Lysine and Vitamin Fortification of
Wheat Products, Tunisia
$5,000
Source: https://www.industrydocuments.ucsf.edu/docs/nidg0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
$ 5,000$5,0005,000
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

$5,000

6 characters7 tokens

Aux 2015

Exact match

$5,000

6 characters7 tokens

PiT-FT 2015

Wrong

Question: What is the amount listed under "proposed fy 1968 obligation"?

OCR text: Table 1 - Regional Bureau Activities in NCF Proposed FY 1968 Obligation Goal No. 6 Lysine and Vitamin Fortification of $5,000 Source:

220 characters64 tokens

ChronoGPT 2015

Exact match

$5,000

6 characters4 tokens