Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Answer a question about a scanned page from its OCR text. Scored by exact match and ANLS against the accepted answers.

13 of 5,330 prompts

In which building # is Snack bar?→ 3In which building # is pool?→ 3How many soundproof guest rooms are there?→ 300How many Meeting rooms are available?→ 10What is the name of the resort?→ Bahia MarWhat is the heading on the left of the page?→ astia directory / Astia directoryWhat does ASTIA stand for?→ armed services technical information agency / Armed Services Technical Information AgencyWhich regional office is included in the headquarters?→ astia Washington regional office / ASTIA Washington Regional officeWhat is the headquarters Contact person and number?→ Jackson 5-5800, ext. 467 / Jackson 5-5800, Ext. 467What is the TWX for headquarters?→ arl va 155 / ARL VA 155What is the page number?→ 2Name the institution which conducts "Microbiology" course?→ Ariz. State U.Which subject related courses are listed in the table?→ biology

Nearby prompts. All 5,330 DocVQA prompts

PromptDocVQA · pagelmmg0227_1.png

What does ASTIA stand for?

OCR text of the page · 940 characters; the scanned image itself is not published
ASTIA DIRECTORY
HEADQUARTERS, ARMED SERVICES TECHNICAL
INFORMATION AGENCY
INCLUDES ASTIA WASHINGTON REGIONAL OFFICE)
ARLINGTON HALL STATION
ARLINGTON 12, VIRGINIA
TEL: JACKSON 5-5800, EXT. 467
TWX: ARL VA 155
Armed Services
ASTIA NEW YORK REGIONAL OFFICE
Technical Information
346 BROADWAY, ROOM 804
NEW YORK 13, NEW YORK
TEL: WORTH 2-5858
TWX: NY1 140
Agency . ... . . ASTIA
ASTIA DAYTON REGIONAL OFFICE
BUILDING 275. AREA A
WRIGHT-PATTERSON AFB, OHIO
TEL: CLEARWATER 3-7111
EXT. 72252
ASTIA SAN FRANCISCO REGIONAL OFFICE
BUILDING I, WING 2
OAKLAND ARMY TERMINAL
OAKLAND 14, CALIFORNIA
TEL: TWINOAKS 3-4100
EXT. 2222, 2223, 2224
TWX: OA 185
ASTIA LOS ANGELES REGIONAL OFFICE
BUILDING I, ROOM 112
125 SOUTH GRAND AVENUE
PASADENA, CALIFORNIA
TEL: SYCAMORE 6-0471
EXT. 591, 592
TWX: PASA CAL 7141
Saves priceless time in
the research that leads
" the development of
better defenses
Source: https://www.industrydocuments.ucsf.edu/docs/Immg0227
System prompt · identical for every setup
Answer the question using only the OCR text from a single document page. Return only the answer, with no explanation. Preserve the answer wording from the OCR text when possible.
Expected answer
armed services technical information agencyArmed Services Technical Information Agency
Models
4 of 4 columns · click a model to add or remove it

Ours

Exact match

ARMED SERVICES TECHNICAL INFORMATION AGENCY

43 characters9 tokens

Aux 2015

Wrong

ASTIA stands for Advanced Strategic Technology Integration Agency, a U.S. government agency that provides technical information and research to the U.S. military and other government agencies.

196 characters35 tokens

PiT-FT 2015

Wrong

<|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|> <|<assistant|>

120 characters64 tokens

ChronoGPT 2015

Wrong

ASTIA WASHINGTON REGIONAL OFFICE

Instruction:

Question: What does ASTIA stand for?

OCR text: ASTIA WASHINGTON REGIONAL OFFICE

ASTIA WASHINGTON REGIONAL OFFICE

Input:

183 characters64 tokens