Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#074 · needle at 29% depth#075 · needle at 77% depth#076 · needle at 76% depth#077 · needle at 80% depth#078 · needle at 25% depth#079 · needle at 55% depth#080 · needle at 85% depth#081 · needle at 97% depth#082 · needle at 42% depth#083 · needle at 5% depth#084 · needle at 52% depth#085 · needle at 75% depth#086 · needle at 2% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for lying-porter is: 8034721.
One of the special magic numbers for dark-functionality is: 5963530.
[… 254 lines omitted for display …]
One of the special magic numbers for fragile-jeweller is: 8664085.
One of the special magic numbers for imaginary-everyone is: 5055889.
One of the special magic numbers for pathetic-officer is: 5285297.
One of the special magic numbers for long-convenience is: 2019349.
One of the special magic numbers for questionable-pacemaker is: 9665536.
One of the special magic numbers for excellent-adviser is: 1066832.
One of the special magic numbers for woebegone-business is: 7722086.
One of the special magic numbers for hissing-caution is: 3671238.
One of the special magic numbers for sharp-meringue is: 3327790.
One of the special magic numbers for short-consequence is: 5368033.
One of the special magic numbers for rapid-packet is: 3063877.
One of the special magic numbers for nutty-supply is: 1865551.
One of the special magic numbers for guiltless-common is: 8042146.
One of the special magic numbers for political-restaurant is: 3499213.
[… 31 lines omitted for display …]
One of the special magic numbers for acrid-paperwork is: 9639008.
One of the special magic numbers for fanatical-curse is: 2809602.
What is the special magic number for hissing-caution mentioned in the provided text?
Original prompt 7,118 tokens, 305 linesShown 20 lines around the needleNeedle at 85% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT5021836Aux 20153671238Ours3671238Ours, no steer.3671238PiT-FT5021836Qwen 27B3671238
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

3671238.

13 characters10 tokens

Aux 2015

Missed

2809602.

13 characters10 tokens

PiT-FT 2015

Missed

: 9871895 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|ass

253 characters128 tokens

ChronoGPT 2015

Missed

: 7781021.<|endoftext|>

23 characters7 tokens