Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#039 · needle at 75% depth#040 · needle at 6% depth#041 · needle at 28% depth#042 · needle at 69% depth#043 · needle at 49% depth#044 · needle at 50% depth#045 · needle at 36% depth#046 · needle at 27% depth#047 · needle at 2% depth#048 · needle at 27% depth#049 · needle at 71% depth#050 · needle at 92% depth#051 · needle at 87% depth

Nearby prompts. All 100 Multi-key · 4K prompts

PromptRULER niah_multikey_2 ·4,096 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for green-voting is: 1283652.
One of the special magic numbers for skillful-leading is: 2532162.
[… 43 lines omitted for display …]
One of the special magic numbers for easy-recruit is: 4912025.
One of the special magic numbers for crooked-cruelty is: 3428195.
One of the special magic numbers for animated-lava is: 3659852.
One of the special magic numbers for crabby-recession is: 2161873.
One of the special magic numbers for kind-flick is: 1394582.
One of the special magic numbers for jumbled-silk is: 9158591.
One of the special magic numbers for loud-quilt is: 5568225.
One of the special magic numbers for incandescent-brushing is: 8185097.
One of the special magic numbers for half-screw-up is: 8652001.
One of the special magic numbers for halting-good-bye is: 9052022.
One of the special magic numbers for abject-livestock is: 1837882.
One of the special magic numbers for aback-inventor is: 3353515.
One of the special magic numbers for naive-mining is: 8789111.
One of the special magic numbers for mammoth-fridge is: 6136686.
[… 83 lines omitted for display …]
One of the special magic numbers for known-attendance is: 2789056.
One of the special magic numbers for damp-pinto is: 2926003.
What is the special magic number for incandescent-brushing mentioned in the provided text?
Original prompt 3,457 tokens, 146 linesShown 20 lines around the needleNeedle at 36% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT1373276Aux 20158185097Ours8185097Ours, no steer.8185097PiT-FT1373276Qwen 27B8185097
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

8185097.

13 characters10 tokens

Aux 2015

Found the needle

8185097.

13 characters10 tokens

PiT-FT 2015

Missed

: 5668083 <|assistant|> The special magic number for orange-waterskiing mentioned in the provided text is: 5668083 <|assistant|> The special magic number for orange-waterskiing mentioned in the provided text is: 5668083 <|assistant|> The special magic number for orange-waterskiing mentioned in the provided text is: 5668083 <|assistant|> The special magic number for orange-waterskiing mentioned in the provided text is: 5668083 <|assistant|>

447 characters128 tokens

ChronoGPT 2015

Missed

: 8491037.<|endoftext|>

23 characters7 tokens