Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#066 · needle at 42% depth#067 · needle at 0% depth#068 · needle at 62% depth#069 · needle at 84% depth#070 · needle at 67% depth#071 · needle at 45% depth#072 · needle at 49% depth#073 · needle at 79% depth#074 · needle at 15% depth#075 · needle at 35% depth#076 · needle at 12% depth#077 · needle at 63% depth#078 · needle at 82% depth

Nearby prompts. All 100 Multi-key · 128K prompts

PromptRULER niah_multikey_2 ·131,072 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for ultra-macadamia is: 4115290.
One of the special magic numbers for learned-llama is: 4716909.
[… 2,784 lines omitted for display …]
One of the special magic numbers for credible-rugby is: 3911502.
One of the special magic numbers for fluttering-monger is: 5437558.
One of the special magic numbers for nutty-signup is: 9439192.
One of the special magic numbers for jagged-printer is: 3247661.
One of the special magic numbers for oval-craw is: 3628468.
One of the special magic numbers for tan-wastebasket is: 7464246.
One of the special magic numbers for gamy-matchmaker is: 6292594.
One of the special magic numbers for noiseless-promise is: 5885382.
One of the special magic numbers for dysfunctional-doggie is: 8953011.
One of the special magic numbers for ugly-shorts is: 3839087.
One of the special magic numbers for literate-gavel is: 3914287.
One of the special magic numbers for selective-comfortable is: 9634545.
One of the special magic numbers for mundane-invoice is: 7371436.
One of the special magic numbers for fertile-prow is: 8582426.
[… 2,914 lines omitted for display …]
One of the special magic numbers for trashy-nut is: 9371869.
One of the special magic numbers for early-king is: 4660068.
What is the special magic number for noiseless-promise mentioned in the provided text?
Original prompt 130,419 tokens, 5,718 linesShown 20 lines around the needleNeedle at 49% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT4502556Aux 20151042499Ours5885382Ours, no steer.5885382PiT-FT4502556Qwen 27B5885382
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

5885382.

13 characters10 tokens

Aux 2015

Missed

8845994.

13 characters10 tokens

PiT-FT 2015

Missed
 [] []. What number is the special magic number for the following text?
[ime]
The special magic numbers for the following text are: 8805769.
The special magic numbers for the following text are: 8805769.
The special magic numbers for the following text are: 8805769.
The special magic numbers for the following text are: 8805769.
The special magic numbers for the following text are: 8805769.
The special magic numbers for the following text are: 8805769.
The special magic numbers for the following text are: 8
511 characters128 tokens

ChronoGPT 2015

Missed

:

Wh:

The special Epicurean Epicurean Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

Stupid-cans is the special Epicurean

424 characters128 tokens