Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

10 of 100 prompts

#090 · needle at 60% depth#091 · needle at 23% depth#092 · needle at 46% depth#093 · needle at 43% depth#094 · needle at 81% depth#095 · needle at 73% depth#096 · needle at 95% depth#097 · needle at 78% depth#098 · needle at 43% depth#099 · needle at 1% depth

Nearby prompts. All 100 Multi-key · 16K prompts

PromptRULER niah_multikey_2 ·16,384 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for exotic-buddy is: 5646428.
One of the special magic numbers for penitent-confusion is: 2446177.
[… 629 lines omitted for display …]
One of the special magic numbers for jumbled-cobweb is: 6712512.
One of the special magic numbers for wrathful-iceberg is: 2724727.
One of the special magic numbers for steep-dictionary is: 7483293.
One of the special magic numbers for obeisant-diary is: 3078836.
One of the special magic numbers for Early-organising is: 2550893.
One of the special magic numbers for homely-handover is: 6644574.
One of the special magic numbers for secretive-teriyaki is: 7296553.
One of the special magic numbers for good-plumber is: 2375655.
One of the special magic numbers for truculent-dam is: 9456986.
One of the special magic numbers for piquant-fleece is: 8657520.
One of the special magic numbers for dead-daffodil is: 5726173.
One of the special magic numbers for taboo-sauerkraut is: 2001443.
One of the special magic numbers for resolute-compromise is: 1509212.
[… 17 lines omitted for display …]
One of the special magic numbers for oval-astronomy is: 9820263.
One of the special magic numbers for tired-establishment is: 6805948.
What is the special magic number for secretive-teriyaki mentioned in the provided text?
Original prompt 15,315 tokens, 665 linesShown 19 lines around the needleNeedle at 95% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT8136455Aux 20156351495Ours7296553Ours, no steer.7296553PiT-FT8136455Qwen 27B7296553
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

7296553.

13 characters10 tokens

Aux 2015

Missed

4321615.

13 characters10 tokens

PiT-FT 2015

Missed

: 8802069. The special magic number for the most part-of-the-world is: 8802069. The special magic number for the most part-of-the-world mentioned in the provided text is: 8802069. The special magic number for the most part-of-the-world mentioned in the provided text is: 8802069. The special magic number for the most part-of-the-world mentioned in the provided text is: 8802069. The special magic number for the most part-of-the-world mentioned in the provided

466 characters128 tokens

ChronoGPT 2015

Missed

: 1325776.<|endoftext|>

23 characters6 tokens