Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#006 · needle at 84% depth#007 · needle at 11% depth#008 · needle at 30% depth#009 · needle at 3% depth#010 · needle at 15% depth#011 · needle at 91% depth#012 · needle at 61% depth#013 · needle at 6% depth#014 · needle at 2% depth#015 · needle at 65% depth#016 · needle at 28% depth#017 · needle at 73% depth#018 · needle at 59% depth

Nearby prompts. All 100 Multi-key · 4K prompts

PromptRULER niah_multikey_2 ·4,096 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for ablaze-recovery is: 7593035.
One of the special magic numbers for gentle-barstool is: 6048065.
[… 85 lines omitted for display …]
One of the special magic numbers for hysterical-key is: 9899034.
One of the special magic numbers for noisy-key is: 3378937.
One of the special magic numbers for gleaming-boogeyman is: 2960284.
One of the special magic numbers for hurried-jodhpurs is: 1740918.
One of the special magic numbers for cultured-citizen is: 9375364.
One of the special magic numbers for guttural-miss is: 7049733.
One of the special magic numbers for new-washer is: 5746702.
One of the special magic numbers for oafish-cook is: 7517462.
One of the special magic numbers for glossy-moccasins is: 5425123.
One of the special magic numbers for rare-hubris is: 6251441.
One of the special magic numbers for nostalgic-activation is: 3767969.
One of the special magic numbers for husky-earnings is: 9242223.
One of the special magic numbers for demonic-bottom-line is: 8381800.
One of the special magic numbers for aspiring-spade is: 4169550.
[… 46 lines omitted for display …]
One of the special magic numbers for hulking-millet is: 3202624.
One of the special magic numbers for lamentable-bourgeoisie is: 2612483.
What is the special magic number for oafish-cook mentioned in the provided text?
Original prompt 3,577 tokens, 151 linesShown 20 lines around the needleNeedle at 61% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT3495209Aux 20153047722Ours7517462Ours, no steer.7517462PiT-FT3495209Qwen 27B7517462
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

7517462.

13 characters10 tokens

Aux 2015

Missed

4466272.

13 characters10 tokens

PiT-FT 2015

Missed

: 2898577 <|assistant|> The special magic number for elfin-forte mentioned in the provided text is: 2898577 <|assistant|> The special magic number for elfin-forte mentioned in the provided text is: 2898577 <|assistant|> The special magic number for elfin-forte mentioned in the provided text is: 2898577 <|assistant|> The special magic number for elfin-forte mentioned in the provided text is: 2898577 <|assistant|> The special magic number

445 characters128 tokens

ChronoGPT 2015

Missed

: 5594635.<|endoftext|>

23 characters7 tokens