Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#036 · needle at 26% depth#037 · needle at 10% depth#038 · needle at 17% depth#039 · needle at 25% depth#040 · needle at 29% depth#041 · needle at 58% depth#042 · needle at 89% depth#043 · needle at 18% depth#044 · needle at 28% depth#045 · needle at 87% depth#046 · needle at 82% depth#047 · needle at 57% depth#048 · needle at 50% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for exuberant-jug is: 6945631.
One of the special magic numbers for coherent-record is: 2669370.
[… 290 lines omitted for display …]
One of the special magic numbers for aspiring-grace is: 1818876.
One of the special magic numbers for nondescript-rise is: 1146114.
One of the special magic numbers for lyrical-boulevard is: 9926699.
One of the special magic numbers for rabid-book is: 3276949.
One of the special magic numbers for madly-oatmeal is: 5993761.
One of the special magic numbers for sassy-ruckus is: 8287859.
One of the special magic numbers for crowded-refuge is: 8675147.
One of the special magic numbers for changeable-bead is: 7936884.
One of the special magic numbers for lazy-health is: 7123790.
One of the special magic numbers for exuberant-celery is: 9335576.
One of the special magic numbers for overconfident-curry is: 4823023.
One of the special magic numbers for shivering-rail is: 6899477.
One of the special magic numbers for tired-upper is: 5869645.
One of the special magic numbers for toothsome-warming is: 7372642.
[… 20 lines omitted for display …]
One of the special magic numbers for cute-temptress is: 7236644.
One of the special magic numbers for aware-pair is: 4877872.
What is the special magic number for changeable-bead mentioned in the provided text?
Original prompt 7,660 tokens, 330 linesShown 20 lines around the needleNeedle at 89% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT7151685Aux 20157936884Ours7936884Ours, no steer.7936884PiT-FT7151685Qwen 27B7936884
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

7936884.

13 characters10 tokens

Aux 2015

Missed

4877872.

13 characters10 tokens

PiT-FT 2015

Missed

: 5537392 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant

259 characters128 tokens

ChronoGPT 2015

Missed

: 7748053.<|endoftext|>

23 characters7 tokens