Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#018 · needle at 34% depth#019 · needle at 88% depth#020 · needle at 50% depth#021 · needle at 65% depth#022 · needle at 29% depth#023 · needle at 47% depth#024 · needle at 37% depth#025 · needle at 64% depth#026 · needle at 93% depth#027 · needle at 50% depth#028 · needle at 72% depth#029 · needle at 68% depth#030 · needle at 6% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for fluffy-lizard is: 6867367.
One of the special magic numbers for shaggy-notice is: 8077565.
[… 114 lines omitted for display …]
One of the special magic numbers for colorful-gazebo is: 8884983.
One of the special magic numbers for detailed-dibble is: 3177321.
One of the special magic numbers for evasive-stall is: 7445760.
One of the special magic numbers for gabby-status is: 9071883.
One of the special magic numbers for powerful-mussel is: 3023574.
One of the special magic numbers for romantic-exhaustion is: 4321309.
One of the special magic numbers for plucky-bore is: 6684926.
One of the special magic numbers for defiant-coast is: 5353234.
One of the special magic numbers for fine-ginseng is: 1676430.
One of the special magic numbers for observant-grandmother is: 8033433.
One of the special magic numbers for shallow-daily is: 2779095.
One of the special magic numbers for abject-pail is: 4054479.
One of the special magic numbers for hapless-reliability is: 1250389.
One of the special magic numbers for uninterested-feeling is: 7146104.
[… 196 lines omitted for display …]
One of the special magic numbers for diligent-settlement is: 5674754.
One of the special magic numbers for obedient-preoccupation is: 8268414.
What is the special magic number for defiant-coast mentioned in the provided text?
Original prompt 7,661 tokens, 330 linesShown 20 lines around the needleNeedle at 37% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT8694639Aux 20155353234Ours5353234Ours, no steer.5353234PiT-FT8694639Qwen 27B5353234
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

5353234.

13 characters10 tokens

Aux 2015

Missed

8861196.

13 characters10 tokens

PiT-FT 2015

Missed

: 7551786 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|ass

253 characters128 tokens

ChronoGPT 2015

Missed

: 1808961.<|endoftext|>

23 characters7 tokens