Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#046 · needle at 82% depth#047 · needle at 57% depth#048 · needle at 50% depth#049 · needle at 70% depth#050 · needle at 75% depth#051 · needle at 88% depth#052 · needle at 41% depth#053 · needle at 78% depth#054 · needle at 9% depth#055 · needle at 47% depth#056 · needle at 20% depth#057 · needle at 92% depth#058 · needle at 17% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for slow-licorice is: 6193154.
One of the special magic numbers for lush-modernity is: 6700176.
[… 126 lines omitted for display …]
One of the special magic numbers for charming-silkworm is: 1063431.
One of the special magic numbers for cagey-polenta is: 7364181.
One of the special magic numbers for uneven-broiler is: 6913267.
One of the special magic numbers for subdued-knee is: 2976741.
One of the special magic numbers for cool-tugboat is: 3928367.
One of the special magic numbers for slimy-veneer is: 6042011.
One of the special magic numbers for naive-chasuble is: 5409764.
One of the special magic numbers for acceptable-roof is: 1083319.
One of the special magic numbers for alluring-salmon is: 5148624.
One of the special magic numbers for big-ex-husband is: 2794806.
One of the special magic numbers for rustic-team is: 3612040.
One of the special magic numbers for screeching-frown is: 8753549.
One of the special magic numbers for standing-multimedia is: 2584305.
One of the special magic numbers for glossy-leveret is: 8292146.
[… 184 lines omitted for display …]
One of the special magic numbers for ancient-mama is: 8200339.
One of the special magic numbers for robust-multimedia is: 2305417.
What is the special magic number for acceptable-roof mentioned in the provided text?
Original prompt 7,649 tokens, 330 linesShown 20 lines around the needleNeedle at 41% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT1247396Aux 20151083319Ours1083319Ours, no steer.1083319PiT-FT1247396Qwen 27B1083319
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

1083319.

13 characters10 tokens

Aux 2015

Missed

8521636.

13 characters10 tokens

PiT-FT 2015

Missed

: 7546922 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|ass

253 characters128 tokens

ChronoGPT 2015

Missed

: 7493330.<|endoftext|>

23 characters7 tokens