Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#006 · needle at 84% depth#007 · needle at 2% depth#008 · needle at 76% depth#009 · needle at 86% depth#010 · needle at 60% depth#011 · needle at 86% depth#012 · needle at 24% depth#013 · needle at 58% depth#014 · needle at 60% depth#015 · needle at 6% depth#016 · needle at 56% depth#017 · needle at 48% depth#018 · needle at 34% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for hapless-gold is: 7107490.
One of the special magic numbers for burly-forelimb is: 7162223.
[… 70 lines omitted for display …]
One of the special magic numbers for gainful-teapot is: 2544323.
One of the special magic numbers for breakable-ounce is: 5116952.
One of the special magic numbers for worthless-stockings is: 2577152.
One of the special magic numbers for onerous-graphic is: 7372844.
One of the special magic numbers for internal-empowerment is: 5049661.
One of the special magic numbers for stereotyped-notepad is: 8873203.
One of the special magic numbers for flawless-freckle is: 1793470.
One of the special magic numbers for ahead-jailhouse is: 6416901.
One of the special magic numbers for hulking-cop-out is: 8569862.
One of the special magic numbers for fancy-stonework is: 7888843.
One of the special magic numbers for cautious-icy is: 1172703.
One of the special magic numbers for vacuous-smog is: 3153928.
One of the special magic numbers for spurious-acupuncture is: 1473717.
One of the special magic numbers for questionable-current is: 8456297.
[… 240 lines omitted for display …]
One of the special magic numbers for overconfident-brood is: 7189628.
One of the special magic numbers for worried-maestro is: 4001846.
What is the special magic number for flawless-freckle mentioned in the provided text?
Original prompt 7,622 tokens, 330 linesShown 20 lines around the needleNeedle at 24% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT4260851Aux 20151793470Ours1793470Ours, no steer.1793470PiT-FT4260851Qwen 27B1793470
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

1793470.

13 characters10 tokens

Aux 2015

Missed

7733149.

13 characters10 tokens

PiT-FT 2015

Missed

: 1413795 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant

259 characters128 tokens

ChronoGPT 2015

Missed

: 755981.<|endoftext|>

22 characters7 tokens