Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#013 · needle at 6% depth#014 · needle at 2% depth#015 · needle at 65% depth#016 · needle at 28% depth#017 · needle at 73% depth#018 · needle at 59% depth#019 · needle at 32% depth#020 · needle at 71% depth#021 · needle at 51% depth#022 · needle at 77% depth#023 · needle at 37% depth#024 · needle at 68% depth#025 · needle at 34% depth

Nearby prompts. All 100 Multi-key · 4K prompts

PromptRULER niah_multikey_2 ·4,096 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for alluring-waterbed is: 7098050.
One of the special magic numbers for hapless-loophole is: 7726643.
[… 38 lines omitted for display …]
One of the special magic numbers for great-scarf is: 5229054.
One of the special magic numbers for telling-donation is: 3954562.
One of the special magic numbers for breezy-governance is: 3377967.
One of the special magic numbers for billowy-plagiarism is: 5595368.
One of the special magic numbers for mere-anagram is: 2370851.
One of the special magic numbers for high-pitched-filth is: 8573586.
One of the special magic numbers for mysterious-chauvinist is: 9804312.
One of the special magic numbers for sincere-injustice is: 9191626.
One of the special magic numbers for fallacious-someone is: 5352802.
One of the special magic numbers for educated-precedence is: 1901645.
One of the special magic numbers for capable-fellow is: 8189612.
One of the special magic numbers for jazzy-arm is: 5981239.
One of the special magic numbers for disillusioned-leverage is: 4332044.
One of the special magic numbers for jumpy-airforce is: 8031349.
[… 88 lines omitted for display …]
One of the special magic numbers for adamant-downturn is: 3190232.
One of the special magic numbers for omniscient-commonsense is: 7008279.
What is the special magic number for sincere-injustice mentioned in the provided text?
Original prompt 3,467 tokens, 146 linesShown 20 lines around the needleNeedle at 32% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT2592385Aux 20155295408Ours9191626Ours, no steer.9191626PiT-FT2592385Qwen 27B9191626
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

9191626.

13 characters10 tokens

Aux 2015

Missed

7152746.

13 characters10 tokens

PiT-FT 2015

Missed

: <|assistant|>

<|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|>

255 characters128 tokens

ChronoGPT 2015

Missed

: 8285931.<|endoftext|>

23 characters7 tokens