Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

12 of 100 prompts

#088 · needle at 17% depth#089 · needle at 77% depth#090 · needle at 69% depth#091 · needle at 21% depth#092 · needle at 62% depth#093 · needle at 84% depth#094 · needle at 78% depth#095 · needle at 19% depth#096 · needle at 2% depth#097 · needle at 66% depth#098 · needle at 50% depth#099 · needle at 63% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for clammy-hourglass is: 3253362.
One of the special magic numbers for purring-priority is: 3874574.
[… 252 lines omitted for display …]
One of the special magic numbers for anxious-bafflement is: 2734858.
One of the special magic numbers for muddy-smuggling is: 6675656.
One of the special magic numbers for scrawny-molding is: 6145036.
One of the special magic numbers for late-babe is: 8020985.
One of the special magic numbers for endurable-eaglet is: 4059125.
One of the special magic numbers for uttermost-stake is: 5204430.
One of the special magic numbers for joyous-application is: 5834859.
One of the special magic numbers for hellish-bankbook is: 6822636.
One of the special magic numbers for abject-main is: 7030535.
One of the special magic numbers for filthy-storey is: 8249840.
One of the special magic numbers for Early-typeface is: 7706895.
One of the special magic numbers for imperfect-query is: 8080915.
One of the special magic numbers for questionable-nondisclosure is: 2681857.
One of the special magic numbers for ignorant-thunderhead is: 5375965.
[… 58 lines omitted for display …]
One of the special magic numbers for obeisant-ascent is: 6105197.
One of the special magic numbers for puny-marketing is: 2178650.
What is the special magic number for hellish-bankbook mentioned in the provided text?
Original prompt 7,666 tokens, 330 linesShown 20 lines around the needleNeedle at 78% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT1476920Aux 20156822636Ours6822636Ours, no steer.6822636PiT-FT1476920Qwen 27B6822636
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

6822636.

13 characters10 tokens

Aux 2015

Missed

9944436.

13 characters10 tokens

PiT-FT 2015

Missed

: 2327860 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant

259 characters128 tokens

ChronoGPT 2015

Missed

: 2849783.<|endoftext|>

23 characters6 tokens