Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#004 · needle at 2% depth#005 · needle at 53% depth#006 · needle at 84% depth#007 · needle at 2% depth#008 · needle at 76% depth#009 · needle at 86% depth#010 · needle at 60% depth#011 · needle at 86% depth#012 · needle at 24% depth#013 · needle at 58% depth#014 · needle at 60% depth#015 · needle at 6% depth#016 · needle at 56% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for equable-plover is: 2566382.
One of the special magic numbers for cagey-worth is: 7788683.
[… 192 lines omitted for display …]
One of the special magic numbers for broad-year is: 9354503.
One of the special magic numbers for ethereal-flip-flops is: 3132636.
One of the special magic numbers for raspy-workhorse is: 1598865.
One of the special magic numbers for panoramic-atheist is: 9033263.
One of the special magic numbers for coherent-deficit is: 2869093.
One of the special magic numbers for flagrant-situation is: 1638058.
One of the special magic numbers for macabre-wrong is: 4095032.
One of the special magic numbers for marked-delight is: 2273983.
One of the special magic numbers for breezy-hardhat is: 2130790.
One of the special magic numbers for frantic-alliance is: 8314004.
One of the special magic numbers for super-blackboard is: 7222701.
One of the special magic numbers for noxious-temperature is: 3395093.
One of the special magic numbers for nutty-religion is: 1688736.
One of the special magic numbers for ugliest-timbale is: 9449329.
[… 118 lines omitted for display …]
One of the special magic numbers for dysfunctional-credit is: 3263974.
One of the special magic numbers for tacky-stretch is: 6034918.
What is the special magic number for marked-delight mentioned in the provided text?
Original prompt 7,671 tokens, 330 linesShown 20 lines around the needleNeedle at 60% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT3458603Aux 20152273983Ours2273983Ours, no steer.2273983PiT-FT3458603Qwen 27B2273983
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

2273983.

13 characters10 tokens

Aux 2015

Missed

6327664.

13 characters10 tokens

PiT-FT 2015

Missed

: 3065490 <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant|> <|assistant

259 characters128 tokens

ChronoGPT 2015

Missed

: 7141140.<|endoftext|>

23 characters7 tokens