Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#001 · needle at 78% depth#002 · needle at 43% depth#003 · needle at 1% depth#004 · needle at 80% depth#005 · needle at 96% depth#006 · needle at 38% depth#007 · needle at 3% depth#008 · needle at 38% depth#009 · needle at 99% depth#010 · needle at 16% depth#011 · needle at 15% depth#012 · needle at 72% depth#013 · needle at 74% depth

Nearby prompts. All 100 Multi-key · 128K prompts

PromptRULER niah_multikey_2 ·131,072 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for muddy-azimuth is: 1359151.
One of the special magic numbers for smelly-premise is: 8057539.
[… 142 lines omitted for display …]
One of the special magic numbers for ashamed-metronome is: 9149446.
One of the special magic numbers for fabulous-sculptural is: 9331737.
One of the special magic numbers for overrated-spite is: 8524734.
One of the special magic numbers for brash-mile is: 8048771.
One of the special magic numbers for attractive-yarn is: 7838641.
One of the special magic numbers for magical-voice is: 1339329.
One of the special magic numbers for happy-park is: 1645275.
One of the special magic numbers for pumped-chronometer is: 2486738.
One of the special magic numbers for wanting-cardigan is: 2365318.
One of the special magic numbers for loving-underestimate is: 6066167.
One of the special magic numbers for scintillating-evaporation is: 9671782.
One of the special magic numbers for available-parrot is: 6768549.
One of the special magic numbers for callous-hard-hat is: 5446005.
One of the special magic numbers for hollow-potato is: 9024103.
[… 5,556 lines omitted for display …]
One of the special magic numbers for torpid-motorboat is: 5293489.
One of the special magic numbers for lean-skull is: 9927457.
What is the special magic number for pumped-chronometer mentioned in the provided text?
Original prompt 130,494 tokens, 5,718 linesShown 20 lines around the needleNeedle at 3% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT4769376Aux 20157921101Ours2486738Ours, no steer.2486738PiT-FT4769376Qwen 27B2486738
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

2486738.

13 characters10 tokens

Aux 2015

Missed

4931266.

13 characters10 tokens

PiT-FT 2015

Missed
 [] []. [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] []
193 characters128 tokens

ChronoGPT 2015

Missed

:

Sat:

The special Epicurus number for the following: The special Epicurus number for the following: The special Dickens number for the following: The special Fecundity number for the following: The special Dickens number for the following: The special Fiction number for the following: The special Grazing number for the following: The special Dickens number for the following: The special Dickens number for the following: The special Hades number for the following: The special Hades number for the following: The special Fecundity number for the following: The special

579 characters128 tokens