Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

13 of 100 prompts

#001 · needle at 85% depth#002 · needle at 44% depth#003 · needle at 63% depth#004 · needle at 73% depth#005 · needle at 55% depth#006 · needle at 84% depth#007 · needle at 11% depth#008 · needle at 30% depth#009 · needle at 3% depth#010 · needle at 15% depth#011 · needle at 91% depth#012 · needle at 61% depth#013 · needle at 6% depth

Nearby prompts. All 100 Multi-key · 4K prompts

PromptRULER niah_multikey_2 ·4,096 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for bumpy-house is: 9114518.
One of the special magic numbers for worthless-nursery is: 5818203.
[… 6 lines omitted for display …]
One of the special magic numbers for strong-satin is: 4904421.
One of the special magic numbers for whimsical-ice is: 5873580.
One of the special magic numbers for encouraging-gynaecology is: 1026523.
One of the special magic numbers for cooperative-corral is: 1402431.
One of the special magic numbers for overt-aftermath is: 2978465.
One of the special magic numbers for expensive-hypothermia is: 7956372.
One of the special magic numbers for motionless-campanile is: 1763232.
One of the special magic numbers for sleepy-ram is: 9874251.
One of the special magic numbers for inexpensive-subset is: 6866154.
One of the special magic numbers for sharp-coach is: 4562439.
One of the special magic numbers for abortive-way is: 4375360.
One of the special magic numbers for victorious-jaguar is: 5724912.
One of the special magic numbers for garrulous-premium is: 2138100.
One of the special magic numbers for hospitable-post is: 6037929.
[… 120 lines omitted for display …]
One of the special magic numbers for various-therapist is: 7357071.
One of the special magic numbers for imperfect-warming is: 9292896.
What is the special magic number for sleepy-ram mentioned in the provided text?
Original prompt 3,462 tokens, 146 linesShown 20 lines around the needleNeedle at 11% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT6098837Aux 20151976175Ours9874251Ours, no steer.9874251PiT-FT6098837Qwen 27B9874251
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

9874251.

13 characters10 tokens

Aux 2015

Missed

8877363.

13 characters10 tokens

PiT-FT 2015

Missed

: 7581234. The special magic number for sordid-sprat mentioned in the provided text is: 7581234. The special magic number for sordid-sprat mentioned in the provided text is: 7581234. The special magic number for sordid-sprat mentioned in the provided text is: 7581234. The special magic number for sordid-sprat mentioned in the provided text is: 7581234. The special magic number for sordid-sprat mentioned in the provided text is: 7581234. The

450 characters128 tokens

ChronoGPT 2015

Missed

: 7278431.<|endoftext|>

23 characters7 tokens