Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

9 of 100 prompts

#091 · needle at 21% depth#092 · needle at 62% depth#093 · needle at 84% depth#094 · needle at 78% depth#095 · needle at 19% depth#096 · needle at 2% depth#097 · needle at 66% depth#098 · needle at 50% depth#099 · needle at 63% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for deranged-brief is: 1541743.
One of the special magic numbers for elite-mat is: 2927186.
[… 212 lines omitted for display …]
One of the special magic numbers for curly-half-sister is: 6293636.
One of the special magic numbers for rambunctious-exception is: 2097082.
One of the special magic numbers for ashamed-babushka is: 8574294.
One of the special magic numbers for phobic-brass is: 2876940.
One of the special magic numbers for soft-petition is: 7636635.
One of the special magic numbers for trashy-bow is: 9317634.
One of the special magic numbers for nondescript-change is: 9223424.
One of the special magic numbers for calm-airplane is: 1661733.
One of the special magic numbers for gentle-synthesis is: 2112331.
One of the special magic numbers for vast-parallelogram is: 7751276.
One of the special magic numbers for lush-suspenders is: 1852300.
One of the special magic numbers for mere-blister is: 5470861.
One of the special magic numbers for scarce-acquisition is: 6675439.
One of the special magic numbers for scary-mule is: 9783925.
[… 98 lines omitted for display …]
One of the special magic numbers for irate-biopsy is: 9000422.
One of the special magic numbers for capable-father-in-law is: 8512711.
What is the special magic number for calm-airplane mentioned in the provided text?
Original prompt 7,674 tokens, 330 linesShown 20 lines around the needleNeedle at 66% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT8360055Aux 20151661733Ours1661733Ours, no steer.1661733PiT-FT8360055Qwen 27B1661733
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

1661733.

13 characters10 tokens

Aux 2015

Missed

4214006.

13 characters10 tokens

PiT-FT 2015

Missed

: 4575025. The special magic number for the most-unusual is: 8577588. The special magic number for the most-unusual is: 8577588. The special magic number for the most-unusual is: 8577588. The special magic number for the most-unusual is: 8577588. The special magic number for the most-unusual is: 8577588. The special magic number for the most-unusual is: 8577588. The special magic number for the most-unusual is: 857

425 characters128 tokens

ChronoGPT 2015

Missed

: 7281430.<|endoftext|>

23 characters7 tokens