Frontier to PiT

Output Explorer

Every prompt in the paper, and what each model wrote back.

Many key–number pairs hidden in filler text; retrieve the one asked for.

8 of 100 prompts

#092 · needle at 62% depth#093 · needle at 84% depth#094 · needle at 78% depth#095 · needle at 19% depth#096 · needle at 2% depth#097 · needle at 66% depth#098 · needle at 50% depth#099 · needle at 63% depth

Nearby prompts. All 100 Multi-key · 8K prompts

PromptRULER niah_multikey_2 ·8,192 tokens
A special magic number is hidden within the following text. Make sure to memorize it. I will quiz you about the number afterwards.
One of the special magic numbers for green-fee is: 5534769.
One of the special magic numbers for accurate-commotion is: 9778987.
[… 157 lines omitted for display …]
One of the special magic numbers for dangerous-timpani is: 3738337.
One of the special magic numbers for gigantic-identification is: 1050394.
One of the special magic numbers for childlike-mailer is: 9167023.
One of the special magic numbers for fertile-anguish is: 3686989.
One of the special magic numbers for shaky-paramedic is: 2390577.
One of the special magic numbers for icy-servant is: 6404707.
One of the special magic numbers for waggish-default is: 6172077.
One of the special magic numbers for empty-signup is: 9519290.
One of the special magic numbers for exultant-meteor is: 2082362.
One of the special magic numbers for deafening-image is: 5360544.
One of the special magic numbers for teeny-knitting is: 7846972.
One of the special magic numbers for poor-deathwatch is: 5198001.
One of the special magic numbers for aboard-pomegranate is: 1071524.
One of the special magic numbers for sweet-ethnicity is: 5841996.
[… 153 lines omitted for display …]
One of the special magic numbers for cute-youth is: 4408721.
One of the special magic numbers for telling-station-wagon is: 7433011.
What is the special magic number for empty-signup mentioned in the provided text?
Original prompt 7,664 tokens, 330 linesShown 20 lines around the needleNeedle at 50% depthHaystacks are built per model’s tokenizer, so the needle differs by model
Expected answer
ChronoGPT9248814Aux 20159519290Ours9519290Ours, no steer.9519290PiT-FT9248814Qwen 27B9519290
Models
4 of 4 columns · click a model to add or remove it

Ours

Found the needle

9519290.

13 characters10 tokens

Aux 2015

Missed

7118269.

13 characters10 tokens

PiT-FT 2015

Missed

: 8447215. The special magic number for awkward-bauch is: 8447215. The special magic number for uncertain-tune is: 8447215. The special magic number for uninteresting-graft is: 8447215. The special magic number for indignant-bauch is: 8447215. The special magic number for unkind-tune is: 8447215. The special magic number for unkind-tune is: 8447215. The special magic number for unkind-tune is: 8

405 characters128 tokens

ChronoGPT 2015

Missed

: 7283898.<|endoftext|>

23 characters7 tokens