The Spotlight
Watch a transformer "read" a sentence and see, word by word, exactly what it pays attention to. Click any word in The tired cat sat on the warm mat and its spotlight lights up every other word by how much attention it gets — the heat under each token is its weight, and every word's row of weights adds up to 100%. It's the clearest way to feel what self-attention actually is: not one link, but a whole weighted lookup over the sentence, with the verb "sat" throwing half its attention onto its subject "cat." Runs entirely in your browser (0 uploads, works offline).
How to use it
- Click any word in the sentence to shine its spotlight — the focused word's attention row is drawn as heat across the other tokens.
- Read the caption: it names which word the spotlight lands on hardest and by how much (for the verb sat, that's cat).
- Scan the attention row panel to see the full sorted distribution and confirm it sums to ~1.
- Use the quick highlights — Verb → subject, Preposition → object, Subject → verb — to jump straight to the instructive rows.
- Open the X-ray to see why a row is a probability distribution, not a single arrow.
What this clears up (the fundamentals)
- Attention is a weighted lookup, not a pointer — each word spreads its attention across all the words, then reads a blend weighted by those numbers. The spotlight shows the whole spread, not just the winner.
- Every row is a distribution — the weights for one word sum to 1 (a softmax). That's why "how much does it attend to X" is a percentage you can compare, and why the total never exceeds 100%.
- Meaning is built from resolutions — the verb sat attending to cat is subject resolution made visible; on, the, and warm all pointing at mat is the object being bound. These links are what "understanding context" is made of.
- One head is one view — this is a single precomputed attention pattern; real models stack many heads and layers, each a different weighted lookup, and combine them.
Where it's used
A concept demo inside AIE-101 (paired with the Prompt Lab playground) and the model-internals track — the fastest way to make "the model attends to context" a thing you can see rather than a claim you take on faith. It's a miatz build-lab concept playable: play the spotlight here, then learn to compute a real attention matrix and render the graph yourself in the build lab.
FAQ
What is self-attention, in one line?
It's each word computing a set of weights over every word in the sentence (including itself), then reading a weighted blend of them — so context flows between tokens. The weights are what this tool visualizes.
Why does every word's attention add up to 100%?
Because attention weights come from a softmax, which turns raw scores into a probability distribution. Each word's row therefore sums to 1, so you can read every weight as a share of that word's attention.
Is this a real language model running?
No. The attention weights here are precomputed and hand-tuned to be linguistically plausible and easy to read. It teaches the shape and behaviour of self-attention without a live model, so it stays instant and fully offline.
What's the "resolution moment" the X-ray talks about?
It's the exact link where a word's spotlight lands on the word that resolves it — here, the verb sat putting most of its attention on its subject cat. Real attention research reads coreference and grammar the same way: by finding which token attends to which.
Is anything uploaded?
No. The whole thing runs on precomputed data in your browser — nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.
Limits
This uses a precomputed, illustrative attention matrix, not a live transformer — the numbers are hand-set to be plausible and legible, so they show how attention behaves rather than what any specific model produces on this exact sentence. Real models have many heads and layers whose patterns shift with training, wording, and context; treat this as a faithful mental model of the mechanism, not a readout of a particular network.
Related
Part of the miatz playgrounds. Explore the rest from the playground catalog.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.