The Lab · Learn

How AI actually works

Small pages that run on this device. Read the plain version, move the sliders, watch what changes, then say what you saw. Tap any word you do not know.

Lesson 9

Attention

When reading a sentence, each word looks back at the other words and decides which ones matter most for its meaning.

It is like: Like reading “she put the cup on the table because it was full”. You check back to work out that “it” means the cup.
Why it matters at work: It is why AI handles long, tangled instructions better than old tools, and why putting the key point clearly still helps.

The exact idea

In attention, each token takes a weighted average of every token's value, with weights from how well its query matches their keys.

Words on this page

Try this

Pick “fed” and look at which token gets the thickest arc.
Slide the score multiplier to 4. The weights pile onto one token. At 0 they are all equal.
Turn on “Only look back” and pick “Sam”. It can only look at itself, so its output is its own value.

Change it

In the Workshop, press Pick a part, then tap the value table and ask for the values to be editable, then move one and watch the output follow.

Why does the colored output dot always land somewhere between the value dots, never outside them?

Pick the one that fits what you saw.

The output is clamped to the edges of the chart.The output is always the value of the token with the highest score.Queries and keys are kept small, so the output cannot travel far.The weights are never negative and add up to 1, so the output is a blend of the values.

Field notes

Reading the notes.