Words you will meet
Every word the twelve Play lessons use, in plain English, with a box to find any of them.
35 words
- Model
- The rule a computer uses to make a guess, learned from examples. Big AI models are huge sets of numbers adjusted in training.
Field notes
- Token
- A piece of text the model reads as one unit. Often a whole short word, sometimes part of a longer one, sometimes a single letter.
Field notes
- Merge
- Gluing two pieces of text that often sit side by side into one new piece. Learning tokens is doing this again and again.
Field notes
- Training
- Letting a model learn from examples. Often that means nudging its numbers each time it guesses wrong, until it guesses well.
Field notes
- Training text
- The pile of writing a model learned from. What was common there feels normal to the model; what was rare stays unfamiliar.
Field notes
- Chance
- How likely something is, from 0 (never) to 1 (always). Half a chance is 0.5.
Field notes
- Softmax
- A small sum that turns a list of scores into chances that add up to 1. The higher the score, the bigger its share.
Field notes
- Temperature
- A dial on how adventurous the model is. Low keeps it to the safest next word; high lets less likely words through.
Field notes
- Top-k
- A rule that only lets the k most likely next words be picked. With k set to 2, only the best two can ever come out.
Field notes
- Embedding
- A word or sentence turned into a list of numbers, so that things with similar meaning end up with similar numbers.
Field notes
- Vector
- A list of numbers you can picture as an arrow pointing somewhere. Two arrows pointing the same way are alike.
Field notes
- Cosine similarity
- A score for how much two arrows point the same way. 1 is the same direction, 0 is unrelated, minus 1 is opposite.
Field notes
- Dot product
- Multiply two lists of numbers place by place and add it all up. It grows when two arrows point the same way, and also when they are simply longer.
Field notes
- Attention
- How each word in a sentence decides which other words to look at most when working out what it means here.
Field notes
- Query and key
- Each word asks a question (its query) and offers a label (its key). Where a question fits a label well, that word gets looked at more.
Field notes
- Value
- What a word hands over once it is looked at. Each word ends up with a blend of the values of the words it looked at most.
Field notes
- Causal mask
- A rule that lets each word look only at itself and the words before it, never ahead. It is how a model writes left to right without peeking.
Field notes
- Context window
- How much text a model can read at once. Anything past the limit is not seen at all, so older parts of a long chat get dropped.
Field notes
- Overfitting
- When a model memorizes its examples, quirks and mistakes included, so it looks great on them and does badly on anything new.
Field notes
- Underfitting
- When a model is too simple to catch the real pattern, so it is wrong on old and new examples alike.
Field notes
- Noise
- Random wobble in data that is not part of the real pattern. A good model ignores it.
Field notes
- Held-back data
- Examples kept aside from training, so you can check the model on something new. Ones used to choose settings are a validation set; a test set waits untouched until the end.
Field notes
- Validation set
- Held-back examples used to choose a setting, such as how bendy a curve may be. Because they helped choose, a final check needs a fresh test set.
Field notes
- Degree
- How bendy a curve is allowed to be. Degree 1 is a straight line, and each step up lets it bend once more.
Field notes
- Loss
- One number for how wrong the model is right now. Training is the work of pushing that number down.
Field notes
- Gradient
- Which way is downhill on the loss, and how steep. It tells training which way to nudge each number.
Field notes
- Learning rate
- How big each nudge is during training. Too small and it crawls; too big and it overshoots and bounces around.
Field notes
- Threshold
- The cut-off score above which something gets flagged. Moving it decides how picky the system is.
Field notes
- Precision
- Of everything the system flagged, how much was really right. High precision means few false alarms.
Field notes
- Recall
- Of everything that should have been flagged, how much the system caught. High recall means little slips through.
Field notes
- F1 score
- One number that balances precision and recall. It is only high when both are high, so a filter cannot score well by being good at just one.
Field notes
- Retrieval
- Looking up the most relevant passages from your own documents and handing them to the model with the question. Often called RAG.
Field notes
- Prompt
- Everything the model is given to read before it answers: your question, any instructions, and any pasted text.
Field notes
- Eval
- A fixed set of test questions with known good answers, used to score a prompt or model the same way every time.
Field notes
- Prompt injection
- Text hidden in an email, page or file that tries to give the model orders, as if it came from you.
Field notes