Warm-up probability · models

Softmax

How a model turns raw scores into odds for its next word.

Write softmax(scores): turn an array of numbers into probabilities that are all positive and add up to 1. Raise e to each score, then divide each by the total.

Subtract the largest score from every score first. The answer is the same, and it stops huge scores from overflowing.

Call show(probs) with your answer and the stage draws it as bars.

Example: softmax([0, 0]) returns [0.5, 0.5].

Tests you can see

softmax([0,0])[0.5,0.5]
softmax([1,2,3])[0.09003057317038046,0.24472847105479764,0.6652409557748219]

More tests are hidden. The server runs them all before a time counts.

The idea behind this, in a two-minute live page

Call show() while your code works and the run plays back here.

javascriptsoftmax()

function softmax(scores) {
  // e to each score, divided by the total
  return scores;
}

function softmax(scores) {
  // e to each score, divided by the total
  return scores;
}

Tab indents · Cmd/Ctrl+Enter runs