Medium training · gradients

Roll downhill

Watch the error fall as a line nudges itself toward the data.

Gradient descent is how almost every model learns: measure the error, work out which way makes it smaller, take a small step that way, repeat.

Write fitLine(xs, ys, rate, steps): fit y = w * x + b. Start with w = 0 and b = 0. On each step, compute the predictions, then the gradients gw = (2 / n) times the sum of (pred - y) * x, and gb = (2 / n) times the sum of (pred - y). Update w = w - rate * gw and b = b - rate * gb. Return [w, b] after all the steps. With no steps, return [0, 0].

Before each update, compute the mean squared error and call show(points), where points is the [stepNumber, error] pairs so far, starting at step 0. The stage draws the error falling.

Example: fitLine([0, 1], [1, 3], 0.1, 1) returns [0.3, 0.4].

Tests you can see

fitLine([0,1], [1,3], 0.1, 1)[0.30000000000000004,0.4]
fitLine([0,1,2,3], [1,3,5,7], 0.1, 150)[2.0000024558525156,0.999994756652685]
fitLine([1,2,3], [4,4,4], 0.1, 100)[0.12936616081989522,3.7059203089823245]

More tests are hidden. The server runs them all before a time counts.

The idea behind this, in a two-minute live page

Call show() while your code works and the run plays back here.

javascriptfitLine()

function fitLine(xs, ys, rate, steps) {
  let w = 0;
  let b = 0;
  // each step: predictions, gradients, update
  return [w, b];
}

function fitLine(xs, ys, rate, steps) {
  let w = 0;
  let b = 0;
  // each step: predictions, gradients, update
  return [w, b];
}

Tab indents · Cmd/Ctrl+Enter runs