up

Problem Set 1 (15.4 - 22.4.07)

This problem set deals with Hebbian learning, as it was introduced at the end of the last semester (see script).

Exercise 1

  1. Implement Hebbian learning of a single linear unit on a set of input vectors (so the whole dataset can be represented as a two dimensional NumPy array). Implement both explicite normalisation and Oja's rule.
  2. Test your implenentation on a two dimensional Gaussian cloud of data points (e.g. 100 points). How fast does it converge?
  3. Plot the convergence of the weight vector (e.g. the scalar product with the final weight vector).

Solution

Python program (v1.2) / plot 1 / plot 2

This is the example solution that I wrote. Note the use of derived classes (though object orientiation does not make a big difference for such a simple example) and lambda expressions.

I tried to adhere to coding conventions (though there are some violations like the longer line lengths) and to document my code. This is the level of convention and documentation that I would also like to see in your code.

Exercise 2 (optional)

  1. Generate an image sequence with an underlying gaussian variance (e.g. different gratings, you can use PIL). Train your Hebbian unit from exercise 1 with the data. Then create a picture of the optimal stimulus.
  2. Generate small image patches from a larger natural image and do the same analysis as in a.
  3. Use multiple decorallated units (e.g. using asymetric inhibitory lateral connections) on the data used in b to learn several different principal components.