Problem Set 3 (18.5 - 1.6)

In this exercise we will implement Hopfield networks, as described in the lecture.

  1. Write a Hopfield network class with no self-connections. The network can be trained with a list of patterns. The update rule should be in a separate method, so that it can be overridden by subclasses (alternatively you can use the strategy pattern). Implement synchronous update and asynchronous ordered update.
  2. Write unittests for the two update rules, which test a small network (e.g. 3 units). The tests should verify that the connection weights and state updates are correct (by comparing them to stored values that were calculated by hand).
  3. Train a reasonably large network with some random patterns. Calculate the fixed points (and limit cycles), see if they match the learned patterns and calculate the attractor area for the fixed points. Either provide the results as simple numbers (e.g. attractor area for pattern blah contains 5 states...) or visualize them in some way (e.g by representing states with colored dots). Do this for both the synchronous and asynchronous update rule.