In this paper we review three kinds of proposed solutions to the catastrophic forgetting problem in neural networks. The solutions are based on reducing hidden unit overlap, rehearsal, and pseudorehearsal mechanisms. We compare the methods and identify some underlying similarities. We then briefly note some potential implications of the rehearsal/pseudorehearsal based methods, including their application to sequential learning tasks.
