When a neural network is trained on multiple analogous tasks,previous research has shown that it will often generate rep-resentations that reflect the analogy. This may explain thevalue of multi-task training, and also may underlie the powerof human analogical reasoning – awareness of analogies mayemerge naturally from gradient-based learning in neural net-works. We explore this issue by generalizing linear analysistechniques to explore two sets of analogous tasks, show thatanalogical structure is commonly extracted, and address somepotential implications.
