Proceedings of the Annual Meeting of the Cognitive Science Society

In the present paper, a principal hidden unit analysis with entropy minimization is proposed to obtain a simple or fundamental structure from original complex structures. The principal hidden unit analysis is com- posed of four steps. First, entropy, defined with respect to the hidden unit activity, is minimized. Second, several principal hidden units are selected, according to il-index, rep- resenting the strength of the response of hid- den units to input patterns. Third, the per- formance of the obtained principal network is examined with respect to the error or generalization. Finally, the internal representa- tion of the obtained principal network must appropriately be interpreted. Applied to a rule-plus-exception, a symmetry problem and an autoencoder, it was confirmed in all cases that by using entropy method, a small num- ber of principal hidden units were selected. With these principal hidden units, principal networks were constructed, producing targets almost perfectly. The internal representation could easily be interpreted especially for simple problems