In this paper, I describe a large variety of clustering methods within a single framework. This paper unifies work across different fields, from biology (numerical taxonomy) to machine learning (concept formation). An important objective for this paper is to show that one can benefit by a knowledge of research across different disciplines. After describing the task from a set of different viewpoints or paradigms, I begin by describing the similarity measures or evaluation functions that form the basis of any clustering technique. Next, I describe a number of different algorithms that use these measures, and I close with a brief discussion of ways to evaluate different approaches to clustering.

This thesis presents a careful experimental examination of the task of concept formation. This learning task involves acquiring concepts or classes that group observed instances. These classes can then be used for the recognition or classification of unseen instances and for predicting values of unknown attributes. To carry out this task, I present CLASSIT, a robust, general-purpose system for incremental, unsupervised concept learning. In order to test the system, the thesis reports on a variety of experimental results. There are two broad categories for the experiments in this thesis: experiments that test the ability of the system with different input data, and experiments that hold the input data constant and test various components of the system by changing the system itself. The latter include parametric studies and comparative studies with cluster analysis methods. The largest modification I suggest for the basic concept formation algorithm is the addition of an attention mechanism. This lets the system focus attention on only the most important features (or attributes) of an instance. With attention, instances can be more quickly recognized, since the less important attributes are simply ignored and treated as missing. Finally, although the accomplishments of the CLASSIT system are modest, I conclude by discussing the potential for using the system as part of a larger cognitive architecture.

Given a set of observations, humans acquire concepts that organize those observations and use them in classifying future experiences. This type of concept formation can occur in the absence of a tutor and it can take place despite irrelevant and incomplete information. A reasonable model of such human concept learning should be both incremental and capable of handling this type of complex experiences that people encounter in the real world. In this paper, we review three previous models of incremental concept formation and then present CLASSIT, a model that extends these earlier systems. All of the models integrate the process of recognition and learning, and all can be viewed as carrying out search through the space of possible concept hierarchies. In an attempt to show that CLASSIT is a robust concept formation system, we also present some empirical studies of its behavior under a variety of conditions.