UCLA

Today most of our machine learning methods and classification methods are gradient based methods. Especially in the area of image classification, methods such as deep learning try to tune large amounts of weights associated with training data over many iterations to be able to make predictions. These models, while highly accurate in prediction require many hours to train and are not very easily changed once trained. For instance, it is difficult to combine pre-trained neural networks into one model, difficult to add new observations to a trained model, and difficult for a neural network to forget specific observations it has trained on. Our goal is to explore non-gradient classification methods based on MinHash and locality-sensitive hashing. Instead of having a fixed network structure and fine tuning weights to respond to certain features, we create hash neurons that are only able to hash data and create new links with other neurons. By using this network of hash maps we create an online model that can learn, combine, forget, and change its own structure. We will also show that this technique, while memory intensive, is fast, scalable, performs well, and will work on any classification problem as long as the training data can be hashed into smaller unique parts.