UC San Diego

Finding optimal feature sets for classification tasks is still a fundamental and challenging problem in the area of machine learning. The human visual system performs classification tasks effortlessly using its hierarchical features and efficient coding in its visual pathway. It is shown that early in the visual system the information is encoded using distributed coding schemes and later in the visual system the sparse coding is utilized. We propose a biologically motivated method to extract features that encode the information according to a specific activation profile. We show how our model much like the visual system, can learn distributed coding in lower layers and sparse coding in higher layers in an online manner. Online feature extraction is used in biometrics, machine vision, and pattern recognition. Methods that can dynamically extract features and perform online classification are especially important for real-world applications. We introduce online algorithms that are fast and efficient in extracting features for encoding and discriminating the input space. We also show a supervised version of this algorithm that performs feature selection and extraction in alternating steps to achieve a fast convergence and high accuracy