UC San Diego

Learning accurate representations is important for any AI model, but is becoming increasingly vital as models are built and trained more to generalize zero-shot to new tasks and less for singular, clearly outlined tasks. Models are often tasked with being flexible to a wider variety of possible inputs while continuing to be immune to noisy or distorted examples. Learning robust representations, regardless of application, means considering the features models should pay attention to in order to complete the task and how features will vary across a normal range of inputs. Visual models should pay attention to visual features that people pay attention to, and be able to make informed predictions despite corruption of data or data outside the distribution of previously seen examples. Vision language models, with joint image and text embeddings, not only learn useful visual features and useful textual features, but learn to encode the relationship between the two. Here robust representations involve understanding relationships between features within and across data types. In other fields, like drug discovery, robust representations mean ensuring that a model is learning to include causal information in the representation instead of learning based on correlated but not causal features. Learned representations are the basis on which machine learning models make predictions. It is often up to the network designer to ensure models learn robust representations. I present work that explores defining what robust means for a task and how to create robust representations across machine learning applications.