UC San Diego

Most physical objects have a material associated with them: from the wooden surface of a table to a velvet upholstery of a couch. The materials play a crucial role in the appearance of objects. Materials define the interaction of a surface with the light. Without good material models, realism in rendering cannot be achieved. In this dissertation, we will discuss how by using neural networks we can compactly and efficiently represent a wide variety of materials. Not only can neural networks provide excellent compression ratios, but they can be easily integrated into existing rendering engines.Our first contribution explores how we can use deep learning to render specular appearances. Many materials are made of highly reflective surfaces: for example, brushed metal or silky fabrics. Traditionally, many methods would model materials explicitly. However, those approaches require high storage requirements and a lot of computation during the simulation. Instead, we propose to use a generative adversarial network (GAN) to learn a high-frequency appearance from real and synthetic materials. We also propose to use texture synthesis on a feature texture. Our second contribution, NeuMIP, allows us to represent a wide variety of materials at different scales using neural networks. Traditional MIPMAP methods fail to downsample complex materials correctly. They naively average parameters over a patch, resulting in an incorrect appearance. NeuMIP avoids this problem by optimizing the neural textures at different levels of detail independently. A Neural Offset Module enables support for geometrically complex materials. NeuMIP assumes the material is applied to an infinitely large surface. As a result, material appearance on real objects would significantly diverge from the reference at the grazing angles. In our third work, we propose to take curvature into account and train on cylinders with different radii. Because of that, we can generate the natural silhouette of the object. Also, we support semitransparent materials. Neural materials are a novel way to represent complex real and synthetic materials and can be easily integrated into existing rendering pipelines. They offer high-quality and fast evaluation. We hope neural-based methods will become a standard way to represent materials in the future.