UC San Diego

Human portraits are ubiquitous in our everyday life. However, after we take the portraits using the camera, the lighting condition of the captured image cannot be easily changed. Previous works have investigated how to change the lighting condition of a portrait, but they usually require special hardware to achieve relighting. In this thesis, we study the problem of portrait relighting, aiming at accuracy and practicality. We present several novel approaches that enable relighting portraits after the capture.With the goal of practical relighting that can be used in everyday life, we first develop single image portrait relighting. We leverage the prior of human portraits, and train a neural network that learns to perform relighting given only a single portrait. Our network can achieve convincing relighting results with very limited computation power, enabling real-time portrait relighting applications. Next, we focus on how to achieve precise relighting under point lights. We capture the portrait using discrete point lights under a light stage, and super-sample the location of the point lights to achieve high-frequency relighting. This enables finer controls on the shadows on human faces. Finally, we investigate changing both the viewpoint and the lighting condition of captured portraits. To this end, we propose neural light-transport field (NeLF). Given only 5 frontal portraits, this neural rendering approach can generate portraits from novel views under arbitrary natural lightings.