UC San Diego

Producing and capturing images with anticipated looks under various illuminations is the ultimate goal of computer graphics and computational photography. Two long-standing problems are blocking the pathway: how to generate pixels to form realistic appearances (image synthesis or rendering) and how to steer pixels to manipulate captured appearances (computational imaging and illumination). In this dissertation, both topics will be covered and discussed as part of the image formation task.

Physically-based rendering creates photo-realistic images by calculating pixel colors from light transport simulation, which has been used by the gaming and film-making industry for years to produce astounding visual effects on screens. However, synthesizing the entire family of illumination effects on the image is computationally expensive. It can readily cost days of time to yield a single frame in the existing production pipeline, especially when lights are challenging to compute by the standard ray tracing algorithm. To remedy the issue, we have proposed multiple methods to accelerate the sampling and reconstruction of different types of illumination, leading to more efficient Computer-Generated Imagery (CGI).

Computational illumination and digital imaging expand the function set of image synthesis in graphics by supporting more flexible appearance alterations during image construction. One of the unexplored functionalities is to adjust pixels structurally through unique patterns carried by coded illuminations. We have developed a dedicated lighting and camera system to collaboratively direct the captured image appearances, enabling unconventional lighting effects such as the selective restructuring of illuminated image segments in a controllable and automatic mode.

From the simulated light transport to real-world computational photography, we have advanced the visual computing technology by producing images with desirable looks and diverse illumination effects. Our conducted research also open up new challenges and opportunities for future studies on images.