UC San Diego

The field of computer vision has grown enormously in the last decade, and significant progress has been made in many different areas. Several reasonable solutions to technically challenging problems like stereo, optical flow, tracking, detection, recognition, shape from shading, photometric stereo \etc have emerged. The principle challenge today is to make these algorithms, designed and tested in laboratory conditions, to work in real life situations. Several assumptions that make vision problems tractable, break down in the real world. The most common among these assumptions is that the reflectance of all objects in an image is Lambertian. This dissertation focuses on algorithms that can be applied to a large class of non-Lambertian materials whose reflectance can be described as a linear combination of diffuse (matte) and a specular (gloss) term. We propose a transformation of the RGB color space that recovers two purely diffuse components of an RGB color image. We extend our approach to the case of multiple colored illuminants in a scene, and derive a class of photometric invariants. These photometric invariants can then be used as inputs to several computer vision algorithms to improve their performance. This is empirically demonstrated for photometric stereo, shape from shading, stereo, optical flow, passive photometric stereo, and color based segmentation. While the problem of recovering two purely diffuse components of a color image is well-posed, the problem of completely separating the specular and diffuse components of a color image is an ill-posed one. We propose a unified approach to solve this problem for images and videos of textureless as well as textured scenes. A partial differential equation is derived that uses neighborhood color information of a pixel to iteratively erode the specular component of color. Finally, we introduce an application called Dichromatic Editing -- the process of independently editing and recombining the two components of color to achieve a variety of visual effects. While the problem of recovering two purely diffuse components of a color image is well-posed, the problem of completely separating the specular and diffuse components of a color image is an ill-posed one. We propose a unified approach to solve this problem for images and videos of textureless as well as textured scenes. A partial differential equation is derived that uses neighborhood color information of a pixel to iteratively erode the specular component of color. Finally, we introduce an application called Dichromatic Editing -- the process of independently editing and recombining the two components of color to achieve a variety of visual effects