UCLA

Recovering three-dimensional (3D) scene geometry from images is an ill-posed problem due to the loss of the extra dimension in the process of projection. Hence, the solution hinges on the choice of regularization or prior assumptions about the scene. We study the effects of various regularization schemes on the 3D reconstruction problem under two problem settings, single image depth prediction and sparse depth completion. Obtaining the 3D scene from a single image is literally an impossible task as there are infinitely many 3D scenes compatible with the given image -- making both problem settings great candidates for evaluating the influence of a given regularizer. We begin by examining the relation between data fidelity residual and the degree of regularization to form a spatially and temporally varying adaptive weighting scheme for single image depth prediction. We additionally explore the use of gravity, as a supervisory signal, to induce a prior on the pose of objects populating a scene. To extend the use case to real world applications, we develop visioning systems to infer dense depth from an image with associated sparse depth measurements. We leverage the abundance of synthetic data to obtain a learned prior for guiding the learning process. Conscious of the limitations of current depth completion methods in processing sparse depth and their growth in parameters, we propose a two-stage approach that approximates the scene with a ``scaffolding'' and refines the approximation with a simple light-weight network. The result is a small and fast, but accurate visioning system that fits in an embedded system. To enable an agent to continuously learn, our systems are completely unsupervised and learn by exploiting geometry and known regularities.