Depth data acquisition has drawn considerable interest in recent years as a result of the rapid development of 3D technology. A large number of acquisition techniques are based on hardware devices, e.g., infra-red sensors, time-of-flight camera, and LiDAR, etc, whereas they have limited performance due to poor depth precision and low resolution. In some situations computational methods are preferred due to its flexibility and low cost. These computational techniques, typically known as depth estimation algorithms, estimate depth maps (in terms of disparities) from a pair of stereo images. However, existing computational techniques are sensitive to various factors such as noise, camera alignment, and illumination, resulting that a few samples are reliable. Therefore, dense depth data reconstruction from sparse samples is a significant technological challenge.
In this thesis, we mainly consider the problem of dense depth data reconstruction from a subset of samples. We present computationally efficient methods to estimate dense depth maps from sparse measurements, and we further extend the work to dense depth video estimation. Working on single depth image, we have three main contributions: First, we provide empirical evidence that depth maps can be encoded much more sparsely than natural images by using common dictionaries such as wavelets and contourlets, and show that disparity maps can be sparsely represented by a combined wavelet and contourlet dictionary. Second, we propose a subgradient algorithm for dense depth image reconstruction, and propose an alternating direction methods of multipliers (ADMM) algorithm with a multi-scale warm start procedure to further speed up the convergence. Third, we propose a two-stage randomized sampling scheme to optimally choose the sampling locations, thus maximizing the reconstruction performance for a given sampling budget. Experimental results show that the proposed methods produce high quality dense depth estimates, and are robust to noisy measurements.
For dealing with depth video sequences, a framework for depth video reconstruction from a subset of samples is proposed. By redefining classical dense depth estimation into two individual problems, sensing and synthesis, we propose a motion compensation assisted sampling (MCAS) scheme and a spatio-temporal depth reconstruction (STDR) algorithm for reconstructing depth video sequences from a subset of samples. Using the 3-dimensional extensible dictionary, 3D-DWT, and applying alternating direction method of multiplier technique, the proposed STDR algorithm possesses scability for temporal volume and efficiency for processing large scale depth data. Exploiting the temporal information and corresponding RGB images, the proposed MCAS scheme achieves an efficient 1-Stage sampling. Experimental results show that the proposed depth reconstruction framework outperforms the existing methods and is competitive comparing to our previous work on sampling single depth image, which requires a pilot signal in the 2-Stage sampling scheme. Finally, to estimate missing reliable depth samples from varying input sources, we present an inference approach using geometrical and color similarities. Applications for depth video super resolution from uniform-grid subsampled data and dense disparity video estimation from a subset of reliable samples are presented.