UC San Diego

With the rapid development of digital video broadcasting technologies, the requirements on image qualities have been increased significantly. Furthermore, the computational power of today's processors is ever increasing and it becomes feasible to use more robust and complex algorithms to perform post-processing tasks without distorting edges seriously. In this dissertation, we explore the application of Markov Random Field (MRF) models in video post-processing. MRF models are able to model the spatial dependencies among pixels in an image and can impose spatial constraints on the processed images. They are a good fit for content-adaptive processing purposes. We formulate the 2-D discontinuity- adaptive smoothness (DAS) constraint and impose it on the processed images via MRF modeling. This constraint assumes piecewise smoothness of images. However, the smoothness does not disturb discontinuity features, such as edges and object boundaries. It helps the processed image to achieve smoothness along edge directions and remain sharp in across-edge directions. Local edge direction information is required when formulating the 2-D DAS constraint. Considering the limitations of the conventional binary representation, (edge direction, non-edge direction), regarding local edge directions, we propose a robust statistic-based approach to measure the local edge direction. In the proposed method, local edge directions are represented using a weight vector. Using a rational number which is normalized to the range of zero to one, we provide the likelihood for each direction to be the true edge direction. The formulated 2-D DAS constraint is applied to motion compensated (MC) de-interlacing and spatial interpolation problems. In MC de-interlacing, an efficient protection strategy against erroneous motion vectors is necessary. By the MRF model, the 2-D DAS constraint is imposed on the de-interlaced frame. The final output of each pixel site is the one that fits its context best under the 2-D DAS constraint. The context includes neighboring pixels from both the available field and newly interpolated field. Especially, the de- interlaced edges are expected to have strong geometric regularity. In spatial interpolation, we propose the MRF- EDI (Edge-Directed Interpolation) method, which uses the local edge direction information through the 2-D DAS constraint in an implicit manner to create clean and sharp edges. Low-complexity implementations of the proposed algorithms are also investigated. One low-complexity implementation is to apply the proposed iterative optimization method only to near edge areas because simple, deterministic interpolation provides satisfactory results in smooth areas. Another low-complexity implementation is to replace the iterative optimization with a single-pass implementation. The complexity is reduced significantly by single-pass while the performance degradation is negligible. In addition to video post-processing topics, this thesis contains some work on wavelet-based and H.264/ AVC-based scalable video coding. In wavelet-based SVC, the Low Band Correction (LBC) technology is preferred for its efficiency in reducing overhead information and bit stream length. However, with the LBC technology, the lowpass filter implementing spatial scalability has to be a wavelet filter, which obeys the strong half-band condition. We develop a parametric design method to design a class of wavelet filters that considers all the design requirements and is suitable for both motion estimation and compression in SVC. In the H.264 AVC-based scalable video model, motion vectors have to be coded and sent for all the spatial layers even in low complexity and low bit- rate applications, which is not efficient. We investigate the possibility to perform inter-layer motion vector prediction/interpolation in low bit-rate applications such that not all motion vectors have to be sent to the decoder side. Several motion vector prediction methods are combined and a novel mode-map is produced to indicate the chosen method for a block