UC San Diego

Digital images and videos are compressed by removing their spatial, temporal and visual redundancies. Although the encoded signals are more compact and easier for storing or transmitting, the correlation between their pixels is distorted. This causes coding artifacts, which degrade the visual quality of the signals and cause annoyance to the viewers. Compressed images and video sequences should be enhanced prior to being sent to the displaying devices. Previous methods on quality enhancement focused on separately improving the quality of each frames. In this way, the temporal consistency between frames is not guaranteed. Furthermore, characteristics of temporal artifacts are not thoroughly studied and exploited for artifact removal. This dissertation investigates these characteristics and proposes novel methods to reduce both spatial and temporal artifacts. The dissertation covers three main topics in spatio-temporal filtering: quality enhancement, coding and data pruning. The dissertation starts with analyzing the usage of further information from surrounding frames beside the information in the current frames. First, a simple linear temporal filter is studied to investigate the quantization error between the filtered output and its original signal. Reduction of the quantization error verifies that using information from surrounding frames can enhance the quality of the current frame. Next, a non-linear fuzzy spatio-temporal filter is proposed to adapt to the characteristics of the coding artifacts. The dissertation also proposes a novel metric for evaluating the flickering artifacts. For applications on coding, these spatio-temporal filters are used in the encoding loop as the enhanced reference frame. They are then optimized to maximize their performance in artifact reduction. For special cases when filter coefficient for the pixel of interest is set to zero, the filter becomes an estimator or an interpolator. The dissertation extends the discussion for the case of implementing the optimal estimator in the encoding phase using multi reference frames. Finally, an edge-directed interpolator is studied and used in data pruning based compression, which can help reducing the bit-rate of the encoded bit-stream. This interpolator is applied to determine the most effective way in dropping the data before compression and to reconstruct the pruned signal back to its original form.