UC San Diego

The objective of this thesis is to investigate algorithms that yield improved image quality for motion compensated frame interpolation or frame rate up-conversion. We address the problems of having broken edges and deformed structures in an interpolated frame by hierarchically refining motion vectors on different block sizes. The proposed novel, low complexity motion vector processing algorithm at the decoder explicitly considers the reliability of each received motion vector based on the received residual energy and motion vector correlation. By analyzing the distribution of residual energies and effectively merging blocks that have unreliable motion vectors, the structure information can be preserved. In addition to the unreliable motion vectors due to high residual energies, there are still other unreliable motion vectors that cause visual artifacts but cannot be detected by high residual energy or bidirectional prediction difference in motion compensated frame interpolation. We further propose a correlation-based motion vector processing to classify motion vector reliability and correct identified unreliable motion vectors by analyzing motion vector correlation in the neighborhood. These unreliable motion vectors are gradually corrected based on their bidirectional difference energy levels so that we can effectively discover the areas where no motion is reliable to be used, such as occlusions and deformed structures. For these areas, we further propose an adaptive frame interpolation scheme by analyzing their surrounding motion distribution and accurately choosing forward or backward predictions. Since the proposed motion vector processing method exploits the spatial information such as residual energy and motion vector correlation, experimental results show that our interpolated results have better visual quality than other methods. However, we still can observe the flickering effects during video display especially in motion boundaries and areas having uniformly distributed texture. Therefore, to further ensure the temporal stability in these motion sensitive areas or video frames, a novel motion vector processing approach based on motion temporal reliability analysis is proposed. For each motion vector candidate, its temporal variation of absolute bidirectional prediction difference along the motion trajectory is examined and classified into several predefined curvatures that are obtained by motion reliability statistic analysis. Any motion vectors that can match one of the predefined curvatures will be considered as possibly temporal reliable motion. This algorithm is employed to improve the motion quality for the proposed motion vector processing method. As a result, the proposed method can effectively improve the motion accuracy for the bidirectional motion vector processing and outperforms other approaches in terms of visual quality, PSNR (Peak Signal to Noise Ratio), and structure similarity.