UC Irvine

Spatially augmented reality (SAR) systems illuminate objects or environments of different shapes or size, starting from table top objects to room-sized environments, using multiple projectors that are casually aligned creating full or partial overlaps with their neighbors. Full overlaps or superposition is used to increase brightness and partial overlaps or tiling is used to increase resolution and scale. Such systems enable applications such as visualization, entertainment and education by projecting data right on the physical 3D model.

Creating a seamless display on complex shapes using multiple projectors entail two primary challenges. Geometric registration which addresses pixel alignment in the overlap of multiple projectors and warping to conform to underlying 3D shape and color registration addresses the spatial variation of color within and across different projectors and also color variation introduced by the complex non-planar shape of the 3D surface.

My thesis presents un uniform approach for consistent calibration of SAR systems enabling easy deployment of spatially augmented reality environments of different sizes (from small objects to large immersive environments), different shapes (inside-looking-out or outside-looking in), and different configurations (tiled or superimposed). First We show that a unified approach that relies on cross-correlation and validation between the devices (projectors and cameras) and display properties achieves a consistent system reconstruction resulting in a highly accurate, robust and scalable registration in multi-projector systems that is not possible by independent calibration of devices and reconstruction of surface properties. Second we present the first method to achieve complete color seamlessness (both luminance and chrominance) for multi-projector displays on completely arbitrary shapes using a novel shape-aware 3D color gamut morphing method. Finally we present multiple interaction modalities, namely interaction of the handheld projector (a) with the surface geometry only, (b) with overlaid digital content only (from other static or dynamic handheld projectors), and (c) with both the digital content and the surface simultaneously.