Locating and labeling objects in images remains a central problem in computer vision, and the challenge of doing so efficiently has taken on increasing importance with the deployment of autonomous vehicles, edge recognition applications, and ever-growing foundation models.We consider a variation of object localization called localized image retrieval, which combines the problems of image retrieval and query localization, by ranking the precise sub-regions in a set of scenes which match the user query. In this work, we will examine the topic of representation learning for localized image retrieval, specifically through the lens of the person search problem. We will show novel work in categorizing and improving efficiency of person search models during training and inference by introducing new modular approaches, comparing query-centric and object-centric methods, and developing methods for weakly-supervised and self-supervised pre-training. Finally, we will discuss how identity search, prompt localization, object detection, and tracking can be unified under the single problem of localized image retrieval, and consider connections to generalist models.

Human Motion capture has been an active area of research for many years and has applications in many fields such as gaming, entertainment, physical therapy, and ergonomics. Most commercially available motion capture systems use numerous markers to be placed on the body of the human subject requiring a significant setup time. In this dissertation, we develop the architecture and algorithms for markerless motion capture with a multi-view structured light system. In contrast to existing markerless approaches that use multiple camera streams, we reconstruct the scene by combining the views from three structured light stations using sinusoidal phase shift patterns, each equipped with one projector, a stereo pair of cameras for phase unwrapping, and a color camera. The three stations surround the subject and are time multiplexed to avoid interference. Phase-shifted sinusoidal patterns offer low decoding complexity, require as few as three projection frames per reconstruction, and are well suited for capturing dynamic scenes. In these systems, depth is reconstructed by determining the phase projected onto each pixel in the camera and establishing correspondences between camera and projector pixels. Typically, multiple periods are projected within the set of sinusoidal patterns, thus requiring phase unwrapping on the phase image before correspondences can be established.

There are three novel contributions to this dissertation; first, we present a novel phase unwrapping algorithm across space and time in order to generate a temporally consistent point cloud. Specifically, we combine a quality guided phase unwrapping approach with absolute phase estimates from the stereo cameras to solve for the absolute phase of connected regions. Second, we develop a calibration method for multi-camera-projector systems in which sensors face each other as well as share a common viewpoint. We use a translucent planar sheet framed in PVC piping as a calibration target which is placed at multiple positions and orientations within a scene. In each position, the target is captured by the cameras while it is being illuminated by a set of patterns from various projectors. The translucent sheet allows the projected patterns to be visible from both sides, allowing correspondences between devices that face each other. The set of correspondences generated between the devices using this target are input into a bundle adjustment framework to estimate calibration parameters. Third, we develop algorithms to reconstruct dynamic geometry of a human subject using a template generated by the system itself. Specifically, we deform the template to each frame of the captured geometry by iteratively aligning each bone of the skeleton. This is done by searching for correspondences between the source template and the captured geometry, solving for rotation of bones, and enforcing constraints on each rotation to prevent the template from taking on anatomically unnatural poses. Once the geometry of the dynamic mesh is reconstructed, the template is textured using the color cameras from the multi-view structured-light system. We demonstrate the effectiveness of our approach both qualitatively and quantitatively for an actual sequence of a moving human subject by synthesizing arbitrary views of the dynamic scene.

In this dissertation, we present several techniques used to automatically generate virtual models of indoor building environments. The interior environment of a building is scanned by a custom hardware system, which provides raw laser and camera sensor readings used to develop these models. Our modeling techniques can be separated into three categories: 2D floor plan models, simplified 2.5D extruded models, and fully complex and detailed 3D models. All models are produced automatically from the output data of a backpack-mounted ambulatory scanning system, which can scan multiple floors of a building efficiently. We can capture the entirety of a large building in only a few hours, including staircases and other hard to reach locations, which improves upon the state-of-the-art static scanning by several orders of magnitude. Our algorithms are capable of producing many kinds of 3D virtual building models while traversing through a GPS-denied environment, including point clouds, 2D floor plans, and 3D building models.

The novel contributions of this dissertation fall into three groups. First, we present multiple methods for producing 2D floor plans in a scalable fashion. We automatically partition these floor plans into separate building levels and separate rooms within each level. These floor plans can also be extruded into simplified 2.5D models of the building environment. Second, we present multiple techniques to produce complex 3D models of the scanned environment, capturing all observed detail. Third, we present several techniques that combine these two types of building models -- simple and complex -- to perform additional analysis of the building environment. Such analysis includes segmenting objects and furniture in the environment as separate models, reducing noise and artifacts within the models, and demonstrating novel visualization techniques. Additionally, we show several example datasets generated by the system in real-world environments, including buildings of over 40,000 square feet. We demonstrate how such building models are applicable in many fields of study, including architecture, building energy efficiency, virtual walk-throughs of buildings, indoor navigation, and augmented and virtual reality.

Current 802.11 networks do not typically achieve the maximum potential throughput despite link adaptation and cross-layer optimization techniques designed to alleviate many causes of packet loss. A primary contributing factor is the difficulty in distinguishing between various causes of packet loss, including collisions caused by high network use, co-channel interference from neighboring networks, and errors due to poor channel conditions.

In current 802.11 networks, there are two adaptations occurring simultaneously but independently each using packet loss as an indication of a different problem. The first is contention window adaptation, which implicitly assumes all losses are due to collision, and the second is rate adaptation, which implicitly assumes all losses are due to channel errors. Thus, in all high-loss scenarios, at least one of these two adaptations behaves incorrectly, unnecessarily increasing the impact of packet loss.

In this thesis, we propose a novel method for estimating various collision type probabilities locally at a given node of an 802.11 network. The key to our approach is a signal we call the busy-idle (BI) signal, which is a binary-valued record of the channel occupancy over time as observed locally by a given node. We show that if the access point(AP) periodically broadcasts its BI signal to associated nodes at an overhead of less than 2%, the nodes can use this information in conjunction with locally observable quantities to obtain partial spatial information about the network traffic. With this spatial information, nodes can estimate their probabilities of various types of loss and make adaptations to improve throughput and/or network utility.

We provide a systematic assessment and definition of the different types of collision, and show how to approximate each of them using local information and the AP's BI signal.

We verify our methods using NS-2 simulations, as well as using the Ath5k open source wireless card driver in an experimental testbed. We show that our proposed methodology accurately estimates overall collision probability to within 5%. This experimental verification demonstrates the feasibility of our collision probability estimation approach and the resulting throughput gains in practice.

In addition, we describe a suite of methods which exploit these loss probability estimates and the BI signal to improve network throughput and utility. We show via NS-2 simulations that we can achieve gains of up to 400% via modulation rate adaptation, 350% via contention window adaptation, 25% via packet length adaptation, and 50% via carrier sense threshold adaptation.

Future lithography systems must produce chips with smaller feature sizes, while maintaining throughput comparable to today's optical lithography systems. This places stringent data handling requirements on the design of any direct-write maskless system. To achieve the throughput of one wafer layer per minute with a direct-write maskless lithography system, using 22 nm pixels for 45 nm technology, a data rate of 12 Tb/s is required. A recently proposed datapath architecture for direct-write lithography systems shows that lossless compression could play a key role in reducing the system throughput requirements. This architecture integrates low complexity hardware-based decoders with the writers, in order to decode a compressed rasterized layout in real time. To this end, a spectrum of lossless compression algorithms have been developed for rasterized integrated circuit (IC) layout data to provide a tradeoff between compression efficiency and hardware complexity. In this thesis, I extend Block Context Copy Combinatorial Code (Block C4), a previously proposed lossless compression algorithm, to Block Golomb Context Copy Code (Block GC3), in order to reduce the hardware complexity, and to improve the system throughput. In particular, the hierarchical combinatorial code in Block C4 is replaced by Golomb run-length code to result in Block GC3. Block GC3 achieves minimum compression efficiency of 6.5 for 1024 x 1024, 5-bit Poly layer layouts in 130 nm technology. Even though this compression efficiency is 15% lower than that of Block C4, Block GC3 decoder is 40% smaller in area than Block C4 decoder.

In this thesis, I also illustrate hardware implementation of Block GC3 decoder with FPGA and ASIC synthesis flow. For one Block GC3 decoder with 8 x 8 block size, 3233 slice flip-flops and 3086 4-input LUTs are utilized in a Xilinx Virtex II Pro 70 FPGA, corresponding to 4% of its resources. The decoder has 1.7 KB internal memory, which is implemented with 36 block memories, corresponding to 10% of the FPGA resources. The system runs at 100 MHz clock rate, with the overall output rate of 495 Mb/s for a single decoder. The corresponding ASIC implementation results in a 0.07 mm^2 design with the maximum output rate of 2.47 Gb/s.

I also explore the tradeoff between encoder complexity and compression efficiency, with a case study for reflective E-beam lithography (REBL) system. In order to accommodate REBL's rotary writing system, I introduce Block RGC3, a variant of Block GC3, in order to adapt to the diagonal repetition of the rotated layout images. By increasing the encoding complexity, Block RGC3 achieves minimum compression efficiency of 5.9 for 256 x 2048, 5-bit Metal-1 layer layouts in 65 nm technology with 40 KB buer; this outperforms Block GC3 and all existing lossless compression algorithms, while maintaining a simple decoder architecture.

GPS-denied indoor mobile mapping has been an active area of research for many years. With applications such as historical preservation, entertainment, and augmented reality, the demand for both fast and accurate scanning technologies has dramatically increased. In this thesis, we present two algorithmic pipelines for GPS-denied indoor mobile 3D mapping using an ambulatory backpack system. By mounting scanning equipment on a backpack system, a human operator can traverse the interior of a building to produce a high-quality 3D reconstruction. In each of our presented algorithmic pipelines, data from a number of 2D laser scanners, a camera, and an IMU is fused together to track the 3D position of the system as the operator traverses an unknown environment.

This thesis presents a number of novel contributions for indoor GPS-denied 2.5 and 3D mobile mapping using a number of 2D laser scanners, a camera, and an IMU. First, for 3D mapping we develop a tightly coupled EKF estimator for fusing data from all sensors into a single optimized 3D trajectory. By formulating each sensor's contributions independently, we demonstrate a modular algorithm that easily scales to an arbitrary number of 2D laser scanners. In contrast to existing work that either assumes a known fixed map or limits the environment to a set of axis aligned planes, we demonstrate the ability to map environments containing horizontal and vertical planes of arbitrary orientation with no a priori information. Additionally, through timing and complexity analysis, we demonstrate that the runtime of the proposed EKF estimator is only linear in the acquisition time. Secondly, by including in our EKF estimator the laser scanner's spatial and temporal calibration parameters, we present a novel laser calibration methodology. Through simulated and real-world data, we validate that the proposed algorithms are capable of calibrating both the extrinsic and temporal misalignments present in our system's laser data. Lastly, we address the scalability of the proposed approach by utilizing a graph optimization post processing step that overcomes any accumulated drift in the EKF estimator. We then validate the proposed 3D end-to-end localization system using 3 multi-story datasets collected from real-world environments. The system's reconstructions are compared against CAD drawings of the buildings and are shown to achieve an intersection over union of over 96% on all datasets. Lastly, we demonstrate accuracy improvements over our 2.5D methods using a comparison test against data collected with a static scanner.

In addition to 3D mapping, we also present a methodology for 2.5D mapping with three novel contributions. First, we present a method for automatically segmenting barometric pressure data based on the floor of the building it was collected from. Specifically, by using Bayesian non-parametrics we are able to demonstrate simultaneous floor detection and the corresponding data segmentation. The data segmentation is then used to extend classical 2D particle filtering across any number of discrete building stories. Secondly, we demonstrate a genetic scan matching algorithm used to estimate loop closure constraints even without an accurate initial condition. Through simulation and real-world experiments we show an improvement over state of the art scan matching techniques. Next, we present two metrics that are used to validate the results of the genetic scan matching algorithm. We use both a correlation and shape metric to demonstrate robust and accurate validation of loop closure constraints in indoor environments. Lastly, we compare and characterize the performance of the proposed 3D and 2.5D mapping techniques developed in this thesis. Although the 2.5D mapping techniques are more computationally lightweight, we show that the accuracy of system is significantly improved using the 3D mapping algorithm.