The simultaneous Localization and Mapping (SLAM) problem for single robot systems has been a topic of intense research for many years. However, in scenarios where a single robot is not sufficient to explore and reconstruct a large or complex environment, multi-robot SLAM can prove to be a promising solution. By working and communicating together, multiple robots can cover larger areas and improve the accuracy of localization and mapping.

A traditional approach for multi-robot SLAM includes using a central server, which communicates with each robot to exchange information and updates. However, this approach is not ideal for real-time scenarios, as it can cause delays and render the computation power of each agent futile. Moreover, if the central server fails, the whole system will break down.

The main aim of this thesis is to propose a distributed filtering algorithm to build an accurate sparse global map by leveraging the computation power of multiple robots and minimizing communication across them. The approach involves each robot estimating its state locally and sharing its observations with its neighbors. The robots must agree upon an estimate of the common observations, which is achieved through consensus optimization. The algorithm is inspired by the distributed stochastic mirror descent approach to solve a constrained variational inference problem that can be decomposed. The optimization algorithm ensures consistency among agents and their measurements and is systematically evaluated in both simulation and real-time datasets.

This thesis considers the problem of safe navigation for autonomous mobile robots working in partially known environments with static and non-adversarial dynamic obstacles. A virtual reference governor system is designed to serve as a local regulation point for the real robot system. We calculate a safe set of the robot-governor system by actively measuring the distance from surrounding static obstacles. The motion of the governor is designed to guide the robot towards the goal, while remaining within the safe set. To avoid dynamic obstacles, a control barrier function is built to further constrain the governor-robot system away from dynamic obstacles. A quadratically constrained quadratic program (QCQP) is formulated to minimally modify the governor control input to ensure dynamic obstacle avoidance. Combining these two techniques, the robot can navigate autonomously in unknown environments -- slowing down when approaching obstacles, speeding up in free space, and reacting to the position and velocity of the surrounding dynamic obstacles. Our techniques are demonstrated in simulations and real-world experiments using a ground robot equipped with LiDAR to navigate among a cluttered environment in the presence of humans.

We consider the model-based reinforcement learning framework where we are interested in learning a model and control policy for a given objective. We consider modeling the dynamics of an environment using Gaussian Processes or a Bayesian neural network. For Bayesian neural networks we must define how to estimate uncertainty through a neural network and propagate distributions in time. Once we have a continuous model we can apply standard optimal control techniques to learn a policy. We consider the policy to be a radial basis policy and compare it's performance given the different models on a pendulum environment.

Neural implicit surface representations are a promising new development in surfacemodeling. However, the challenges inherent in training neural networks in a continual fashion are still holding them back from being widely used in real-time, incremental scene mapping. We propose a method for learning a neural representation of a signed distance function from trajectories of posed depth images that is both computationally efficient and avoids the problem of catastrophic forgetting. We demonstrate our approach by producing high-quality scene reconstructions in 2D and 3D and incrementally building 2D neural-implicit maps.

Tropical rainforests worldwide are negatively impacted from a variety of human-caused threats. Unfortunately, our ability to study these rainforests is impeded by logistical problems such as their physical inaccessibility, expensive aerial imagery, and/or coarse satellite data. One solution is the use of low-cost, Unmanned Aerial Vehicles (UAV), commonly referred to as drones. Drones are now widely recognized as a tool for ecology, environmental science, and conservation, collecting imagery that is superior to satellite data in resolution. We asked: Can we take advantage of the sub-meter, high-resolution imagery to detect specific tree species or groups, and use these data as indicators of rainforest functional traits and characteristics? We demonstrate a low-cost method for obtaining high-resolution aerial imagery in a rainforest of Belize using a drone over three sites in two rainforest protected areas. We built a workflow that uses Structure from Motion (SfM) on the drone images to create a large orthomosaic and a Deep Convolutional Neural Network (CNN) to classify indicator tree species. We selected: 1) Cohune Palm (Attalea cohune) as they are indicative of past disturbance and current soil condition; and, 2) the dry-season deciduous tree group since deciduousness is an important ecological factor of rainforest structure and function. This framework serves as a guide for tackling difficult ecological challenges and we show two additionally examples of how a similar architecture can help count wildlife populations in the Arctic.

Many real-world mobile robot applications, such as disaster response, military reconnaissance, and environmental monitoring, require operating in unknown and unstructured environments. This calls for algorithms that empower robots with active information gathering capabilities in order to autonomously and incrementally build a model of an environment. In this dissertation, we present a novel 3-D multi-class online mapping approach using a stream of range and semantic segmentation observations. Moreover, we derive a closed-form expression for the Semantic Shannon Mutual Information (SSMI) between our proposed map representation and a sequence of future sensor observations. Using an octree data structure, we reduce the memory footprint of the map storage for large-scale environments, while simultaneously accelerating the computation of mutual information. This allows real-time integration of new sensor measurements into the map, and rapid evaluation of candidate future sensor poses for exploration. Additionally, we introduce a differentiable approximation of the Shannon mutual information between grid maps and ray-based measurements, enabling gradient-based occlusion and collision-aware active mapping. The gradient-based active mapping in the continuous space of sensor poses reduces the optimization complexity from exponential in the number of robots to linear, paving the way for extension from a single agent to a team of robots. We formulate multi-robot exploration as a combination of multi-robot mapping and multi-robot planning, where both sub-problems are specified as an instance of multi-agent Riemannian optimization. We propose a general distributed Riemannian optimization algorithm that solves both mapping and planning in fully decentralized manner. Our method, named Riemannian Optimization for Active Mapping (ROAM), enables distributed collaborative multi-robot exploration, with only point-to-point communication and no central estimation and control unit. Lastly, we deploy our active mapping method on a team of ground wheeled robots in both simulation and real-world environments, and compare its performance with other autonomous exploration approaches.

Semantic understanding and reconstruction of the surrounding 3D environment is a necessary requirement for intelligent robots to autonomously fulfill various tasks like environment exploration, surveillance, autonomous driving, indoor household and healthcare service, to name a few. Although the progress on semantic understanding of 2D images is impressive, building a 3D consistent, meaningful yet compact and scalable semantic map for robotics applications is still challenging. In this dissertation, we develop 3D semantic map approaches for robots in different form, including the dense semantic map and the object-level semantic map. We propose TerrainMesh, a dense semantic map in the form of 3D mesh, for the terrain reconstruction from the aerial images and the sparse depth measurements. The joint 2D-3D and geometric-semantic learning framework is proposed to reconstruct the local semantic meshes and the global semantic mesh can be constructed by merging the local meshes with the help of the SLAM algorithm. We investigate the object-level semantic map constructed from 3D measurements. We propose CORSAIR, a retrieval and registration algorithm for point cloud objects. A 3D sparse convolution neural network model is trained to extract the global features for similar shape retrieval and the local per-point features for correspondence generation for pose registration with the help of symmetry. We develop ELLIPSDF, a bi-level object shape model including a coarse level of 3D ellipsoid and a fine level of 3D continuous signed distance function (SDF). We also design the approach to initialize and optimize the object pose together with the bi-level shape from the multiple depth image observations. We also propose the object-level semantic map from the 2D images and investigate its connections with the localization task. We introduce the object mesh model and its observation model of semantic instance segmentation and semantic keypoints. We derive the observation residual function and minimize it to optimize both the object states and the camera poses. We develop OrcVIO, object residual constrained visual-inertial odometry with object ellipsoid and semantic keypoints model. We implement the observation residual model between the ellipsoid and its 2D semantic bounding box and semantic keypoints and connect this with MSCKF framework to implement the online tightly coupled estimation of object and IMU-camera states. The object-level semantic map also provides a meaningful yet efficient representation of the environment. Finally we discuss the potential directions to further extend the 3D semantic understanding technique for robotics.

This dissertation considers the problem of safe navigation for autonomous mobile robots working in partially known and unknown environments with static and non-adversarial moving obstacles. Given a geometric path generated by standard path planner, we develop reference-governor based tracking control policy to continuous generate proper set-points along the path for downstream low-level stabilizing controller.

This new method systematically puts planning, motion prediction and safety metric design together to achieve environmentally adaptive and safe navigation. Our algorithm balances optimality in travel distance and safety regarding passing clearance. Robots adapt progress speed adaptively according to the sensed environment, being fast in wide open areas and slowdown in narrow passages and taking necessary maneuvers to avoid dangerous incoming obstacles. Directional distance measure, motion prediction and custom costmap are integrated properly to evaluate system risk accurately with respect to local geometry of surrounding environments. Using such risk estimation, reference governor technique and control barrier function are worked together to enable adaptive and safe path tracking in dynamical environments. We validate our algorithm extensively both in simulations and hardware platforms in challenging real-world environments.

Recent advances in the field of neural implicit surface representation has led to the use of signed distance functions as a continuous representation of complex surfaces. Though signed distance functions allow for a very simple and concise mapping of the environment, rendering these surfaces from signed distance functions are non-trivial and require iterative methods such as ray marching, leading to high render times. The inclusion of direction in these functions have been proposed and applied to various object-level applications, but have yet to be proposed for scene-level applications. This thesis explores the idea of scene-level directional distance functions and some of the problems that it faces. It then proposes three different methods through which the directional distance function can be implemented and potential solutions to some of the drawbacks of the directional distance function.

These days, robots are contributing to many aspects of our lives and this role is growing. One of the fundamental problems in robotics is shape representation and mapping, that is necessary for most robotic applications. In this regard, from a stream of lidar scans or RGB-D images we model an object shape or map an environment. One of the main challenges in this regard is an appropriate shape representation with appropriate ray-tracing speed and accuracy. There are many other challenges for mapping like being 3D, continuous, probabilistic, online, containing semantic information, and the possibility of the mapping algorithm to be distributed. In this thesis we propose a novel mapping algorithm based on existing shape representation signed distance function and Gaussian Processes which is able to reconstruct an online, continuous, probabilistic 3-D representations of the geometric surfaces and semantic classes in the environments. Then we extend it to a distributed mapping algorithm that is able to be perform in a network of robots. Additionally, we propose a novel shape representation signed directional distance function (SDDF) that measures signed distance from a query location to a set surface in a desired viewing direction. The main advantage of SDDF, compared to existing shape representations, is that ray tracing can be performed as a look-up operation through a single function call. Additionally, we propose a deep neural network model for SDDF shape learning. A key property of our DeepSDDF model is that it encodes by construction that distance to the surface decreases linearly along the viewing direction. This ensures that the model represents only valid SDDF functions and allows reduction in the input dimensionality and analytical confidence in the distance prediction accuracy even far away from the surface. We show that our DeepSDDF model can represent entire object categories and interpolate or complete shapes from partial views.