UC Santa Cruz

Today's software depends heavily upon low-level systems techniques that are largely taken for granted, including dynamic loaders, binary formats (e.g., ELF) and dependency resolution protocols. These foundational techniques were developed decades ago and designed for the computing environment of the day: resource-constrained machines with limited memory, storage, bandwidth, and running less complex software stacks than seen today. As the assumptions underlying these foundational techniques have become outdated, predicated on assumptions and conventions from a bygone era, our foundational techniques have become bottlenecks and points of fragility in software development, debugging, and performance management. As we progress through the 21st century, code-bases are poised to grow ever larger and more connected, further exacerbating these issues.This thesis identifies and targets three key issues that arise due to the contrast of modern software scale and the features offered by legacy foundational systems techniques. First, it causes significant performance bottlenecks, particularly during application startup. Second, it limits observability into runtime behavior and actual dependency bindings. Third, it complicates dependency management, often leading to the dependency chaos prevalent in layered software stacks. The sheer size and intricate nature of contemporary software demand a critical re-evaluation of these low-level primitives.This dissertation confronts these challenges directly and makes the key insight that present-day systems achieve periods of stability between updates, yet many existing tools fail to exploit this fact, perpetuating these aforementioned issues. First, addressing the intricacies of composed software stacks common in HPC, we analyze dependency resolution pitfalls (e.g., RPATH/RUNPATH interactions) and introduce Shrinkwrap, a technique that explicitly encodes the full dependency graph into binaries using absolute paths and raising all dependencies to the top of the graph, enhancing runtime robustness and performance by eliminating ambiguous, convention-based library searches. Second, we present sqlelf, a tool that reimagines ELF binary analysis by modeling shared objects within a relational database. Leveraging the expressive power of SQL, sqlelf enables flexible, powerful, and aggregate analysis across entire software installations, overcoming the single-file, text-dump limitations of traditional tools like readelf. Finally, we developed stable loading, realized in the MATR system, a novel approach that decouples dependency resolution from runtime execution by pre-calculating and materializing relocation mappings during distinct management times. This reclaims substantial startup performance and provides guaranteed observability of symbol bindings throughout execution epochs. Collectively, these contributions provide pathways to faster, more observable, and more manageable software systems by modernizing the often overlooked foundations upon which contemporary complex software is built.