UC San Diego

The growth in size and popularity of the Internet has led to several wide area computing efforts where the computing power of several thousands of machines are combined together. As these systems grow in size, it becomes increasingly difficult to control and monitor them using a client-server-like architecture. Introducing decentralization and autonomy in these systems seems to be the obvious solution. Autonomy has several advantages. These include easy scaling, removal of single points of failure, cost-effective deployment and greater promotes collaborative environments. However, autonomous systems present certain challenges as well. The absence of server- like nodes make it difficult to locate and manage available resources, schedule tasks efficiently across them, assure security and develop applications that can make use of these powerful platforms. This dissertation addresses some of these issues. We present autonomous techniques for resource location, task scheduling and programmability in large scale computing systems. Our work tries to answer the question 'Do autonomous solutions of these issues achieve same (or comparable) performance benefits as their centralized counterparts?' We propose GUARD, a protocol for locating computing resources in a completely autonomous environment. GUARD can locate resources matching single or multiple parameters and it provides a trade off between efficiency and probability of success. We then discuss A-FAST, a decentralized scheduling technique for homogeneous independent tasks, that uses the notion of pressure from fluid networks to autonomously guide tasks to resources. Simulations show A- FAST to achieve a high level of efficiency (≈98% of optimal steady state throughput) for both computation and communication dominated tasks. The dissertation also discusses a programming platform that we designed and implemented to allow the development of applications for decentralized environments. Our platform allows programmers to develop code without having knowledge of the underlying system and allows platform-specific information to be fed to the program at runtime. We show how this platform can be used to generate code that adapts to several situations without changing the original program or involving the programmer in the scheduling decisions