In this thesis, we study Distributed Averaging Dynamics and its main application, i.e. Distributed Optimization. More specifically, the results of this thesis can be divided into two main parts: 1) Ergodicity of distributed averaging dynamics, and 2) Distributed optimization over dependent random networks.

First, we study both discrete-time and continuous-time time-varying distributed averaging dynamics. We show a necessary and a sufficient condition for ergodicity of such dynamics. We extend a well-known result in ergodicity of time-homogeneous (time-invariant) averaging dynamics and we show that ergodicity of a dynamics necessitates that its (directed) infinite flow graph has a spanning rooted tree. Then, we show that if groups of agents are connected using a rooted tree and the averaging dynamics restricted to each group is $\Pst$ and ergodic, then the dynamics over the whole networks is ergodic. In particular, this provides a general condition for convergence of consensus dynamics where \textit{groups} of agents capable of reaching consensus follow each other on a time-varying network.

Then, we study the averaging-based distributed optimization solvers over random networks for both convex and strongly convex functions. We show a general result on the convergence of such schemes for a broad class of dependent weight-matrix sequences. In addition to implying many of the previously known results on this domain, our work shows the robustness of distributed optimization results to link-failure. Also, it provides a new tool for synthesizing distributed optimization algorithms. To prove our main theorems, we establish new results on the rate of convergence analysis of averaging dynamics and non-averaging dynamics over (dependent) random networks. These secondary results, along with the required martingale-type results to establish them, might be of interest to broader research endeavors in distributed computation over random networks.