Self-organized criticality is a ubiquitous phenomenon that appears in many complex dynamical systems. During this special phase, the system experiences long-range correlation in space and in time. In order to understand its origin, we developed a method that can predict whether or not a stochastic dynamical system will be chaotic. We hypothesize that self-organized criticality and chaos originate from breaking the supersymmetries of the complex dynamical systems; therefore, the eigenvalue spectrum for the Fokker-Planck Hamiltonian should have pairs of complex eigenvalues on its imaginary axis. By applying the Fokker-Planck equation to the Chua oscillator, we show that the eigenvalues move closer to the imaginary axis as the system becomes chaotic. Also, we built a self-organized critical circuit using CMOS transistors. The circuit exhibits "avalanche" behaviors, in which groups of oscillators become out of synchronization together. The avalanche statistics show power laws in both avalanche size and inter-avalanche time.

This doctoral dissertation is a comprehensive study on a novel method based on unitary synaptic weights to construct intrinsically stable neural systems. By eliminating the need to normalize neural activations, unitary neural networks deliver faster inference speeds and smaller model sizes while maintaining competitive accuracies for image recognition. In addition, unitary networks are drastically more robust against adversarial attacks in natural language processing systems because unitary weights are resilient to small input perturbations. The last portion focuses on a small demo that implements unitary neural nets in quantum computing. With the comprehensive performance evaluation in classical machine learning, the rigorous framework in mathematics, and the exploration of quantum computing, this dissertation establishes a solid foundation for unitary neural networks in the future of deep learning.