Discovered by Segal and Bargmann in 1960s, the Segal--Bargmann trasnform is an important tool in mathematical physics, intertwining the Heisenberg and Fock pictures in quantum mechanics. We will discuss representations of the two-parameter {\em free} unitary Segal--Bargmann transform, which is the large-$N$ limit of Segal--Bargmann--Hall transform on unitary group. Motivated by a conditional expectation interpretation of the Segal-Bargmann transform, we derive the integral kernel for the large-$N$ limit of the two-parameter Segal-Bargmann-Hall transform over the unitary group, and explore its limiting behavior. We also extend the notion of circular systems to more general elliptic systems, giving an alternate construction of our new two-parameter {\em free} unitary Segal-Bargmann-Hall transform via a Biane-Gross-Malliavin type theorem.

We study the complex-time Segal-Bargmann transform B_{s,tau}^{K_N} on a compact type Lie group K_N, where K_N is one of the following classical matrix Lie groups: the special orthogonal group SO(N,R), the special unitary group SU(N), or the compact symplectic group Sp(N). Our work complements and extends the results of Driver, Hall, and Kemp on the Segal-Bargman transform for the unitary group U(N). We provide an effective method of computing the action of the Segal-Bargmann transform on \emph{trace polynomials}, which comprise a subspace of smooth functions on K_N extending the polynomial functional calculus. Using these results, we show that as N -> infinity, the finite-dimensional transform B_{s,tau}^{K_N} has a meaningful limit G_{s,tau} which can be identified as an operator on the space of complex Laurent polynomials.

This thesis contributes to the emerging field of nonlinear random matrix theory and deep learning theory. The main contributions are summarized as follows:

In the linear-width regime, where the network widths scale proportionally with the sample size, we include proof of the global laws of the neural tangent kernel (NTK) and conjugate kernel (CK) matrices across layers. For datasets with low-dimensional signal structures, we characterize the outlier eigenvalues and eigenvector alignments of the CK matrices, extending recent results on spiked covariance models.In the ultra-wide regime, where the first layer's width is much larger than the sample size, we show that spectra of both CK and NTK matrices converge to a deformed semicircle law.

Going beyond random initialization, we investigate the spectral properties of trained weight, CK, and NTK matrices through empirical and theoretical analyses. Empirically, it demonstrates the invariance of bulk spectra under small learning rates, the emergence of outliers with large learning rates, and the heavy-tailed distributions after adaptive gradient training, correlating them with feature learning.Theoretically, we prove the invariance of bulk spectra under small constant learning rates and characterize the feature learning phenomenon, where gradient descent optimizes the first-layer weights, leading to a rank-one spiked structure in the weight and CK matrices, with spike eigenvectors aligning with test labels.

A combination of the method of perturbation formulas and polynomial approximation is employed to compute the $k^{\text{th}}$ derivatives of (1) maps on symmetrically normed ideals of a unital Banach algebra induced by a holomorphic function, (2) maps on the self-adjoint elements of symmetrically normed ideals of a unital $C^*$-algebra induced by functions of a real variable that are ``slightly better than $C^k$,'' and (3) maps on the self-adjoint elements of integral symmetrically normed ideals of a von Neumann algebra induced by functions of a real variable that are ``slightly better than $C^k$ and Lipschitz.'' Along the way, the ``separation of variables'' approach to defining multiple operator integrals on non-separable Hilbert spaces is developed. As an application to free probability, a free It\^o formula of P. Biane and R. Speicher is extended, reinterpreted, and made more computationally flexible.