We study the complex-time Segal-Bargmann transform B_{s,tau}^{K_N} on a compact type Lie group K_N, where K_N is one of the following classical matrix Lie groups: the special orthogonal group SO(N,R), the special unitary group SU(N), or the compact symplectic group Sp(N). Our work complements and extends the results of Driver, Hall, and Kemp on the Segal-Bargman transform for the unitary group U(N). We provide an effective method of computing the action of the Segal-Bargmann transform on \emph{trace polynomials}, which comprise a subspace of smooth functions on K_N extending the polynomial functional calculus. Using these results, we show that as N -> infinity, the finite-dimensional transform B_{s,tau}^{K_N} has a meaningful limit G_{s,tau} which can be identified as an operator on the space of complex Laurent polynomials.