The ability to manipulate magnetic properties over ultrafast timescales has great potential for applications in magnetic memory and spintronic devices. Advances in pump-probe spectroscopy have enabled the excitation and observation of magnetization dynamics on femtosecond to picosecond timescales in a variety of magnetic materials, including ferromagnets and complex oxides. However, an understanding of the underlying physical processes and mechanisms that govern magnetism at ultrafast timescales remains a challenge. In this thesis, I studied the magnetization dynamics of epitaxial and amorphous Fe thin films excited by an ultrafast THz pump and simulated these dynamics with a Landau-Lifshitz-Gilbert (LLG) model. The dynamics of the Fe films were found to be independent of the film crystallinity. Both epitaxial and amorphous films showed a magnetization response that was coherent with the field of the THz pulse with no demagnetization observed at longer timescales. This behavior was modeled using an overdamped LLG equation. The observed dynamics were found to be dependent on the THz pulse properties rather than the crystallinity of the films. I also investigated the dynamics of a complex oxide La0.7Sro.3CoO3 (LSCO) thin film with optical pump-probe spectroscopy across its ferromagnetic to paramagnetic phase transition at 200 K. For LSCO, a similar ultrafast response in both the ferromagnetic and paramagnetic phases was observed. An oscillating component in optical reflectivity was also observed which was attributed to either an acoustic phonon or polariton-phonon mode. However, as the temperature of the film approached the Curie temperature, the relaxation timescales of the film increased dramatically, indicating a slowing down of spin-lattice relaxation processes. These studies highlight the complex behavior observed following excitation in metallic heterostructures and correlated oxides at ultrafast timescales.