A scanning tunneling microscope (STM) capable of performing pump-probe spectroscopy integrates unmatched atomic-scale resolution with high temporal resolution. In recent years, the union of electronic, terahertz, or visible/near-infrared pulses with STM has contributed to our understanding of the atomic-scale processes that happen between milliseconds and attoseconds. This time-resolved STM (TR-STM) technique is evolving into an unparalleled approach for exploring the ultrafast nuclear, electronic, or spin dynamics of molecules, low-dimensional structures, and material surfaces. Here, we review the recent advancements in TR-STM; survey its application in measuring the dynamics of three distinct systems, nucleus, electron, and spin; and report the studies on these transient processes in a series of materials. Besides the discussion on state-of-the-art techniques, we also highlight several emerging research topics about the ultrafast processes in nanoscale objects where we anticipate that the TR-STM can help broaden our knowledge.