Neural activity patterns of recent experiences are reactivated during sleep in structures critical for memory storage, including hippocampus and neocortex. This reactivation process is thought to aid memory consolidation. Although synaptic rearrangement dynamics following learning involve an interplay between slow-wave sleep (SWS) and rapid eye movement (REM) sleep, most physiological evidence implicates SWS directly following experience as a preferred window for reactivation. Here, we show that reactivation occurs in both REM and SWS and that coordination of REM and SWS activation on the same day is associated with rapid learning of a motor skill. We performed 6 h recordings from cells in rats' motor cortex as they were trained daily on a skilled reaching task. In addition to SWS following training, reactivation occurred in REM, primarily during the pre-task rest period, and REM and SWS reactivation occurred on the same day in rats that acquired the skill rapidly. Both pre-task REM and post-task SWS activation were coordinated with muscle activity during sleep, suggesting a functional role for reactivation in skill learning. Our results provide the first demonstration that reactivation in REM sleep occurs during motor skill learning and that coordinated reactivation in both sleep states on the same day, although at different times, is beneficial for skill learning. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.