During learning, synaptic connections between excitatory neurons in the brain display considerable dynamism, with new connections being added and old connections eliminated. Synapse elimination offers an opportunity to understand the features of synapses that the brain deems dispensable. However, with limited observations of synaptic activity and plasticity in vivo, the features of synapses subjected to elimination remain poorly understood. Here, we examined the functional basis of synapse elimination in the apical dendrites of L2/3 neurons in the primary motor cortex throughout motor learning. We found no evidence that synapse elimination is facilitated by a lack of activity or other local forms of plasticity. Instead, eliminated synapses display asynchronous activity with nearby synapses, suggesting that functional synaptic clustering is a critical component of synapse survival. In addition, eliminated synapses show delayed activity timing with respect to postsynaptic output. Thus, synaptic inputs that fail to be co-active with their neighboring synapses or are mistimed with neuronal output are targeted for elimination.

Motor learning is accompanied by widespread changes within the motor cortex, but it is unknown whether these changes are ultimately funneled through a stable corticospinal output channel or whether the corticospinal output itself is plastic. We investigated the consistency of the relationship between corticospinal neuron activity and movement through in vivo two-photon calcium imaging in mice learning a lever-press task. Corticospinal neurons exhibited heterogeneous correlations with movement, with the majority of movement-modulated neurons decreasing activity during movement. Individual cells changed their activity across days, which led to changed associations between corticospinal activity and movement. Unlike previous observations in layer 2/3, activity accompanying learned movements did not become more consistent with learning; instead, the activity of dissimilar movements became more decorrelated. These results indicate that the relationship between corticospinal activity and movement is dynamic and that the types of activity and plasticity are different from and possibly complementary to those in layer 2/3.

The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca²⁺/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.