UCSF

When controlling a motor prosthetic device or computer cursor using a brain-machine interface (BMI), users guide their movements relying solely on visual feedback. In contrast, our natural ability to plan and execute movements relies on feedback from multiple sensory signals, and in particular on proprioception---the sense of the body's position in space. Proprioception guides movement without the need for continuous visual monitoring and is integrated with vision, when both are available, to improve behavioral performance. Such multisensory integration appears to be learned from multisensory experience, a process that theoretically can be driven solely by the shared statistical structure of the inputs. To achieve naturalistic control over prosthetic devices, BMIs will likewise need a proprioceptive feedback signal, one that can ultimately be integrated with natural vision. Here, we demonstrate a novel, learning-based approach to artificial sensory feedback. We show that monkeys can learn to use, in a naturalistic way, a continuous, multi-dimensional, multi-channel intracortical microstimulation (ICMS) signal that encodes task-relevant feedback. After training with correlated ICMS and visual feedback, monkeys could perform a goal-directed reaching task without vision, using only the ICMS signal. Additionally, the animals integrated the natural and artificial sensory inputs, combining both into a minimum-variance sensory estimate of hand position relative to the target. The ICMS signal further resembled natural sensation in its plasticity.For example, when a misalignment was imposed between vision and ICMS, the ICMS estimate adapted back towards the rewarded visual cue. Furthermore, the monkey's valuation of the accuracy of the ICMS estimate was updated in response to trial conditions. When the ICMS is randomly, but not consistently, perturbed, the monkey's estimate using artificial feedback grows less precise. Finally, we discussed theoretical ways to improve the ICMS signal reliability by changing the parameters of the signal. Together these results demonstrate that a learning-based approach can be used to provide a rich artificial sensory signal, suggesting a new strategy for restoring proprioception to patients with BMIs as well as a powerful new tool for studying the adaptive mechanisms of sensory integration.