UCLA

Advances in technologies that can record and stimulate deep-brain activity in humans have led to impactful discoveries within the field of neuroscience and contributed to the develop- ment of novel closed-loop stimulation therapies for neurological and psychiatric disorders. Human neuroscience research based on intracranial electroencephalography (iEEG) is con- ducted on voluntary basis during various stages of participant’s disease treatment using both external (in-clinic) and implantable systems. In clinical practice, external systems serve as monitoring and testing ground for biomarker extraction and closed-loop neuromodulation, which are, once approved, translated into a compact and low compute resource implantable version for disorder treatment.External systems allow recordings with fine spatiotemporal resolution at the expense of participant’s mobility due to their large size, while implantable devices have reduced record- ing capabilities and they are not restricted to clinical environment. Due to high transmission and processing latencies across multiple devices, external systems have limited support for testing computationally expensive online biomarker detection and machine-learning based closed-loop electrical stimulation paradigms including online stimulation programmability.

The motivation for this work comes from the need to extend capabilities of externalized systems, allowing more naturalistic (freely-moving) human neuroscience experiments with fine spatiotemporal resolution. Additionally, externalized systems should provide flexible and local hardware resources that can support real-time and moderately complex embedded neural decoders (biomarker extraction), which in turn could be used to trigger adaptive closed-loop stimulation with low latency. In order to demonstrate initial proof-of-concept technology, this work incorporates: 1. A small versatile neuromodulation platform that can be wearable and lightweight, supporting up to 16 depth electrode arrays; 2. A high-rate (∼4 MB/s on all channels) interfacing of the analog sensing and stimulation front-ends with wearable hardware suitable for embedded machine learning algorithms including artificial neural networks (usually >100M multi-accumulate operations or MACs); 3. A state of the art, performance-driven, neural decoder, small enough to run on an embedded hardware and large enough to generalize across participants; 4. Real-time training and inference with millisecond latency; 5. Closing the loop from the decoder output to the stimulation engines.

Therefore, we developed a wearable, miniaturized, embedded, and external neuromodula- tion platform built from previously reported integrated circuits for sensing and stimulation, and interfaced with Edge Tensor Processing Unit (TPU) for real-time neural analysis. The Neuro-stack can record and decode single-neuron (32 channels), local field potential (LFP; 256 channels) activity, and deliver highly programmable current-controlled stimulation (256 channels) during stationary and ambulatory behaviors in humans. The TPU Dev Board was chosen because of the ability to perform 2 trillion MACs per second (64 � 64 MAC matrix at 480 MHz) using 2 W of power, with data bandwidth of 40 MB/s. Additionally, the system contains a field-programmable gate array (FPGA) for data pre-processing (filtering, down-sampling) and ARM-based microprocessor (TPU Dev Board) for data management, device control, and secure wireless access point. The Neuro-stack interfaces with the brain through commonly used macro- and micro-electrodes. The Neuro-stack validation includes in-vitro testing of recorded signal quality and measurement of system induced delays (e.g., closed-loop delay from sensing to stimulation site - 1.57 � 0.19 ms). We provide in-vivo single-unit, LFP, iEEG, and stimulation delivery recorded (2 – 40 channels) from twelve hu- man participants who had depth electrodes implanted for epilepsy evaluation. Among this data are also the first recordings of single-neuron activity during human walking.

To utilize hardware capabilities of the Neuro-stack, we developed a software decoder based on prerecorded human LFP data, which uses TensorFlow artificial neural network (sequential convolutional 1D and recurrent layers) to predict the outcome of a memory task from raw data with higher performance (F1-score 88.6 � 5.5%) than current state of the art that use shallow machine learning methods (∼70%) under a latency constraint. To shorten the signal processing latency of our decoder, while keeping the accuracy high, the trained and tested model was then ported (coefficient quantization from 16-bit floating-point to 8-bit fixed-point) to the TPU co-processor to make the prediction in real-time on the Neuro- stack. Additionally, we utilized transfer learning approach to update the TPU model with coefficients that were fine-tuned to each participant in real-time. The Neuro-stack decoder was in-vitro validated as part of the time adaptive closed-loop stimulation delivery with pre-configured stimulation current parameters based on the LFP decoder outputs that were predictive of unsuccessful memory encoding. We also used the Neuro-stack to perform human in-vivo real-time binary prediction (69% F1-score) of memory task performance from medial temporal lobe (MTL) regions. Each inference step was executing 193M MACs in 2.8 ms on average, for total round-trip delay of 4.4 ms.

The Neuro-stack is a wearable and versatile neuromodulation platform, able to record and stimulate large number of iEEG and single-unit channels, and process the raw data using artificial neural networks in real-time. These functionalities were not available so far on a single low-latency device. Thus, the Neuro-stack can improve existing or allow completely new research studies. By using the Neuro-stack, researchers could, for example, determine the neural mechanisms underlying human freely-moving behaviors (e.g., spatial navigation) to identify spatially selective neurons and their modulation by cognition that have been previously discovered only in animals. Also, the Neuro-stack decoder could be used to identify more complex multimodal biomarkers as well as to record and characterize their exact changes under stimulation with known or previously not possible parameters. This could lead to developing novel neuromodulation therapies for patients with brain disorders, while they participate in hospital trials resembling real world environments.