Working memory (WM) is the ability to hold information for online processing. As the basis of long-term memory formation and a fundamental construct of thinking, it is paramount that we understand how WM works. Distributed network models posit that the prefrontal cortex (PFC) supports WM by coordinating top-down control over other regions involved in sensory representation and long-term memory. We utilized an episodic memory paradigm that probes WM for identity, spatial, and temporal information to examine the PFC dependent model of WM. In two studies, multimodal electrophysiology data reveal that PFC control over WM is fundamentally dynamic in nature, and that WM is dependent on activity distributed across anterior and posterior cortical regions. Results challenge the simple PFC model of WM.

Ch. 1 presents evidence from intracranial recordings that frontal and medial temporal lobe (MTL) theta rhythms carry WM-related activity, and uncovers two WM systems. The PFC-MTL system exhibits bidirectional interaction that shifts with msec precision in response to task demands. In contrast, MTL rhythms direct activity in the orbitofrontal cortex via theta rhythms that do not vary with task demands. These findings support a bidirectional PFC-MTL system in humans – in which theta rhythms subserve executive control during episodic memory formation. Ch. 2 presents evidence from patients with unilateral PFC damage, which shows that a posteriorly-sourced alpha-beta network provides adequate resources for well above-chance WM accuracy. However, when the PFC is intact, PFC low theta activity increases commensurate with executive demand, and PFC-sourced slow rhythms and posteriorly-sourced alpha-beta rhythms travel in opposite directions to support optimal WM.

Ch. 3 reviews 15 years of intracranial research on human memory, and considers the potential of intracranial electrophysiology as a technique to address unresolved questions in the neuroscience of human memory. Ch. 4 presents key themes from this work for younger readers; specifically, it introduces the concepts of cross-frequency coupling between theta rhythms and fast activities in the MTL, and inter-regional PFC-MTL synchrony for memory formation. In a second public outreach piece, appendix 1 introduces the logic of neuropsychology to younger readers to understand why memories of music are resilient to the deleterious effects of amnesia and dementia. Appendix 2 shows that WM develops in children along with increases in sustained attention, and appendix 3 reviews evidence that executive control develops commensurate with PFC connectivity across distributed neural networks. Finally, appendices 4-5 present applications of research on WM and control, together delineating behavioral and neural underpinnings of optimal relational reasoning in neurologically healthy adults.

The discovery of directional tuned neurons in the primary motor cortex has advanced motor research in several domains. For instance, in the area of brain machine interface (BMI), researchers have exploited the robust characteristic of tuned motor neurons to allow monkeys to learn control of various machines. In the first chapter of this work we examine whether this phenomena can be observed using the less invasive method of recording electrocorticographic signals (ECoG) from the surface of a human's brain. Our findings reveal that individual ECoG channels contain complex movement information about the neuronal population. While some ECoG channels are tuned to hand movement direction (direction specific channels), others are associated to movement but do not contain information regarding movement direction (non-direction specific channels). More specifically, directionality can vary temporally and by frequency within one channel. In addition, a handful of channels contain no significant information regarding movement at all. These findings strongly suggest that directional and non-directional regions of cortex can be identified with ECoG and provide solutions to decoding movement at the signal resolution provided by ECoG.

In the second chapter we examine the influence of movement context on movement reconstruction accuracy. We recorded neuronal signals recorded from electro-corticography (ECoG) during performance of cued- and self-initiated movements. ECoG signals were used to train a reconstruction algorithm to reconstruct continuous hand movement. We found that both cued- and self-initiated movements could be reconstructed with similar accuracy from the ECoG data. However, while an algorithm trained on the cued task could reconstruct performance on a subsequent cued trial, it failed to reconstruct self-initiated arm movement. The same task-specificity was observed when the algorithm was trained with self-initiated movement data and tested on the cued task. Thus, the correlation of ECoG activity to kinematic parameters of arm movement is context-dependent, an important constraint to consider in future development of BMI systems.

The third chapter delves into a fundamental organizational principle of the primate motor system - cortical control of contralateral limb movements. However, motor areas also appear to play a role in the control of ipsilateral limb movements. Several studies in monkeys have shown that individual neurons in primary motor cortex (M1) may represent, on average, the direction of movements of the ipsilateral arm. Given the increasing body of evidence demonstrating that neural ensembles can reliably represent information with a high temporal resolution, here we characterize the distributed neural representation of ipsilateral upper limb kinematics in both monkey and man. In two macaque monkeys trained to perform center-out reaching movements, we found that the ensemble spiking activity in M1 could continuously represent ipsilateral limb position. We also recorded cortical field potentials from three human subjects and also consistently found evidence of a neural representation for ipsilateral movement parameters. Together, our results demonstrate the presence of a high-fidelity neural representation for ipsilateral movement and illustrates that it can be successfully incorporated into a brain-machine interface.

The prefrontal cortex (PFC) is critical for organizing thought and behavior in accordance with internal goals. Patients with prefrontal lesions do not exhibit easily observable sensory deficits or simple behavioral deficits, but instead are unable to select appropriate responses based on internal representations. Neuroimaging in humans and single unit studies in non-human primates show PFC involvement across many task contexts requiring higher order control of behavior. However, despite converging evidence from neuroimaging and electrophysiology, the neuronal mechanism of flexible thought and behavior remains one of the most fundamental and elusive questions in neuroscience. One hypothesis posits that the PFC functionally bridges stimulus evaluation and response execution across time and cortical space, yet how this region implements the stimulus-to-response transformation is unknown. The first study described here defines how response selection is implemented during goal-directed behavior by examining the timing, magnitude, and spatial distribution of local cortical activation in humans. These results demonstrate that intrinsically sustained stimulus-to-response activation provides the foundation for binding stimulus processing with response execution on a single trial basis. Notably, this effect is seen across multiple tasks and sensory modalities. The second study introduces a novel method for probing how oscillations contribute to cognitive control by algorithmically extracting the center frequencies and bandwidths of oscillatory components in electrophysiological power spectra. The results demonstrate that the use of individualized frequency bands, which account for oscillatory heterogeneity, unmasks relationships between task and behavior that are hidden when using predefined, canonical frequency bands. Oscillatory dynamics support neuronal communication, and as such the method presented here is a critical step in understanding how the PFC exerts cognitive control over distributed cortical and subcortical regions supporting goal-directed behavior.

It has long been thought that sensory systems operate by representing information in a hierarchy of sensory features, and that these features build upon one another. From low-level information such as spectral content, to high-level information such as word content, the sensory system must rapidly extract all of these features from the world. However, the precise nature of these levels of representation, as well as how they interact with one another, is not well-understood. In audition, intermediate sensory representations are often studied in animals, using techniques that treat neurons as a linear filter for incoming sensory inputs. If those inputs are spectro-temporal features (e.g., a spectrogram), then the result is a Spectro Temporal Receptive Field (STRF). This describes how the neural unit in question (e.g., a neuron) will respond to patterns in spectro-temporal space. It has been a crucial tool in understanding sensory processing in low-level neural activity. Using this approach it is also possible to study how this neural representation changes under different experimental conditions. STRF plasticity has been shown in both reward- and context-modulated experiments in animals.

In recent years, it has been suggested that similar techniques may work in modeling the activity of neural signals recorded from humans. As we cannot generally record from single unit activity in humans, this approach relies on proxies for neural activity – specifically in the high-frequency activity (HFA) of electrocorticography electrodes. This poses a unique opportunity for two reasons: First, human language is a natural stimulus set for studying hierarchical feature representations in the brain. There are many ways to decompose speech into both auditory and linguistic components, and each of these could serve as inputs to the modeling technique described above. Second, humans are especially skilled at using high-level context such as their experience and assumptions about the world in order to change their behavior. This poses a unique opportunity to study the plasticity of speech representations in the brain.

This thesis reports several new approaches towards studying the sensory representation of speech in the human brain, as well as how these representations may change due to experience. It aims to bridge the literature in rodents and songbirds with ideas in human electrophysiology in order to pursue new approaches to studying perception in humans.

Herein I research the role of the basal ganglia and prefrontal cortex in visual working memory and attention by examining patients with focal, unilateral lesions to these brain regions. By combining patient-based behavioral research with scalp electroencephalography (EEG) I study the specific deficits caused by focal frontal brain lesions and explore the effects that such lesions have on diverse cortical network functioning related to working memory and attention. Furthermore, I investigate the role that neuroplasticity plays in compensating for damage to the prefrontal cortex as relates to working memory and attention.

By examining the localization of cognitive functions in the brain and how these seemingly fixed locations may reflect flexible neural networks that can change in response to brain damage, I show how the intact homologous prefrontal cortex compensates for the damaged hemisphere in patients with unilateral prefrontal lesions when these patients are cognitively challenged. I then expand on this notion of cognitive compensation by demonstrating that behavioral performance is reduced when we block the fidelity of visual information transferred from the damaged to the intact hemisphere. Finally, in a methodological analysis of a unique patient cohort, I address the advantages and limitations of scalp EEG.

Defining specific brain regions by function does not necessarily inform us about how cognitive functions arise or change and adapt during development and in response to brain injury or disease. Rather, I argue that we must adopt a dynamic view of cognition wherein cortical regions are but nodes in fluctuating, malleable networks that give rise to the complexities of human behavior.

For over a century, an abundance of research has tried to elucidate the neurobiological basis of language processing in the human cortex. Neuroimaging and lesion studies have provided great insight into what functions different brain structures subserve. While these techniques provide a high spatial resolution they are limited in the temporal domain. Conversely, contributions from non-invasive electrophysiology provided a high temporal resolution with a limited ability to localize cortical sources. The combined spatial and temporal dynamics of cortical processing during language perception and production remains largely unknown. This dissertation addresses this issue by employing unique neuronal population recordings from neurosurgical patients performing linguistic tasks. The studies described here elucidate the timing, magnitude and spatial extent of cortical processing during perception and production of language. The results provide evidence on the level of single-trial that: 1) A rich network of independent and spatially distinct functional sub-regions of cortex subserve perception and production of language. 2) Neighboring sub-regions 4 mm apart can exhibit inverse functional specific responses to linguistic stimuli and self produced speech. 3) Broca's area is not involved in the actual act of articulation but rather in speech preparation and interfacing perception and production. Taken together, these results defy century old dogmas and suggest that language is supported by a complex network of independent sub-regions, with Broca's area acting as a mediator between perception and production rather than as the seat of articulation.

One of the core problems the brain has to solve is how to navigate and interact with the external world. This requires a complex analysis of sensory input, the translation of perceptual input to goal-directed behavior, followed by motor planning and execution. In this thesis we investigated two crucial aspects of this perception-action cycle. First, we examined the underlying neural mechanisms that support response inhibition. Here, novel sensory information is integrated on very short time-scales to cancel an already planned action. The frontal cortex is believed to play a crucial role in the temporal organization of goal-directed behavior and cognitive control and is implicated in stopping a motor response. Using the high spatiotemporal resolution of electrocorticography (ECoG), we found evidence for two distinct processes localized to the middle frontal gyrus (MFG). High-frequency band (HFB) power increased in stop-trials before the stop-signal reaction time (SSRT), showing no difference between successful and unsuccessful stops. We interpret this activation as contributing to the stopping process, either by signaling the stop-signal itself, or by implementing attentional control. A second HFB activation was observed after the go and stop processes have finished, and was larger for unsuccessful stops, and is likely related to behavioral monitoring. Our results support the notion that frontal cortex implements different functions related to stopping.

Implementing the perception-action cycle not only involves re-acting to novel information from the senses in a bottom-up manner. It is believed that the brain also implements a strategy anticipating future events based on prior knowledge. Here we investigated how anticipation of sounds influences auditory processing. Using both EEG and ECoG, we employed a task with omissions of expected sounds, thereby isolating endogenous responses to expectations in auditory cortex. We found that a subset of auditory active electrodes in lateral superior temporal gyrus (STG) and superior temporal sulcus (STS) showing HFB power increases to omissions. We were able to successfully decode whether the subject heard the syllable ‘Ba’ or ‘Ga’. However, which sound was omitted could not be decoded from auditory active sites, nor from the omission HFB increase specifically. We also observed a negative ERP in posterior STG in the intracranial data, which may be related to an auditory cortical generator of the N2 component. In a separate EEG studies we also observed both an N2 negativity, as well as a negativity occurring before the intracranial negativity, the source of which may be in A1, a region which we could not access intracranially. Finally, a P3a ERP was observed in EEG, which points to both the HFB and ERP effects in posterior STG to be signatures of auditory-specific salience or mismatch detection.

Interaction with our environment involves a constant influx of visual inputs. There has been significant investigation into the complex neural transformations that reflect sensory input as a series of neural activity patterns. However, a paucity of human studies have captured the canonical early and late neural activity patterns that occur across stages of visual processing. Thus, we leverage the millimeter spatial and millisecond temporal resolution of invasive recordings in neurosurgical patients to explore how visual information is rapidly represented in the human brain over time. Chapter 1 investigates visual repetition suppression, the phenomenon where repeated presentations of a stimulus produce a decreased neural response. Repetition suppression occurs ubiquitously across sensory modalities and has been widely studied using non-human primate electrophysiology and human neuroimaging. Here, we investigate the temporal profile of repetition suppression in humans when images are repeated over long time lags. We find robust repetition suppression associated with smaller peak magnitudes, lower total responses, and earlier peak responses. We discuss these findings in the context of theoretical models of repetition suppression and the early and late temporal windows of perception. Chapter 2 explores visual perception using binocular rivalry, a unique experience in which different images are simultaneously presented to each eye, and perception dynamically alternates between the two images. This provides a rare opportunity to disentangle conscious perception and sensory inputs since perception alternates between the two possible percepts, while the stimulus remains static. Significant debate surrounds whether binocular rivalry is the result of competitive interactions between populations in visual areas, top-down influence from associative cortices, or a combination of the two. Chapter 2 attempts to resolve these disagreements using intracranial recordings across the human brain. The first portion explores the initial perception of an ambiguous binocular image. We compare neural responses to ambiguous and unambiguous images and explore their temporal profile across regions. We find that early neural activity predicts subsequent perception. The latter portion investigates changes in perception by studying the neural correlates of endogenous switching from seeing one percept to the other. We find that associative cortices initiate changes in perception and precede sensory reactivation in visual cortices. We find a bi-directional relationship distributed across visual and associative cortices that i) encode and represent ambiguous percepts and ii) are engaged at temporally distinct periods across perception. Overall, this dissertation reports several novel findings in studying sensory networks supporting perception. It aims to bridge the literature in nonhuman primates and human neuroimaging with novel data in human electrophysiology to further understand how the human brain supports visual perception.

The amount of sensory input received by the human brain far surpasses its capacity for conscious processing. Attention serves to gate incoming inputs to enable the receiver to select aspects of the environment for further processing based on its relevance to higher-level goals. When attention is impaired, behavior goes awry, and this often has serious consequences. While it is clear that attentional processes are critical for humans’ ability to interact effectively with the environment, the lack of a clear and conclusive definition of attention itself has, at times, impeded research progress on attention in the neural and behavioral sciences.

In this dissertation, I examine the construct of human attention from both empirical and theoretical standpoints. In Chapter 1, I provide an introduction to the concept of attention, including diverging definitions. I also articulate the urgency of obtaining a clearer understanding of the neurocognitive processes that support attention. Chapter 2 presents an empirical study of a specific type of attention, known as visual search. The study uses rare intracranial recordings in humans to present evidence consistent with the view that visual search, a classical attention behavior, may need to be reinterpreted as navigation in visual space. In Chapter 3, I present a theoretical discussion of the distinctions and similarities between attention and expectation, including interpretational pitfalls in experimental designs aiming to directly compare the two. Chapter 4 provides closing thoughts with emphasis on future directions for this field of research.