A method to study global spatial patterns related to sensory perception in scalp EEG

PriorstudiesofmultichannelECoGfromanimalsshowedthatbetaandgammaoscillationscarriedpercep- tualinformationinbothlocalandglobalspatialpatternsofamplitudemodulation,whenthesubjectswere trained to discriminate conditioned stimuli (CS). Here the hypothesis was tested that similar patterns could be found in the scalp EEG human subjects trained to discriminate simultaneous visual-auditory CS. Signals were continuously recorded from 64 equispaced scalp electrodes and band-pass ﬁltered. The Hilbert transform gave the analytic phase, which segmented the EEG into temporal frames, and the analytic amplitude, which expressed the pattern in each frame as a feature vector. Methods applied to the ECoG were adapted to the EEG for systematic search of the beta–gamma spectrum, the time period after CSonset,andthescalpsurfacetolocatepatternsthatcouldbeclassiﬁedwithrespecttotypeofCS.Spatial patterns of EEG amplitude modulation were found from all subjects that could be classiﬁed with respect tostimuluscombinationtypesigniﬁcantlyabovechancelevels.Thepatternswerefoundinthebetarange (15–22Hz)butnotinthegammarange.TheyoccurredinthreeshortburstsfollowingCSonset.Theywere non-local, occupying the entire array. Our results suggest that the scalp EEG can yield information about the timing of episodically synchronized brain activity in higher cognitive function, so that future studies in brain–computer interfacing can be better focused. Our methods may be most valuable for analyzing data from dense arrays with very high spatial and temporal sampling rates.


Introduction
Studies of the neural correlates of cognition in experimental animals (Freeman and Grajski, 1987;Freeman and Van Dijk, 1987; Barrie et al., 1996;Freeman and Barrie, 2000;Freeman and Burke, 2003;Freeman, 2005) have provided a candidate for the neural "code" used by sensory cortex to express the content of a percept. This code is carried by a brief burst of oscillatory activity in the beta or gamma frequency range. During such a burst or "frame" of activity, oscillations become phase synchronized at a shared instantaneous frequency. This synchrony is not necessarily zero-lag (i.e., the oscillations are not necessarily perfectly in-phase) but during this time period the phase relationships between different spatial sites remain constant. Within this synchronized spatial domain (frame), the content of the subjective percept is encoded as a spatial pattern of amplitude. Thus the code can be considered as a form of spatial amplitude modula-tion (AM) of a temporal carrier frequency carried in intermittent frames.
By means of ECoG records from animals trained to respond to conditioned stimuli (CSs), this "code" has been identified in the olfactory bulb, (Freeman and Grajski, 1987) and neocortex of rabbit (Barrie et al., 1996), monkey (Freeman and Van Dijk, 1987), gerbil (Ohl et al., 2000), cat (Freeman and Burke, 2003), and human (Freeman et al., 2006a,b;Panagiotides et al., 2008). The brain activity patterns have been classified with respect to conditioned stimuli using animal ECoG signals (Freeman and Barrie, 2000;Freeman, 2005Freeman, , 2006. In every species the classification was maximal when all ECoG channels were used. Deletion of any subset diminished the goodness of classification in proportion to the number removed, irrespective of which channels were deleted (Barrie et al., 1996), showing that the synchronized neural activity sustaining the AM pattern was uniformly distributed with respect to information density. This finding held also for experiments in which the channels were divided into sub-arrays that were fixed on the visual, auditory, somatic, and entorhinal cortices and the olfactory bulb (Freeman et al., 2003b;Freeman and Rogers, 2003), showing that the intermittent phase-locked synchrony encompassed all areas of cortex examined.
The spatial amplitude patterns identified in this work had a relatively fine 'grain' or spatial density, determined by the relatively close spacing of recording electrodes on the surface of the cortex. However the multicortical distribution seen engendered the hypothesis that a similar "code" of intermittently synchronized neural activity may occur at a sufficiently large spatial scale to make AM patterns accessible from scalp EEG, despite the obvious distortions introduced by gyrification, distance from cortex to scalp, pattern attenuation by the shunting effects of the fluid layers embedding the brain, and corruption by scalp EMG. Three existing lines of evidence supported this hypothesis. (a) Suggestions that multiple modules perform cognitive functions by coordinating their dynamics (Pribram, 1971;Baars, 1998;Bressler and Kelso, 2001;Fingelkurts and Fingelkurts, 2001;Breakspear et al., 2004;Kelso and Tognoli, 2007). (b) Multichannel recordings showing widespread intermittent synchrony of oscillations in the beta and gamma range of the EEG (Freeman et al., 2003a;Pockett et al., 2009) and MEG (Stam et al., 2003;Bassett et al., 2006). (c) Numerous reports on the correlation of EEG oscillations, particularly in the gamma range, with a variety of cognitive functions (Tallon-Baudry et al., 1998;Miltner et al., 1999;Rodriguez et al., 1999;Bertrand and Tallon-Baudry, 2000;Haig et al., 2000).
Based on these diverse findings a preliminary search was undertaken to extract from scalp EEG temporal frames containing AM patterns in the beta range. Initially control data from the free available EEG database on the UCI Machine Learning Repository were used, in which subjects were performing a simple sensory discrimination task (Ingber, 1999). The results from applying the techniques developed for extraction of AM patterns from ECoG were positive (Ruiz et al., 2009a). The present report is based on data from experiments designed to facilitate the search for scalp correlates of sensory perception (Pockett et al., 2009) by having subjects engage in multisensory discrimination, in order to optimize the likelihood of multicortical synchronization (Dumenko, 2000).

Experimental design and signal pre-processing
Data from six male subjects between 23 and 44 years old were analyzed. One of the six subjects was left handed (subject JL). Data were collected in the Psychology Dept. of the University of California Berkeley and the study was approved by the UC Berkeley Institutional Review Board. All subjects gave informed consent.
EEG was recorded using BioSemi TM amplifiers with a 64electrode cap using Ag/AgCl electrodes. The sampling rate was 512 Hz and the analog filter was set from DC to 134 Hz. The reference was a point between the CMS and DRL electrodes of the BioSemi TM system, to which the closest approximation is electrode POz.
Continuous records were taken, with the times of various sorts of paired visual and auditory stimuli and responses marked in a 65th recording channel. The experimental design and recording procedure have been documented elsewhere (Pockett et al., 2009). The visual stimuli consisted of a 125 ms flash during which a computer screen turned either entirely red or entirely blue. The auditory stimulus was either a comfortably loud or a much softer 100 ms burst of white noise from two computer speakers. Stimuli were delivered in the following combinations: Red-Loud, Blue-Soft or Blue-Loud. The subject's task was to learn by trial and error which computer key they should press in response to which combination of stimuli. For the first 180 stimuli presentations, positive or negative feedback was delivered at random (Block1). For the next 200 presentations, "Correct" feedback was delivered lawfully (Block2). For the present analysis, data from Red-Loud and Blue-Soft stimu-lus combinations were selected to perform the classification assays. Red-Loud was defined as class1 and Blue-Soft as class2. 60/70 trials of each class from Block1 or Block2 were analyzed separately. A notch filter was applied to remove 60 Hz line noise prior to the extraction of specific data epochs from the continuous recording. Each data trial consisted of four seconds of signal, covering one second before stimulus onset and three seconds after stimulus onset.
Thus far there is some experimental evidence that supports the idea that the sequence formation of frames begins with the abrupt resetting of phase values on every channel, followed by resynchronization and spatial pattern stabilization within the frame (Freeman, 2006). The phase resetting in ECoG and EEG signals had been studied by the use of the Hilbert Transform (Freeman et al., 2003a(Freeman et al., , 2006aRuiz et al., 2009b) showing that the time lapse between phase resetting showed properties similar to a "Rice distribution" (Rice, 1944;Freeman, 2009).
Before the application of the Hilbert transform a temporal filter is desirable in order to select the range of frequency under study (Le Van Quyen et al., 2001;Pikovsky et al., 2002;Freeman and Rogers, 2003). In this work ten temporal filters centered at 16 or 64 Hz with bandwidth was varying from 2 to 30 Hz were used to study the effects of the bandwidth on the distribution of the time lapse between phase resetting. Histograms of the time lapse between phase resetting were constructed and the modal frequency, in Hz, was estimated as the inverse of the modal interval, in ms, which was determined from the histogram maxima.

Location of the AM patterns
Ten steps were required to localize temporal frames in which to calculate amplitude feature vectors for classification.
Step 1: The offset of each channel was removed using the MAT-LAB "detrend" function; this function fits a linear polynomial to the data and then subtracts it from the data.
Step 2: The EEG was normalized by dividing by the global standard deviation of each trial.
Step 3: EEG signals were bandpass filtered by convolution in the time domain with finite impulse response (FIR) filters estimated using the Parks-McClellan algorithm (Matlab "firpm"). Different filter were designed covering the beta (12-25 Hz) and gamma (25-50 Hz) frequencies. The cutoff frequencies of the filters were varied in 3 Hz steps and in 5 Hz steps for the beta and gamma range respectively.
Step 4: The Hilbert Transform (Matlab "hilbert") was applied to the filtered signal in order to obtain the real and imaginary parts of the analytic signal (Barlow, 1993;Pikovsky et al., 2002;Freeman, 2004). Analytic power was squared sum of the real and imaginary parts of the analytic signal. Analytic amplitude was the square root of analytic power. Analytic phase was the ratio of the arctangent of the imaginary part of the analytic signal to the arctangent of the real part of the analytic signal. The analytic phase was unwrapped using the MATLAB "unwrap" function.
Step 5: Instantaneous frequency, the rate of change in phase with time (Hz), was estimated as the successive differences of the mean unwrapped analytic phase divided by the digitizing step and 2 (Freeman, 2004;Freeman et al., 2006a). The instantaneous gradient, the rate of change in phase with distance (rad/mm), was estimated as the slope of the line between the points of minimum and maximum phase for each time sample, where "x" coordinate was the position of the electrode and "y" coordinate the value of phase (Ruiz et al., 2009a). The EEGLAB toolbox was used to generate and store the channel location coordinate map.
Step 6: An initial set of frames candidates were detected in the time series for which (a) the instantaneous frequency was within the temporal band used, (b) the sign of the instantaneous gradient did not change from one time sample to the next and (c) the spatial position of the maximum or the minimum analytic phase did not change from one sample to the next. After that, frame frequency (F N in Hz) and gradient ( N in rad/mm) were calculated by Eqs. (1) and (2): where n was the number of time steps across which a frame candidate was defined, F i was the instantaneous frequency and was the instantaneous gradient (step 5). Other physiological parameters, such as frame phase velocity (B in m/s) analog to the conduction velocity of action potentials on pre-synaptic axons, and frame diameter (D x in mm), the spatial size of the frame (see Eqs. (3) and (4)) (Freeman, 2004;Freeman et al., 2006a), were derived from these parameters. The frame duration was given by the number of digitizing steps multiplied by the digitizing step in s over which the frequency, the phase gradient, and the location of the maximum or the minimum analytic phase remained within the specified limits.
Step 7: Then supplemental anatomical and physiological evaluations were made of the acceptable frame parameter rank in order to exclude spurious frames from the analysis. The phase velocity had to be within the range of conduction velocities of cortical axons (1-10 m/s), the frame had to be stable for more than 3 time samples, and frame diameter had to be smaller than the width of the cerebrum, here taken as 200 mm (Freeman, 2004;Freeman et al., 2006a).
Step 8: For each time sample, covariance over the 64 channels of both analytic amplitude and analytic phase were estimated (Ruiz et al., 2008). Analytic amplitude and analytic phase were computed as in step 4. Covariance was calculated using the MATLAB "cov" function. After an optimization assay, upper and lower covariance thresholds (te2 and te1) were fixed at the values that maximized the number of correctly classified patterns. The final set of stable frames were selected as those frames that fulfilled all of the technical criteria (steps 5 and 6), all of the physiological criteria (step 7) and contained only time samples with analytic amplitude covariance higher than te1 and analytic phase covariance lower than te2.
Step 9: The spatial ensemble average (SEA) of the 64 time series of analytic power was calculated, by taking the mean of the analytic power over all the channels at each time sample. The time sample of maximum SEA analytic power was identified for each frame. Patterns were then extracted as the 64 individual channel values of analytic power at the identified time sample, normalized by dividing each of the 64 values by the mean value for that time sample. This vector defined a point in 64-space. Spatial maps of the centroid of these vectors for each class were generated using the EEGlab function "topoplot".
Step 10: Vectors from each class were divided in two subsets, one for training and the other for testing. The center of gravity or centroid for each class was calculated as the mean value of all 64 × 1 vectors in the training subset for each class. Classification of patterns as class1 or class2 was determined by calculating the Euclidean distances of all patterns in the testing set to the centroid of the training set. This procedure was repeated by cross-classifying all patterns in both subsets. A spatio-temporal pattern was classified correctly if the distance between that pattern and the centroid of its own class was less than the distance between the same pattern and the centroid of the opposite class.
The probability (p) that the number of patterns classified correctly, on each subject and block, with respect to the type of stimulation was significantly different from chance was tested using a binomial distribution. Classification rates from all subjects and blocks were pooled for statistical testing. Statistical significance of the differences in the mean classification rates across the 12 observations (two blocks for each of 6 subjects) and between initial, control and test frames was evaluated by ANOVA. The statistics supplied by ANOVA were analyzed using the MATLAB function "multcompare" in order to search for frames that carried more information, which is reflected in a mean classification rate significantly higher than the others.
During the frame detection procedure some parameters were optimized in order to maximize the number of correctly classified post-stimulus patterns. The parameters to be optimized were (a) the temporal passband cut-off frequencies and (b) the threshold of the analytic amplitude covariance (te1) and (c) the threshold of the analytic phase covariance (te2). Optimal values were found by varying a selected parameter in steps across an appropriate range and searching for the maximum number of correctly classified patterns. Some trials showed channels with movements of the base line or noise artifacts in which usually no frames were detected, so any trial in which there were no frames was omitted from the classification attempt. If more than 10 trials were defective in this way, the value of the parameter producing this situation was disallowed.

Locating the classificatory information in the spatial patterns
In order to analyze which channels (if any) were more or less important for classification, and how the goodness of classification was affected by the number of channels deleted each channel was removed independently and subsets of channels also were removed. Groups of 4,8,16,24,32,40,48 and 56 random channels to be removed were generated by the "rand" function of MATLAB. The relevance of each channel to the spatial AM pattern was tested by the changes on the achieved classification rate when (a) each channel was removed one by one, (b) one channel (as a subset of other randomly deleted channels) was absent. This analysis was iterated 80 times for a 4 channel deletion and 40 times in the other cases. Each channel was removed at least twice during the 80/40 iterations.
The influence of the reference on the spatial distribution of the patterns was also studied, by changing the reference site from the occipital position (electrode POz) to the frontal position (electrode FPz). The original signal was re-referenced by subtracting the signal from the new reference channel from the original signal.

Results
In the time domain, intermittent peaks and valleys of analytic amplitude were observed in all channels, at all frequencies. The top panel of Fig. 1 shows that very small values of analytic amplitude were concomitant with large phase jumps (phase resetting), for both beta (12-25 Hz) and low gamma (20-50 Hz, not shown) ranges. Analytic frequency usually remained within the temporal band used, except at time samples where phase resetting occurred. The middle panel of Fig. 1 shows that, while these phase resetting were not completely simultaneous over large numbers of channels, they were clustered in time. The peaks of mean analytic power were coextensive with plateaus of relatively constant analytic phase differences (white areas in the middle frame or low values of spatial standard deviation of analytic phase differences (SDx) in the bottom frames). Relationship between analytic amplitude and phase jumps. Top: analytic amplitudes (black) and unwrapped analytic phase differences (grey dash) for one channel. Middle: analytic phase differences for each channel. Light areas represent plateaus of phase differences producing analytic frequency within the pass band used. Dark areas represent phase jumps, where the value of analytic frequency was outside the passband used. Bottom: spatial standard deviation of analytic phase differences (black) and mean analytic power, normalized by the standard deviation (gray dash). For all panels, stimulus onset was at 0 ms. Data from the beta band (12-25 Hz) of Block1 of subject F, the other subjects, frequency band and block showed similar results.
The quantity of the phase resetting and its modal frequency were proportional to the filter bandwidth but without any relation with the center frequency (Fig. 2 top). As for ECoG signals, the distribution and rate of the phase jumps showed properties similar to a "Rice distribution" (Rice, 1944;Freeman, 2009) with small deviations from the 0.641 value in the ratio between filter bandwidth vs. model frequency at bandwidths narrower than 12 Hz (Fig. 2 bottom). Thus far, it had been argued that the deviations from 0.641 at wider bandwidths could be introduced by the non-ideal frequency response of the FIR filters (Freeman, 2009).
During the classification-directed optimization process, different temporal filter settings in the beta and low gamma ranges were explored in the search for stable frames carrying AM patterns. Covariance thresholds were varied from 2.5% to 25% or from 10% to 50% of the maximum value of analytic amplitude or analytic phase covariance respectively, searching for the values that maximized the number of correctly classified post-stimulus patterns. The optimized covariance thresholds were slightly different for each subject; so the optimized value of te1 was from 0.0375 to 0.05, representing 3-5% of the maximum amplitude covariance and the optimized value of te2 was from 1 to 2, representing 12.5-25% of the maximum phase covariance, depending on the subject.
Numerous frames fulfilled the initial technical criteria (Methods step 6, Figure S1 top). The application of physiological criteria (Methods step 7, Figure S1 middle) reduced the number of qualifying frames. Finally the optimization of covariance threshold revealed the final set of frames (Methods step 8, Figure S1 bottom). The frames shortly after stimulus onset carried enough information related to the stimulus to achieved classification rates well above chance level. The hypotheses were tested that (a) the AM patterns of analytic power following Red-Loud stimulus arrival would differ from those following Blue-Soft stimulus arrivals (b) the patterns in the pre-stimulus period for both stimulus combinations would not differ significantly.
A subset of frames in the pre-and post-stimulus periods was selected to extract the patterns to be classified and test the hypotheses. In the pre-stimulus period the first three frames at the beginning of each trial (initial frames I) and the last three frames before the stimulus onset (control frames C) were selected. During the post-stimulus period, six consecutive frames (test frames T), starting from 50 ms after the stimulus onset, were selected. In the post-stimulus interval some frames showed up before the 50 ms, but they were dropped from the classification assays because the evoked potentials related to the stimulus started at around 50 ms in almost all subjects.
Frame gradient, frequency, velocity and diameter in the preand post-stimulus interval were estimated from Eqs. (1)-(4). None showed any important differences between pre-and post-stimulus periods, between blocks or between stimulus types. Top panel of Fig. 3 illustrates the results of the classification optimization procedure. It can be seen that analytic power features (extracted in step 9 and classified according to step 10) gave statistically significant classifications in the 15-22 Hz band for the first three frames after stimulus onset (T1, T2 and T3), for both blocks of each subject. T1, T2 and T3 frames always occurred in the first 500 ms after the stimulus onset. Initial (I1, I2 and I3), control (C1, C2 and C3) and late test (T4, T5 and T6) frames showed classification rates at chance levels (around 50%) in all cases. The same procedure was repeated in the other pass bands, according to the filters designed in step 3, but the results did not show statistical significance.
The ANOVA analysis over the 12 groups (one for each frame) revealed a p = 4.3299e−015 which demonstrated that at least one group was significantly different from the others. Bottom panel of Fig. 3 show the results of the multiple comparison test in order to find which means are significantly higher.
The mean classification of all subjects and blocks for patterns extracted from T1, T2 and T3 frames were significantly higher than the mean classification rates for the other frames (p = 0.01). There was no significant difference between the mean classification values of T1, T2 and T3. Table S1 shows the classification rate of each subject and block independently in the 15-22 Hz band and the corresponding threshold values. Fig. 4 shows the spatial maps of the centroid of the features extracted from T2 frames; the areas of high activity were similar for each stimulus class but the overall pattern over the 64 electrode was different.
The contribution of each electrode to the patterns classification was explored by deleting each channel or group of channels for every subject and block and then re-computing the crossclassification analysis for T1, T2 and T3 features. The results for the T1, T2 and T3 vector were qualitatively similar. The classification rate after each individual channel was removed remained within the 95% confidence limit on all subject and blocks. When groups of channels were removed, the mean classification rate dropped below the 95% confidence limit in only some subjects, when more than 8 channels were removed (Fig. 5). Classification rate decreased in proportion to the increasing number of channels were removed.
When the signal was re-referenced to the frontal position, the maximal and minimal values of analytic power changed from the frontal position to the occipital position. Phase resetting was also dependent on the reference (Ruiz et al., 2009b) so the frame locations were also slightly different. However the classification rates were similar for the two references using their respective frames located by the optimization process. Fig. 6 shows an example of the centroid pattern of the analytic power of the features extracted from T2 frames using the two references; the classification rate achieved in both cases was similar, at 71.42% for the original reference and 72.31% for the frontal reference. Fig. 7 shows the effect on the analytic phase of changing the reference from the occipital location to a frontal electrode. Some channels in the near vicinity of the reference electrode gave phase values that were 180 • (rad) out of phase with the spatial average, but that were in phase prior to the change in reference. This result emphasizes no channel lies outside the fields to be classified, but that the location of the reference has no significant effect on the multivariate classification. The phase reversal indicates that the signal oscillation is lower than the reference oscillation. The power from the reference oscillation is the same on every channel, and it is removed by the normalization procedure in step 9.

Discussion
Both theory and experimental findings have led to the concept that the brain is a self-organized system, which continuously reorganizes its activity under the influence of internal and external stimuli (Lehmann et al., 1998;Ohl et al., 2000; , 2004;Fingelkurts and Fingelkurts, 2004;Freeman and Holmes, 2005;Freeman, 2006Freeman, , 2007Kelso and Tognoli, 2007). Previous work on ECoG data has demonstrated that stable states or frames, carrying AM patterns related to cognition, emerge after sudden jumps in cortical activity called state transitions (Freeman and Van Dijk, 1987;Barrie et al., 1996;Ohl et al., 2000;Freeman, 2006;Panagiotides et al., 2008). Each state transition begins with an abrupt change in phase, followed by synchronization at a new frequency and the stabilization of a new AM pattern. Abrupt phase resetting has also been observed in the scalp EEG (Freeman et al., 2003a;Ruiz et al., 2009b), and although these phase resettings were not simultaneous over large numbers of channels, they were clustered in time, suggesting that the phase discontinuities necessary for the emergence of brain activity patterns related to cognition can also be studied using EEG signals, despite the difference in scale, corruption by electromyographic noise, distortions by gyrification, distance, etc. The phase resettings are independent of the central frequency of the temporal filter, but dependent on the bandwidth of the filter in a proportion close to 0.641 as predicted by Rice (1944). The selection of the optimal temporal filter is a key point to extract the patterns. In our case the favored bandwidth was 7 Hz covering the 15-22 Hz. According to the proportionality claim by Rice (1944) the bandwidth of 7 Hz correspond to a modal frequency of 4.5, demonstrating that the repetition rate of the frames is on the theta range (Freeman et al., 2003a;Freeman, 2006Freeman, , 2009. During the optimization procedure other bandwidth from 5 Hz to 12 Hz were studied but the results did not show statistical significance.
In the present work, a classification-directed optimization procedure was used to locate frames carrying stimulus-related information within the frequency spectrum and in time. In the 15-22 Hz band the AM patterns linked to the frames which showed up shortly after the stimulus onset became reorganized in a new pattern carrying enough stimulus-related information to achieve a classification rate above chance level. At the same time, the classification rate of frames in the pre-stimulus period or late poststimulus period was not statistically different from chance.
Despite the common finding that perception is associated with gamma band oscillations (Tallon-Baudry et al., 1998;Miltner et al., 1999;Bertrand and Tallon-Baudry, 2000;Lachaux et al., 2005;Wyart and Tallon-Baudry, 2008;Luo et al., 2009), in our results only the beta 15-22 Hz band provided spatial patterns allowing significant classification of the data as following either a Red-Loud stimulus or a Blue-Soft stimulus. The lack of classification in the gamma band may be due to contamination of the present data by scalp muscle activity at frequencies above 20 Hz (Whitham et al., 2007). This suggestion is supported by the fact that previous work using intra-cranial data showed significant classification in both beta and gamma ranges (Barrie et al., 1996;Freeman, 2005Freeman, , 2006, taking advantage of the greater spatial resolution of ECoG signal, and the lack of contamination by electromyography noise (Freeman et al., 2003c;Goncharova et al., 2003;Whitham et al., 2007).
The classification rates achieved in the present study were comparable to those for ECoG (Freeman, 2005(Freeman, , 2006, but not as high as in previous works on EEG (Ruiz et al., 2009a). The differences can be attributed in part to differences in experimental design. In previous work, all subjects were exposed to either a single stimulus in order to familiarize them to that stimulus, or two stimuli in order to perform a discrimination task (Ingber, 1999). In the present study,  combinations of two stimulus modalities, auditory and visual, were used, on the grounds that such stimuli would engage more association areas of the brain, as revealed by the work of Dumenko (2000).
The original experiment for which the data studied here were collected (Pockett et al., 2009) was designed to investigate trial and error learning-in particular, to look for any perceptual or cognitive differences that might emerge when subjects experienced the 'aha' realization of which key should be pressed in response to each bimodal stimulus. In the present study we limited our search to the EEG patterns directly following two of the stimulus Fig. 7. Example of the change in analytic phase with the shift of the reference lead in referential recording of EEG from an occipital location to a frontal location (as in Fig. 6). White is in phase with the spatial average; black is 180 • (rad) out of phase. Data from Block1 of subject F, 15-22 Hz band, the other subjects and block showed similar results. types, and found no difference in classification rates between preinsight data (Block1) and post-insight data (Block2). This suggests that there was no major change in perception of the stimuli after insight. However, trial and error learning is likely to be an internally driven, cognitive process rather than an externally driven, perceptual one and it remains possible that later patterns, perhaps those immediately preceding responses rather than those immediately following stimuli, may show differences between pre-and post-insight data.
Overall, given the similarity of the stimulus pairs studied here, the remarkable finding was that scalp EEG data could be classified with respect to stimulus type at all. The second remarkable result was that all electrodes contributed information to the spatial patterns which served to classify the EEG epochs with respect to the stimuli class, regardless of amplitude or variance. No one channel's data were vital, but the deletion of an increasing number of channels impaired the classification rate, demonstrating that the classificatory information density was uniform all over the electrode array. Our finding is consistent with the evidence from widespread intermittent synchronization of ECoG patterns in rabbits and cats (Freeman and Burke, 2003;Freeman and Rogers, 2003) and intermittent synchronization of EEG patterns from a 1D array extending over 189 mm of the scalp (Freeman et al., 2003a,c).
Our results showed that AM patterns of brain activity that are classified well above chance levels can also be extracted from the scalp EEG signal. The methods include spectral localization by narrow band-pass filtering over a range of center frequencies; the Hilbert transform to calculate the analytic phase for temporal localization; and multivariate pattern classification of the analytic amplitudes as feature vectors for spatial localization. Our algorithm could provide new data inexpensively and non-invasively for modeling the global cerebral dynamics of learning, and it may enable new advances in brain-computer interfacing. Our results also highlight the major impediment to further advances, which is the scalp EMG. Some preliminary results (Freeman et al., 2010) suggest that scalp EMG can be attenuated by low-pass spatial filtering, using presently available arrays of 256 electrodes (Poolman et al., 2008) and foreseeable arrays with exceptionally high density of recording. Sampling rates of 5000/s or more will also be necessary to measure adequately the analytic phase of gamma frequencies in the large number of EEG signals.