Use of eye tracking improves the detection of evoked responses to complex visual stimuli during EEG in infants.
ABSTRACT: To improve the reliability of detecting EEG responses evoked by complex visual stimuli to the level required for clinical use by integrating an eye tracker to the EEG setup and optimizing the analysis protocol.Infants were presented with continuous orientation reversal (OR), global form (GF), and global motion (GM) stimuli. Eye tracking was used to control stimulus presentation and exclude epochs with disoriented gaze. The spectral responses were estimated from 13 postcentral EEG channels using a circular variant of Hotelling's T2 test statistic.Among 39 healthy infants, statistically significant (p < 0.01) responses to OR/GF/GM stimuli were found from 92%/100%/95% recordings, respectively. The specificity test of the detection algorithm, using non-stimulated baseline EEG, did not yield any false-positive findings. Taken together, this yields 15% improvement on average in the detection performance compared to that in the current literature.Changes to the test protocol and incorporation of the eye tracking information improves the detection of responses to complex visual stimuli in infants.This work presents a test protocol suitable for use in a clinical environment at a level of reliability that allows individual diagnostics.
Project description:In Parkinson's disease, a precursor phenomenon to visual hallucinations presents as 'pareidolias' which make ambiguous forms appear meaningful. To evoke and detect pareidolias in patients, a noise pareidolia test was recently developed, although its task-dependent mechanisms are yet to be revealed. When subjected to this test, we hypothesized that patients exhibiting pareidolias would show altered top-down influence of visual processing allowing us to demonstrate the influence of pareidolic illusionary behaviour in Parkinson's disease patients. To that end, we evaluated eye-movement strategies and fixation-related presaccadic activity on scalp EEG when participants performed the test. Twelve healthy controls and 21 Parkinson's disease patients, evaluated for cognitive, visuo-spatial and executive functions, took a modified computer-based version of the noise pareidolia test in a free-viewing EEG eye-tracking experiment. Eye-tracking metrics (fixation-related durations and counts) documented the eye movement behaviour employed in correct responses (face/noise) and misperceptions (pareidolia/missed) during early and late visual search conditions. Simultaneously, EEG recorded the presaccadic activity in frontal and parietal areas of the brain. Based on the noise pareidolia test scores, we found certain Parkinson's disease patients exhibited pareidolias whereas others did not. ANOVA on eye-tracking data showed that patients dwelled significantly longer to detect faces and pareidolias which affected both global and local search dynamics depending on their visuo-perceptual status. Presaccadic activity in parietal electrodes for the groups was positive for faces and pareidolias, and negative for noise, though these results depended mainly on saccade size. However, patients sensitive to pareidolias showed a significantly higher presaccadic potential on frontal electrodes independent of saccade sizes, suggesting a stronger frontal activation for pareidolic stimuli. We concluded with the following interpretations (i) the noise pareidolia test specifically characterizes visuo-perceptual inadequacies in patients despite their wide range of cognitive scores, (ii) Parkinson's disease patients dwell longer to converge attention to pareidolic stimuli due to abnormal saccade generation proportional to their visuo-perceptual deficit during early search, and during late search, due to time-independent alteration of visual attentional network and (iii) patients with pareidolias show increased frontal activation reflecting the allocation of attention to irrelevant targets that express the pareidolic phenomenon. While the disease per se alters the visuo-perceptual and oculomotor dynamics, pareidolias occur in Parkinson's disease due to an abnormal top-down modulation of visual processing that affects visual attention and guidance to ambiguous stimuli.
Project description:Although the mechanisms of steady-state visual evoked potentials (SSVEPs) have been well studied, none of them have been implemented with strictly experimental conditions. Our objective was to create an ideal observer condition to exploit the features of SSVEPs. We present here an electroencephalographic (EEG) eye tracking experimental paradigm that provides biofeedback for gaze restriction during the visual stimulation. Specifically, we designed an EEG eye tracking synchronous data recording system for successful trial selection. Forty-six periodic flickers within a visual field of 11.5° were successively presented to evoke SSVEP responses, and online biofeedback based on an eye tracker was provided for gaze restriction. For eight participants, SSVEP responses in the visual field and topographic maps from full-brain EEG were plotted and analyzed. The experimental results indicated that the optimal visual flicking arrangement to boost SSVEPs should include the features of circular stimuli within a 4-6° spatial distance and increased stimulus area below the fixation point. These findings provide a basis for determining stimulus parameters for neural engineering studies, e.g. SSVEP-based brain-computer interface (BCI) designs. The proposed experimental paradigm could also provide a precise framework for future SSVEP-related studies.
Project description:The ability to detect users' emotions for the purpose of emotion engineering is currently one of the main endeavors of machine learning in affective computing. Among the more common approaches to emotion detection are methods that rely on electroencephalography (EEG), facial image processing and speech inflections. Although eye-tracking is fast in becoming one of the most commonly used sensor modalities in affective computing, it is still a relatively new approach for emotion detection, especially when it is used exclusively. In this survey paper, we present a review on emotion recognition using eye-tracking technology, including a brief introductory background on emotion modeling, eye-tracking devices and approaches, emotion stimulation methods, the emotional-relevant features extractable from eye-tracking data, and most importantly, a categorical summary and taxonomy of the current literature which relates to emotion recognition using eye-tracking. This review concludes with a discussion on the current open research problems and prospective future research directions that will be beneficial for expanding the body of knowledge in emotion detection using eye-tracking as the primary sensor modality.
Project description:Eye movements alter visual perceptions in a number of ways. During smooth-pursuit eye movements, previous studies reported decreased detection threshold for colored stimuli and for high-spatial-frequency luminance stimuli, suggesting a boost in the parvocellular system. The present study investigated the underlying neural mechanism using EEG in human participants. Participants followed a moving target with smooth-pursuit eye movements while steady-state visually evoked potentials (SSVEPs) were elicited by equiluminant red-green flickering gratings in the background. SSVEP responses to colored gratings were 18.9% higher during smooth pursuit than during fixation. There was no enhancement of SSVEPs by smooth pursuit when the flickering grating was defined by luminance instead of color. This result provides physiological evidence that the chromatic response in the visual system is boosted by the execution of smooth-pursuit eye movements in humans. Because the response improvement is thought to be the result of an improved response in the parvocellular system, SSVEPs to equiluminant stimuli could provide a direct test of parvocellular signaling, especially in populations where collecting an explicit behavioral response from the participant is not feasible.NEW & NOTEWORTHY We constantly move our eyes when we explore the world. Eye movements alter visual perception in various ways. The smooth-pursuit eye movements have been shown to boost color sensitivity. We recorded steady-state visually evoked potentials to equiluminant chromatic flickering stimuli and observed increased steady-state visually evoked potentials when participants smoothly pursued a moving target compared with when they maintained fixation. This work provides direct neurophysiological evidence for the parvocellular boost by smooth-pursuit eye movements in humans.
Project description:<h4>Purpose</h4>We determined the latencies of orienting responses during a preferential looking task in children with normal vision and in children with visual impairments between 6 and 12 years old, and assessed the feasibility of scoring grating detection in these populations with video-based eye tracking.<h4>Methods</h4>Children performed a computerized preferential looking test, while a remote eye tracker measured the children's eye movements. The stimuli consisted of a 2 × 2 grid, with three uniform grey fields and one target field consisting of a black-and-white square wave grating. The grating was presented randomly at one of the four locations. The spatial frequencies (1.05, 2.11 and 7.02 cyc/deg) were randomly interleaved, with 10 trials per spatial frequency. Three different methods were used to score the accuracy of the responses: (1) primary saccade ends on target, (1) gaze 50% of the presentation time on target, and (3) a combination of method 1 and 2 (i.e. primary saccade ends on target, and/or gaze 50% of the presentation time on target).<h4>Results</h4>The combined scoring method was most reliable to determine whether children could resolve the gratings. Children with visual impairments had significantly lower accuracies than children with normal vision with all three scoring methods. In addition, saccade latencies decreased with age and were significantly longer (62 ± 15 ms) in children with visual impairments.<h4>Conclusion</h4>The use of eye tracking to assess grating detection with a preferential looking task in clinical populations provides valuable additional information, including objective detection measures and developmental delays in saccade latencies.
Project description:Eye movements are an important index of the neural functions of visual information processing, decision making, visuomotor coordination, sports performance, and so forth. However, the available optical tracking methods are impractical in many situations, such as the wearing of eyeglasses or the presence of ophthalmic disorders, and this can be overcome by accurate recording of eye movements by electrooculography (EOG). In this study we recorded eye movements by EOG simultaneously with high-density electroencephalogram (EEG) recording using a 128-channel EGI electrode net at a 500-Hz sampling rate, including appropriate facial electrodes. The participants made eye movements over a calibration target consisting of a 5×5 grid of stimulus targets. The results showed that the EOG methodology allowed accurate analysis of the amplitude and direction of the fixation locations and saccadic dynamics with a temporal resolution of 500 Hz, under both cued and uncued analysis regimes. Blink responses could be identified separately and were shown to have a more complex source derivation than has previously been recognized. The results also showed that the EOG signals recorded through the EEG net can achieve results as accurate as typical optical eye-tracking devices, and also allow for simultaneous assessment of neural activity during all types of eye movements. Moreover, the EOG method effectively avoids the technical difficulties related to eye-tracker positioning and the synchronization between EEG and eye movements. We showed that simultaneous EOG/EEG recording is a convenient means of measuring eye movements, with an accuracy comparable to that of many specialized eye-tracking systems.
Project description:<h4>Background</h4>Eye-tracking technology has indicated that daily smokers actively avoid pictorial cigarette package health warnings. Avoidance may be due to a pre-cognitive perceptual bias or a higher order cognitive bias, such as reduced emotional processing. Using electroencephalography (EEG), this study aimed to identify the temporal point at which smokers' responses to health warnings begin to differ.<h4>Method</h4>Non-smokers (n=20) and daily smokers (n=20) viewed pictorial cigarette package health warnings and neutral control stimuli. These elicited Event Related Potentials reflecting early perceptual processing (visual P1), pre-attentive change detection (visual Mismatch Negativity), selective attentional orientation (P3) and a measure of emotional processing, the Late Positive Potential (LPP).<h4>Results</h4>There was no evidence for a difference in P1 responses between smokers and non-smokers. There was no difference in vMMN and P3 amplitude but some evidence for a delay in vMMN latency amongst smokers. There was strong evidence for delayed and reduced LPP to health warning stimuli amongst smokers compared to non-smokers.<h4>Conclusion</h4>We find no evidence for an early perceptual bias in smokers' visual perception of health warnings but strong evidence that smokers are less sensitive to the emotional content of cigarette health warnings. Future health warning development should focus on increasing the emotional salience of pictorial health warning content amongst smokers.
Project description:Electroencephalography (EEG) records fast-changing neuronal signalling and communication and thus can offer a deep understanding of cognitive processes. However, traditional data analyses which employ the Fast-Fourier Transform (FFT) have been of limited use as they do not allow time- and frequency-resolved tracking of brain activity and detection of directional connectivity. Here, we applied advanced qEEG tools using autoregressive (AR) modelling, alongside traditional approaches, to murine data sets from common research scenarios: (a) the effect of age on resting EEG; (b) drug actions on non-rapid eye movement (NREM) sleep EEG (pharmaco-EEG); and (c) dynamic EEG profiles during correct vs incorrect spontaneous alternation responses in the Y-maze. AR analyses of short data strips reliably detected age- and drug-induced spectral EEG changes, while renormalized partial directed coherence (rPDC) reported direction- and time-resolved connectivity dynamics in mice. Our approach allows for the first time inference of behaviour- and stage-dependent data in a time- and frequency-resolved manner, and offers insights into brain networks that underlie working memory processing beyond what can be achieved with traditional methods.
Project description:We present the Zurich Cognitive Language Processing Corpus (ZuCo), a dataset combining electroencephalography (EEG) and eye-tracking recordings from subjects reading natural sentences. ZuCo includes high-density EEG and eye-tracking data of 12 healthy adult native English speakers, each reading natural English text for 4-6?hours. The recordings span two normal reading tasks and one task-specific reading task, resulting in a dataset that encompasses EEG and eye-tracking data of 21,629 words in 1107 sentences and 154,173 fixations. We believe that this dataset represents a valuable resource for natural language processing (NLP). The EEG and eye-tracking signals lend themselves to train improved machine-learning models for various tasks, in particular for information extraction tasks such as entity and relation extraction and sentiment analysis. Moreover, this dataset is useful for advancing research into the human reading and language understanding process at the level of brain activity and eye-movement.
Project description:In the premature infant, somatosensory and visual stimuli trigger an immature electroencephalographic (EEG) pattern, "delta-brushes," in the corresponding sensory cortical areas. Whether auditory stimuli evoke delta-brushes in the premature auditory cortex has not been reported. Here, responses to auditory stimuli were studied in 46 premature infants without neurologic risk aged 31 to 38 postmenstrual weeks (PMW) during routine EEG recording. Stimuli consisted of either low-volume technogenic "clicks" near the background noise level of the neonatal care unit, or a human voice at conversational sound level. Stimuli were administrated pseudo-randomly during quiet and active sleep. In another protocol, the cortical response to a composite stimulus ("click" and voice) was manually triggered during EEG hypoactive periods of quiet sleep. Cortical responses were analyzed by event detection, power frequency analysis and stimulus locked averaging. Before 34 PMW, both voice and "click" stimuli evoked cortical responses with similar frequency-power topographic characteristics, namely a temporal negative slow-wave and rapid oscillations similar to spontaneous delta-brushes. Responses to composite stimuli also showed a maximal frequency-power increase in temporal areas before 35 PMW. From 34 PMW the topography of responses in quiet sleep was different for "click" and voice stimuli: responses to "clicks" became diffuse but responses to voice remained limited to temporal areas. After the age of 35 PMW auditory evoked delta-brushes progressively disappeared and were replaced by a low amplitude response in the same location. Our data show that auditory stimuli mimicking ambient sounds efficiently evoke delta-brushes in temporal areas in the premature infant before 35 PMW. Along with findings in other sensory modalities (visual and somatosensory), these findings suggest that sensory driven delta-brushes represent a ubiquitous feature of the human sensory cortex during fetal stages and provide a potential test of functional cortical maturation during fetal development.