{"coupon":"","discount":"","doc_name":"Part of Ph.D. Dissertation","doc_id":67175,"editorId":"144","email":"stahn@gist.ac.kr","message":"Overview<br \/><br \/>Brain-computer interface (BCI) has gained great attention over the past several decades. It can give a new communication between human and brain without any physical movement. Electroencephalography (EEG) based BCI is the most widely used due to its fine temporal resolution, ease of use, portability and low set-up cost. EEG is the recording of electrical activity along the scalp and measures voltage fluctuations resulting from ionic current flows within the neurons of the brain. The electric potentials generated by single neurons are far too small to be picked by EEG. EEG activity therefore always reflects the summation of the synchronous activity of thousands or millions of neurons that have similar spatial orientation. If the cells do not have similar spatial orientation, their ions do not line up and create waves to be detected. Pyramidal neurons of the cortex are thought to produce the most EEG signal because they are well-aligned and fire together. Because voltage fields fall off with the square of distance, activity from deep sources is more difficult to detect than currents near the skull. <br \/>In this dissertation, I attempted to find the possibility for reducing ill-iterate subjects by incorporating two different EEG features based on motor imager and tactile selective attention tasks as a novel hybrid BCI. Further, I tried to find neuro-physiological correlates using EEG and hemodynamic changes from functional near-infrared spectroscopy (fNIRS) in two different. fNIRS is an emerging portable and noninvasive brain functional imaging technique and its combination can identify the workload or mental fatigue. Therefore, correlates of two different simulated driving conditions were estimated by EEG and fNIRS techniques.<br \/>Motivation<br \/>To solve the problems of conventional BCIs, many novel approaches have been used to BCI as mentioned above. Among them, hybrid BCI is one of the possible candidates and visual-based hybrid BCIs are well-known and widely used approaches [7,15]. However, one major drawback of visual methods is that the subjects need voluntary gaze control. Patients that suffer from locked-in syndrome (LIS), or are in late stages of amyotrophic lateral sclerosis (ALS), lose their volitional eye control. <br \/>First of all, I have deeply investigated the hybrid BCI and developed a novel hybrid BCI to reduce ill-iterate subjects. In addition to use of two EEG features, I extend to other neuroimaging technique using fNIRS. By measuring the hemodynamic changes of the brain using fNIRS system and electrical activity using EEG, I attempted to observe driving fatigue of two different driving conditions (well-rested and sleep-deprived).<br \/>Outline of the dissertation<br \/>In this dissertation, introduction and current status of BCI will be described in Chapter 2. It includes the general concept and representative examples of BCI systems. <br \/>In Chapter 3, I will describe the hybrid BCI deeply. It contains the origin, concept and possible examples of the hybrid BCI and explains advantages and disadvantages of current hybrid BCI. Several combinations are possible to establish the hybrid BCI and some remarkable combinations will be described. <br \/>Chapter 4 describes our novel hybrid BCI paradigms and improved results. By combining two different EEG features, I found out the interaction of two features and yielded the reducing ill-iterate subjects compared to conventional BCI paradigm. Further, the reasons of improved results will be explained and I will suggest the direction of future hybrid BCI paradigms with tactile features.<br \/>Chapter 5 contains attempting to electrical activity using EEG and hemodynamic changes using fNIRS. Features of EEG and fNIRS were investigated individually and their features were combined to detect the driving fatigue of two different conditions. This study can give us to combining EEG and fNIRS is possible hybrid approach.<br \/>Chapter 6 discusses our findings and summarized all works. Further, future works will be explained for better system.<br \/><br \/><br \/>Brain-computer interface (BCI) <br \/>1.1 Basic principle of BCI<br \/>In general, EEG was firstly found by Hans Berger who was German psychiatrist and EEG-based BCI was firstly introduced in 1970s [23]. BCI is one of the fascinating technique because it gives an opportunity for controlling specific objects without any limb movement for paralyzed patients. In addition to patients, healthy subjects can also easily use BCI system as a new communication method. Basic procedure of a BCI system is shown in Figure 2.1. First of all, EEG signal was obtained from subject&rsquo;s scalp (signal acquisition). Analog-to-digital converter digitized the analog to digital signal in order to input to computer. In computer, EEG signal was pre-processed and extracted as feature set. From feature set, various pattern recognition methods were applied to signal to decode subject&rsquo;s intention. Finally, decoded control signal was transmitted to external devices such wheelchair or robot. In online BCI system, subjects can receive real-time feedback about movement of devices or their own brain signals to be decoded. This kind of loop system was repeated continuously to establish the closed-loop BCI system. In addition to EEG, other neuroimaging techniques such as fNIRS can be applied to this procedure.<br \/> <br \/><br \/>1.2 Limitations and future hybrid BCIs<br \/>Most of hybrid BCI studies have argued that combination of two or more systems can yield better performance than one and it leads future BCI systems. However, there are some limitations of hybrid BCIs. Combining two or more features does not guarantee for establishing better system than previous one. Sometimes features of one disrupted the other features and the classification performances were decreased when two different feature set were combined even in classifiers. As the number of measurements goes by, it took more time to prepare, set-up and recording. Subjects who participated in experiment could feel uncomfortableness due to long preparation time and cumbersome measurement devices. An advanced technology is that the more simpler measurement the more the best measurement for users. For these reasons, hybrid BCIs achieved better performance but sometimes users could feel more uncomfortable. <br \/>In future hybrid BCIs, systems have to be simpler and more intuitive to use. Because BCI technology eventually is for users who do not have arm or cannot move their arms. User-friendly hybrid BCIs have to be developed for advanced future hybrid BCIs. <br \/>1.3 Current status of hybrid BCI<br \/>Although hybrid BCI gives us to overcome the problems of conventional BCIs, there still exist some obstacles in current hybrid BCIs. Most of the researchers in BCI field have focused on improvement of classification accuracy. In my opinion, only improvement of classification accuracy does not solve with current problems of BCIs. Intendix (public P300 speller) was developed in 2009 and it makes BCI comes a new technology in near future. However, nowdays eye tracker with high information transfer rate (ITR) is the most fascinating technology for patients who can control their eyes naturally. This eye tracker system outperformed the current advanced BCIs in terms of ITR, reliability, ease of use and accuracy. To make BCI effectively for patients or healthy person, it must have a specific strong advantage compared to eye tracker. For example, preparation time of EEG setup must be reduced with channel optimization. Attaching 30 channels on the scalp is quite cumbersome task and it bothers with subjects and experimenter. Let&rsquo;s imagine that only 1 channel (preparation for just 1 minute) is enough to extract the optimized feature of EEG. Improving the accuracy and reliability is a meaningful try but we have to focus on ease of use. Although hybrid BCIs can reduce the ill-iterate subjects and improve the accuracy its long preparation time and inconvenience due to other devices makes more uncomfortable.<br \/>1.4 Development of novel paradigm and transfer<br \/>All of the BCI experimental paradigms are almost similar. Subjects sat in comfortable chair and waited for instruction of screen. After instruction, a specific task is started for about 5 seconds and it was repeated about 100 times to make it statistically significant. In conventional BCI paradigms, most of the subjects can feel drowsiness and cannot consistently concentrate on task for 2 or 3 hours. It is very boring condition and no one want to sat in chair for 3 hours without saying anything. For this reason, we have to make subjects concentrate easily and effectively with short time duration. According to the perceptual load theory [168], when a subject is fully attentive using full capacity in relevant attention processing, there may be no spare capacity for perception of distracting interference (high perceptual load). On the other hand, any capacity not allocated to the relevant attention processing may involuntarily spill over to the perception of distracting interference (low perceptual load). This hypothesis may be applicable not only to the psychology experiment, but to the BCI experiment as well. First one establishing effective BCI experiment is that experimental paradigms make subjects concentrate well. Second one is that whole experiment time must be reduced. BCI needs long training time to make a classifier and generate control signal. Recently, BCI with zero-training using short noise data was introduced [147] in terms of session to session and its methodological approach is very useful in public BCIs. In addition to sessions, subject to subject transfer BCI is the most useful BCI for future BCI. This approach is still under investigation by several BCI researchers [169,170] and I strongly suggest BCI becomes a new real future technology if subject to subject transfer is possible to all subjects.<br \/>1.5 Toward a brain-brain interaction<br \/>BCI generally means the communication brain and computer with only brain signal. Recently, several researchers have investigated inter-brain synchronization beyond single brain [171]. Hyper-scanning technique [172] enables to study with multiple brains simultaneously and its novel findings were very impressive. For this reason, inter-brain synchronization in social interaction using hyper-scanning technique becomes remarkable research field in the brain research. In hyper-scanning studies, brain activities of two or more subjects were recorded by EEG, MEG, fNIRS and fMRI and subjects performed same tasks simultaneously or turn-taking tasks. Several studies related to social interaction have yielded the possibility of brain-brain interaction using only their brains. Beyond single BCI, multi-subject BCI and brain-brain interface is now fascinating future technology [173].<br \/>However, current problem of hyper-scanning studies is that there is no standard experiment paradigm such as conventional motor imagery. Actually, scope of social interaction is very wide speared. For example, auditory, gesture, visual, synchronized movement or other interacting situations are possible for studying inter-brain synchronization. As a result, findings of these studies varies on experimental paradigms. For example, some investigations yielded frontal alpha suppression but other studies could not find this phenomenon. These variations of findings must be solved by occurrence of standard social interaction paradigm. Although inter-brain synchronization using hyper-scanning technique is very appealing study in the brain study, there are many critical problems that must be solved as soon as possible.<br \/><br \/><br \/>1.6 Summary of dissertation and future works<br \/>Studies of BCI now well-known and widely investigated by numerous researcher in the world. However, there still exist critical problems of BCI and its solutions were actively studied such as hybrid, novel paradigm or advanced machine learning approaches. In my dissertation, therefore, I attempted to solve the problems of conventional BCIs by hybrid BCI with motor imagery using ERD and tactile selective attention using SSSEP. As a result, I found that the number of ill-iterate subjects were decreased with some remarkable results. To investigate multimodal neuroimaging, further, combination of EEG, ECG and fNIRS was studied in a driving condition. As a result, combination approach outperformed the others and fusion hemodynamic changes and electrical activity makes BCI effective. Although my novel approaches cannot fully overcome the problems of BCIs, it must propose that the hybridizing signals can be an alternative technology for future BCIs. <br \/>So far, I have experienced about signal processing of EEG and fNIRS data. From these experiences, I have learned that how to deal with EEG signals and understood principles of other neuroimaging techniques. My future research will be focused on inter-brain synchronization using hyper-scanning technique. Social interaction plays an important role between humans and its neuro-physiological findings when we are in various interacting situations helps to establish future brain-brain interactions. To establish convincing hyper-scanning study, well-designed hardware setup and standard experimental paradigm is necessary. EEG, fNIRS and MEG can be used in this study and its different findings will be helped to understand the social interaction in real-life.<br \/><br \/>","price_select":"2","temp_doc_name":"upload_571eebff3603e7.99753398.docx","word_count":"1990","subtotal":39.8,"total_amt":39.8,"firstName":"Sangtae","lastName":"Ahn","lastfour":"7671","exp":"0317","address":"#508 Dasan Bldg.,  123 Cheomdangwagi-ro, Buk-gu, Gwagnju, South Korea","city":"Gwagnju","zip":"61005","state":"Gwagnju","country":"Korea, Republic of (South Korea)","transactionReference":"8350061914"}