Continuous Brain-Actuated Control of an Intelligent Wheelchair by Human EEG
F. Galán, M. Nuttin, D. Vanhooydonck, E. Lew, P. W. Ferrez, J. Philips, J.del R. Millán (2008)
4th International Brain-Computer Interface Workshop & Training Course , To appear.
Abstract
The objective of this study is to assess the feasibility of controlling an asynchronous and
non-invasive brain-actuated wheelchair by human EEG. Three subjects were asked to mentally
drive the wheelchair to 3 target locations using 3 mental commands. These mental commands
were respectively associated with the three wheelchair steering behaviors: turn left, turn right,
and move forward. The subjects participated in 30 randomized trials (10 trials per target).
The performance was assessed in terms of percentage of reached targets calculated in function
of the distance between the final wheelchair position and the target at each trial. To assess
the brain-actuated control achieved by the subjects, their performances were compared with
the performance achieved by a random BCI. The subjects drove the wheelchair closer than 1
meter from the target in 20%, 37%, and 7% of the trials, and closer than 2 meters in 37%,
53%, and 27% of the trials, respectively. The random BCI drove it closer than 1 and 2 meters
in 0% and 13% of the trials, respectively. The results show that the subjects could achieve a
significant level of mental control, even if far from optimal, to drive an intelligent wheelchair,
thus demonstrating the feasibility of continuously controlling complex robotics devices using
an asynchronous and non-invasive BCI.
A Brain-Actuated Wheelchair: Asynchronous and Non-Invasive Brain-Computer Interfaces for Continuous Control of Robots
F. Galán, M. Nuttin, E. Lew, P. W. Ferrez, G. Vanacker, J. Philips, J.del R. Millán (2008)
Clinical Neurophysiology , 119(9). pp. 2159-2169.
Abstract
Objective: To assess the feasibility and robustness of an asynchronous and non-invasive
EEG-based Brain-Computer Interface (BCI) for continuous mental control of a wheelchair.
Methods: In experiment 1 two subjects were asked to mentally drive both a real and a
simulated wheelchair from a starting point to a goal along a pre-specified path. Here we
only report experiments with the simulated wheelchair for which we have extensive data in
a complex environment that allows a sound analysis. Each subject participated in 5 experimental
sessions, each consisting of 10 trials. The time elapsed between two consecutive
experimental sessions was variable (from one hour to two months) to assess the system
robustness over time. The pre-specified path was divided in 7 stretches to assess the system
robustness in different contexts. To further assess the performance of the brain-actuated
wheelchair, subject 1 participated in a second experiment consisting of 10 trials where he
was asked to drive the simulated wheelchair following 10 different complex and random
paths never tried before. Results: In experiment 1 the two subjects were able to reach 100%
(subject 1) and 80% (subject 2) of the final goals along the pre-specified trajectory in their
best sessions. Different performances were obtained over time and path stretches, what indicates
that performance is time and context dependent. In experiment 2, subject 1 was
able to reach the final goal in 80% of the trials. Conclusions: The results show that subjects
can rapidly master our asynchronous EEG-based BCI to control a wheelchair. Also,
they can autonomously operate the BCI over long periods of time without the need for
adaptive algorithms externally tuned by a human operator to minimize the impact of EEG
non-stationarities. This is possible because of two key components: first, the inclusion of
a shared control system between the BCI system and the intelligent simulated wheelchair;
second, the selection of stable user-specific EEG features that maximize the separability
between the mental tasks. Significance: These results show the feasibility of continuously
controlling complex robotics devices using an asynchronous and non-invasive BCI.
R. Chavarriaga, F. Galán, J.del R. Millán (2008)
16th European Signal Processing Conference (EUSIPCO 2008), Lausanne, Switzerland, August 2008.
Abstract
The characterization and recognition of electrical signatures of brain activity constitutes a real challenge. Applications such as
Brain-Computer Interfaces (BCI) are based on the accurate identification of mental processes in order to control external devices.
Traditionally, classification of brain electrical patterns relies on the assumption that the neurological phenomena that characterize
mental states is continously present in the signal. However, recent evidence shows that some mental processes are better characterized
by episodic activity that is not necessarily synchronized with external stimuli.
In this paper, we present a method for classification of mental states based on the detection of this episodic activity. Instead of perform
classification on all available data, the proposed method identifies informative samples based on the class sample distribution in a projected
canonical feature space. Classification results are compared to traditional methods using both artificial and EEG recordings.
N. Bourdaud, R. Chavarriaga, F. Galán, J.del R. Millán (2008)
IEEE Transactions on Neural Systems and Rehabilitation Engineering , To appear.
Abstract
This study aims to characterize the EEG correlates of exploratory behavior. Decision making in an uncertain
environment raises a conflict between two opposing needs: gathering information about the environment and exploiting
this knowledge in order to optimize the decision. Exploratory behavior has already been studied using fMRI. Based
on a usual paradigm in reinforcement learning, this study has shown bilateral activation in the frontal and parietal
cortex. Up to our knowledge, no previous study has been done on it using EEG.
The study of the exploratory behavior using EEG signals raises two difficulties. First, the labels of trial as
exploitation or exploration cannot be directly derived from the subject action. In order to access this information, a
model of how the subject makes his decision must be built. The exploration related information will be then derived
from it. Second, because of the complexity of the task, its EEG correlates are not necessarily time locked with the
action. So the EEG processing methods used should be designed in order to handle signals that shift in time across
trials.
Using the same experiment protocol than the fMRI study, results show that the bilateral frontal and parietal areas
are also the most discriminant which strongly suggests that the EEG signal conveys also the information about the
exploratory behavior.
Non-Invasive Brain-Machine Interaction
J.del R. Millán, P. W. Ferrez, F. Galán, E. Lew, and R. Chavarriaga (2008)
International Journal of Pattern Recognition and Artificial Intelligence , To appear.
Abstract
The promise of Brain-Computer Interfaces (BCI) technology is to augment human capabilities by
enabling interaction with computers through a conscious and spontaneous modulation of the
groups have designed brain-actuated devices that provide alternative channels for communication,
entertainment and control. Thus, a person can write messages using a virtual keyboard on a
computer screen and also browse the internet. Alternatively, subjects can operate simple computer
games, or brain games, and interact with educational software. Work with humans has shown that it
is possible for them to move a cursor and even to drive a wheelchair. This paper briefly reviews the
field of BCI, with a focus on non-invasive systems based on electroencephalogram (EEG) signals. It
also describes three brain-actuated devices we have developed: a virtual keyboard, a brain game, and
a wheelchair. Finally, it shortly discusses current research directions we are pursuing in order to
improve the performance and robustness of our BCI system, especially for real-time control of brainactuated
robots.
The Use of Brain-Computer Interfacing for Ambient Intelligence
G. Garipelli, F. Galán, R. Chavarriaga, P. W. Ferrez, E. Lew, and J.del R. Millán (2007)
First International Workshop on Human Aspects in Ambient Intelligence ( AmI-07), Darmstadt, Germany, Nov 2007.
Abstract
This paper is aimed to introduce IDIAP Brain Computer Interface
(IBCI) research that successfully applied Ambience Intelligence
(AmI) principles in designing intelligent brain-machine interactions. We
proceed through IBCI projects describing how machines can read and
react to the human mental commands, cognitive and emotive states.
We show how effective human-machine interaction for brain computer
interfacing (BCI) can be achieved through, 1) asynchronous and spontaneous
BCI, 2) shared control between the human and machine, 3) online
learning and 4) the use of cognitive state recognition to improve BCI performance.
Identifying common principles in BCI research and ambiance
intelligence (AmI) research, we discuss IBCI applications in the light of
AmI. With the current studies on recognition of human cognitive states,
we argue for the possibility of designing empathic environments or devices
that have a better human like understanding directly from brain
signals.
Visuo-spatial Attention Frame Recognition for Brain-computer Interfaces
F. Galán, J. Palix, R. Chavarriaga, P. W. Ferrez, C.A. Hauert, and J.del R. Millán (2007)
Symp. on Adv. Signal Processing Techniques for Brain Data Analysis ( ICCN 2007), Shanghai, China, Nov 2007.
Abstract
Objective: To assess the feasibility of recognizing visual
spatial attention frames for Brain-computer interfaces (BCI)
applications. Methods: EEG data was recorded with 64
electrodes from 2 subjects executing a visual spatial attention
task indicating 2 target locations. Continuous Morlet wavelet
coefficients were estimated on 18 frequency components and 16
preselected electrodes in trials of 600 ms. The spatial patterns
of the 16 frequency components frames were simultaneously detected
and classified (between the two targets). The classification
accuracy was assessed using 20-fold crossvalidation.
Results: The maximum frames average classification
accuracies are 80.64% and 87.31% for subject 1 and 2
respectively, both utilizing coefficients estimated at frequencies
located in gamma band.
An Asynchronous and Non-Invasive Brain-Actuated Wheelchair
F. Galán,, M. Nuttin, E. Lew, P. W. Ferrez, G. Vanacker, J. Philips, H. Van Brussel, and J. del. R. Millán (2007)
13th International Symposium on Robotics Research (ISRR 2007), Hiroshima, Japan, Nov 2007.
Abstract
Objective: To develop a robust asynchronous and
non-invasive brain-computer interface (BCI) for brain-actuated
wheelchair driving, and to assess the system robustness over time
and context (physical environment). Methods: Two subjects
were asked to mentally drive a simulated wheelchair from a
starting point to a goal following a pre-specified path in a
simulated environment. Each subject participated in 5 experimental
sessions integrated by 10 trials each. The experimental sessions
were carried on with different elapsed times between them (since
one hour to two months) to assess the system robustness over time.
The path was divided in seven stretches to assess the robustness
over context.
Results: The two subjects were able to
reach 90% (subject 1) and 80% (subject 2) of the final goals one
day after the calibration of the BCI system, and 100% (subject 1)
and 70% (subject 2) two months later. Different performances were
obtained over the different path stretches.
Non-Invasive Brain-Actuated Interaction
J. del R. Millán, P. W. Ferrez, F. Galán, E. Lew and R. Chavarriaga (2007)
2nd Symposium on Brain, Vision and Artificial intelligence (BVAI 2007), Naples, Italy, Oct 2007.
Abstract
The promise of Brain-Computer Interfaces (BCI) technology is to augment
human capabilities by enabling interaction with computers through a conscious
and spontaneous modulation of the brainwaves after a short training period.
Indeed, by analyzing brain electrical activity online, several groups have designed
brain-actuated devices that provide alternative channels for communication, entertainment
and control. Thus, a person can write messages using a virtual keyboard on a computer
screen and also browse the internet. Alternatively, subjects can operate simple computer
games, or brain games, and interact with educational software. Work with humans has shown
that it is possible for them to move a cursor and even to drive a wheelchair. This paper
briefly reviews the field of BCI, with a focus on non-invasive systems based on
electroencephalogram (EEG) signals. It also describes three brain-actuated devices we have
developed: a virtual keyboard, a brain game, and a wheelchair. Finally, it shortly discusses
current research directions we are pursuing in order to improve the performance and robustness
of our BCI system, especially for real-time control of brain-actuated robots.
Feature Extraction for Multi-class BCI using Canonical Variates Analysis
F. Galán, P. W. Ferrez, F. Oliva, J. Guàrdia, and J. del R. Millán (2007)
IEEE International Symposium on Intelligent Signal Processing (WISP 2007), Madrid, Spain, Oct 2007.
Abstract
Objective: To propose a new feature extraction method
with canonical solution for multi-class Brain-Computer Interfaces
(BCI). The proposed method should provide a reduced number of
canonical discriminant spatial patterns (CDSP) and rank the
channels sorted by power discriminability (DP) between classes.
Methods: The feature extractor relays in Canonical Variates
Analysis (CVA) which provides the CDSP between the classes. The
number of CDSP is equal to the number of classes minus one. We
analyze EEG data recorded with 64 electrodes from 4 subjects
recorded in 20 sessions. They were asked to execute twice in each
session three different mental tasks (left hand imagination
movement, rest, and words association) during 7 seconds. A ranking
of electrodes sorted by power discriminability between classes and
the CDSP were computed. After splitting data in training and test
sets, we compared the classification accuracy achieved by Linear
Discriminant Analysis (LDA) in frequency and temporal domains.
Results: The average LDA classification accuracies over the
four subjects using CVA on both domains are equivalent (57.89% in
frequency domain and 59.43% in temporal domain). These results,
in terms of classification accuracies, are also reflected in the
similarity between the ranking of relevant channels in both
domains. Conclusions: CVA is a simple feature extractor
with canonical solution useful for multi-class BCI applications
that can work on temporal or frequency domain.
Canonical Feature Extraction and EEG Channel Ranking for Multiclass Brain-Computer Interfaces
F. Galán, P. W. Ferrez, F. Oliva, J. Guàrdia, and J. del R. Millán (2007)
Swiss Society of Biomedical Engineering 2007 Meeting (SSBE 2007), Neuchâtel, Switzerland, Sep 2007.
Abstract
Our work is focused on
development of asynchronous and non-invasive
electroencephalogram (EEG) based brain-computer
interfaces (BCI) to control robots and wheelchairs. The users drive such devices by learning to
voluntary control specific EEG features. To
facilitate this learning process it is necessary to
select those subject-specific features that allow to
generate the maximum number of discriminant
patterns. The present work proposes a new feature
extraction method with canonical solution for
multi-class BCI. The proposed method, based on
canonical variates analysis (CVA), provides a
reduced number of canonical discriminant spatial
patterns (CDSP), equal to the number of classes
minus one, and ranks the EEG channels sorted by
power discriminability (DP) between the eventrelated
(de)synchronization effects involved in
the execution of different mental tasks.
Shared-Control of a Brain-Actuated Wheelchair in Natural Environments
J. del R. Millán, F. Galán, E. Lew, P.W. Ferrez, J. Philips, G. Vanacker, and M. Nuttin (2007)
Swiss Society of Biomedical Engineering 2007 Meeting (SSBE 2007), Neuchâtel, Switzerland, Sep 2007.
Abstract
The asynchronous IDIAP
brain-computer interface (BCI) has been
recently integrated with the intelligent wheelchair
of KU Leuven to allow a person to mentally
drive it in natural environments (see Fig. 1). This
brain-actuated wheelchair incorporates advances in
adaptive shared autonomy, EEG signal processing,
and machine learning developed in the framework
of the MAIA project.
Context-based Filtering for Assisted Brain-Actuated Wheelchair Driving
G. Vanacker, J. del R. Millán, E. Lew, P. W. Ferrez, F. Galán, J. Philips, H. Van Brussel, and M. Nuttin (2007)
Computational Intelligence and Neuroscience 2007
Abstract
Controlling a robotic device by using human brain signals is an interesting and challenging task.
The device may be complicated to control and the nonstationary nature of the brain signals provides
for a rather unstable input. With the use of intelligent processing algorithms adapted to the task at
hand, however, the performance can be increased. This paper introduces a shared control system that helps
the subject in driving an intelligent wheelchair with a noninvasive brain interface. The subject's steering
intentions are estimated from electroencephalogram (EEG) signals and passed through to the shared control
system before being sent to the wheelchair motors. Experimental results show a possibility for significant
improvement in the overall driving performance when using the shared control system compared to driving without
it. These results have been obtained with 2 healthy subjects during their first day of training with the
brain-actuated wheelchair.
Adaptive Shared Control of a Brain-Actuated Simulated Wheelchair
J. Philips, J. del R. Millán, G. Vanacker, E. Lew, F. Galán, P. W. Ferrez, H. Van Brussel, and M. Nuttin (2007)
10th International Conference on Rehabilitation Robotics (ICORR 2007), Noordwijk, The Netherlands, Jun 2007.
Abstract
The use of shared control techniques has a
profound impact on the performance of a robotic assistant
controlled by human brain signals. However, this shared control
usually provides assistance to the user in a constant and
identical manner each time. Creating an adaptive level of
assistance, thereby complementing the user’s capabilities at any
moment, would be more appropriate. The better the user can
do by himself, the less assistance he receives from the shared
control system; and vice versa. In order to do this, we need
to be able to detect when and in what way the user needs
assistance. An appropriate assisting behaviour would then be
activated for the time the user requires help, thereby adapting
the level of assistance to the specific situation. This paper
presents such a system, helping a brain-computer interface
(BCI) subject perform goal-directed navigation of a simulated
wheelchair in an adaptive manner. Whenever the subject has
more difficulties in driving the wheelchair, more assistance will
be given. Experimental results of two subjects show that this
adaptive shared control increases the task performance. Also,
it shows that a subject with a lower BCI performance has
more need for extra assistance in difficult situations, such as
manoeuvring in a narrow corridor.
Using Mental Tasks Transitions Detection to Improve Spontaneous Mental Activity Classification
F. Galán, F. Oliva, and J. Guàrdia (2007)
Medical and Biological Engineering and Computing , 45(6). pp. 603-609.
Abstract
This paper presents an algorithm based on canonical variates transformation (CVT) and distance based discriminant
analysis (DBDA) combined with a mental tasks transitions detector (MTTD) to classify spontaneous mental activities
in order to operate a brain-computer interface working under an asynchronous protocol. The algorithm won the BCI
Competition III -Data Set V: Multiclass Problem, Continous EEG- achieving an averaged classification accuracy over
three subjects of 68.65% (79.60, 70.31 and 56.02%, respectively) in a three-class problem.
High Frequency Bands and Estimated Local Field Potentials to Improve Single-Trial Classification of Electroencephalographic Signals
P. W. Ferrez, F. Galán, A. Buttfield, S. L. Gonzalez, R. Grave de Peralta and J. del R. Millán (2006)
3rd International Brain-Computer Interface Workshop & Training Course, G.R. Müller-Putz, C. Brunner, R. Leeb, R. Scherer, A. Schlögl, S. Wriessnegger, & G. Pfurtscheller, Verlag der TU Graz, Graz University of Technology, Austria. Verlag der Technischen Universität Graz.
Abstract
Non-invasive brain-computer interfaces are traditionally based on mu rhythms, beta rhythms, slow cortical potentials or P300
event-related potentials. However, there is mounting evidence that neural oscillations up to 200 Hz play important roles
in processes such as attention, perception, motor action and conscious experience. In this preliminary study we propose
to extend the investigations to the complete frequency spectrum and to compare the high frequency bands with the usual
low frequencies. It appears that the 70-130 Hz band and the 170-230 Hz band performs better than the traditional 2-40 Hz
band. In a second step we applied the same analysis to the estimated local field potentials from the scalp EEG. The same
frequency bands show the best performances, and the use of eLFP leads to an increase of performances of ~5%.