An Object Oriented Approach To Bio-Feedback Applications For Disabled People

1. An Object Oriented Approach To Bio-Feedback Applications For Disabled People ICBEM 2000 Luigi Bianchi1,5, Fabio Babiloni2, Febo Cincotti3, Serenella Salinari4, Maria Grazia Marciani 1,3 1 Dip. Neuroscienze, Università “Tor Vergata”, Rome, ITALY 2 Dip. Fisiologia Umana e Farmacologia, Università “La Sapienza”, Rome, ITALY 3 IRCCS, Fondazione “S. Lucia”, Rome, ITALY 4 Dip. Informatica e Sistemistica, Università “La Sapienza”, Rome, ITALY 5 Brainware, Rome, ITALY E-mail: icbem@luigibianchi.com

2. ICBEM 2000 Introduction (1) One of the main problems encountered in the development of computer-based systems for handicapped people is that it is very difficult to optimize them in a wide range of situations. This happens mainly because every potential user has residual capabilities that are specific to his condition and that make him in some way unique. It should be extremely useful to use all of the voluntarily controlled activities (VCA) such as eye movements, muscle contractions, EEG activity, etc. as inputs for a processing unit that could take care of recognizing them and then translating them into desired tasks like opening doors, playing-back some pre-recorded phrases, using the telephone and so on. Unfortunately, building a system for even a well-defined pathology might be difficult: optimal use of residual capabilities in different subjects could require modifying it at a level that is incompatible with practical needs.

3. ICBEM 2000 Introduction (2) However, while the number and type of the voluntarily controlled biological signals is specific to every patient, the number of tasks that one wants to execute is quite homogeneous. Furthermore, even if the nature of the utilized signals may vary among different pathological situations, the way in which a biofeedback system works is quite stereotyped:after the data have been acquired, there is a DSP pre-processing stage that performs some basic operations on the input signals, then a classification stage in which some features are extracted (FE), manifested in some way to the user and eventually attributed to one of the subject VCA (discriminator), and finally, a stage in which a task can be executed.

4. ICBEM 2000 Introduction (3) Finally it is very frequent that these systems provide different operating modalities such as training, testing, setup, and running. This is a situation in which an object oriented programming (OOP) approach reaches his best results: inheritance, virtual functions and templates are all instruments created to reuse code and to allow generic programming. In our case it is possible to describe the operative flow that is common to all biofeedback applications leaving to be defined only those aspects that are specific to the single implementation, such as the algorithms and the classification rules. Then, in a separate step, several systems can be realized just defining the points that are intentionally left unspecified. Here we describe an object orientedsoftware framework that can be used as a skeleton for the practical implementation of a wide range of Bio-feedback systems. It is important to notice that the proposed solution does not make any assumption on both the operating system and the hardware used.

5. ICBEM 2000 Data Acquisition Hardware Algorithm Modalities Utilities Bio-Feedback System • A Bio-Feedback system is usually composed of various hardware and software modules that interact among themself and that implement different tasks. The main sub-systems are: • A Data Acquisition Board that collects biological signals; • A hardware that receives collected data (e.g. a Personal Computer) and that interacts with the environment, driving special devices; • Some algorithms that extract the desired features from the biological signals; • Different operating modalities, such as training, exercise, running. • Several supporting utilities for configuring the system in all the aspects.

6. ICBEM 2000 Modalities (1) Another aspect that is always encountered in different systems is the fact that they must operate in more modalities such as training, testing, running and setup. The training modality is relative to a period in which some parameters relative to the user are extracted in order to tune the system to him. During this phase, one asks to the subject to perform an action several times in order to collect enough reference data that will be used later on by the classifier. This procedure is repeated for all the VCAs that the patient intends to use. a) training Ref 1 The running modality is devoted to the identification of a VCA and to the triggering of an event that might start an action in the case of a successful classification. Ref 2 ... VCA 1 Ref m VCA 2 ... ACQ DSP F.E. VCA m b) running Ref 1 Ref 2 Task 1 ... VCA 1 state Task 2 Ref m VCA 2 ... Task ... ACQ DSP F.E. Discr. driver Task n VCA m

7. ICBEM 2000 Modalities (2) The testing modality is somehow in the middle between the previous two: the system asks to the subject to perform one of the VCAs, and then try to classify it. After that, an internal score is updated and an event is triggered in order to signal a success or a failure. In this way it is possible to evaluate the performance of the whole system for every single performed task. The setup modality is relative to the hardware configuration and to the loading and saving of the references and other parameters computed during previous training sessions.

8. ICBEM 2000 Algorithms Even if Bio-Feedback systems are mainly characterized by the classification rules and the feature extraction algorithms, most of them share the same DSP engines at least at a pre-processing stage. This includes FFT computation and digital filtering. Furthermore, advanced matrix computation routines are commonly required for the complete implementation of a particular tool such as the ones based on Signal Space Projectionor on the Fisher Discriminant Linear Classifiers. Other algorithms which are based on Neural Networks, still need a pre-processing stage for DSP. Anyway, the workflow of all these systems are still the one illustrated in the introduction section, or at least it is easily re-conducible to it.

9. ICBEM 2000 Hardware-Data Acquisition Actually there are several implementations for the realization of the various systems. They are mainly characterized by: • Platform used (PC, Macintosh, Palm, proprietary, etc…) • Operating System (Win32, WinCE, MacOS, Linux, Unix, BeOS, proprietary, etc…) • A/D and D/A converter (proprietary, National Instruments, Analog Devices, etc…) • Biological signal treated (EMG, EEG, Voice, Breath, etc…) • Amplifiers • I/O protocols and peripheral (RS232, microphones and speakers, TTL triggers, digital cameras, USB, IEEE488, FireWire, etc…)

10. ICBEM 2000 A modern approach • A different way to realize a versatile and easily configurable Bio-Feedback system is to make few assumptions about it. In particular it must: • be an open architecture; • operate with a high hardware abstraction level; • be easily configurable; • be easily customizable; • maximize the software source code reuse; • maximize the software binary code reuse; • interact with the toolsanddevicesalready available; • facilitate the diffusion of new algorithms.

11. ICBEM 2000 A modern Answer (1) Object Oriented Programming can give a lot of answers to the previously listed key-points. Out implementation uses C++ for the following reasons: C compatibility, in which almost all the operating system are written; Speed, because c++ generated binary code is generally faster than the one generated by other languages ones such as Java; Diffusion, because it is the most widely used; Hardware control, because the vast majority of peripherals and IO board are easily driven in C; Language features, such as virtual functions, inheritance, template, that allows generic programming; Cross-platform portability, because it is possible to re-compile an ANSI C++ program under almost every platform.

12. ICBEM 2000 A modern Answer (2) Another key issue is that it is fundamental to keep separate the hardware IO aspects from the implementation of the different algorithms. This goal can be achieved by making a simple assumption: the acquired data must be stored on a buffer whose size is large enough to hold the data required for the real-time processing. It is not important how the data are collected: it is sufficient to notify in some way when and where new biological acquired data are available. Other relevant information are related to every single acquired channel, such as gain, sampling rate, and coordinate position. This last can be used by special algorithms such as those based on Laplacian estimation for example in some Brain Computer Interface (BCI) systems.

13. ICBEM 2000 A modern Answer (3) To solve all these problems, a hierarchy of classes have been implemented, in order to keep into account different requirements. The first class is the BFEnv, which implements the operative flow of an Bio-Feedback application as described in the introduction. All the timing aspects are treated here, and special virtual functions such as DecisionRule() and ComputeAlgorithm() must be defined to characterize a specific system. Other virtual functions such as OnSuccess() and OnFailure() are automatically triggered every time a recognition based on the specific DecisionRule and ComputeAlgorhitm implementations occurs. Another object derived from BFEnv, called BFSpeEnv, adds some FFT capabilities, while BFAdvEnv gives also matrix computation routines. Note that all the source described here is cross-platform and to be considered freeware for any non commercial use and can be obtained at www.luigibianchi.com.

14. ICBEM 2000 SSP implementation (1) MS Windows Linux Bio-Feedback Software Development Kit (with matrix computation and basic DSP routines) SSP Mac OS BeOS At this point we have decided to implement a Signal Space Projection (SSP) algorithm. In order to do that, we have decided to adopt MS Windows as operating system, and the EEG signal as biological input.

15. ICBEM 2000 SSP implementation (2) Anyway, the implementation was performed into two separate levels: the first one for all the necessary processing, including the DecisionRule and ComputeAlgorithm implementation (and this still save the cross-platform compatibility); the second one to add all aspects that are dependent on the operating system used such as the Graphic User Interface (GUI) and other external tool support such as Text to Speech, Voice Command capabilities (SAPI 4.0) and Dual Monitor support. This last feature was necessary to give to different views to the user (cursor movement) and to the technician (full EEG signal display with real-time 3D spectral maps) In an immediate future step, accessibility options integrated in all the Win32 platform will be also used. All the implementations were done using Borland C++Builder 4 compiler.

16. ICBEM 2000 SSP implementation (3) It is important to note that all the implementation without the GUI and some other OS specific aspects, required less that 200 lines of C++ code, so divided: 20 lines for pseudo-inverse matrix computation; 30 lines to implement the DecisionRule and the ComputeAlgorithm functions; 60 lines for constructors, destructors, allocation and de-allocation; 80 lines for other things, including dual monitor support, in order to give to different views to the user (cursor movement) and to the technician (full EEG signal display with real-time 3D spectral maps)

17. ICBEM 2000 SPP Screenshot

18. ICBEM 2000 Conclusions A model for a generic biofeedback system was described. Its main advantages are the net distinction of the functional blocks that are relative to the hardware, the input signals, the algorithms and the type and degree of disabilities. In this way it is possible to reuse a huge amount of code, thus minimizing the time required in the development and adaptation of a system to a wide number of patients.