parallel and distributed computing for neuroinformatics n.
Skip this Video
Loading SlideShow in 5 Seconds..
Parallel and Distributed Computing for Neuroinformatics PowerPoint Presentation
Download Presentation
Parallel and Distributed Computing for Neuroinformatics

Loading in 2 Seconds...

play fullscreen
1 / 62

Parallel and Distributed Computing for Neuroinformatics - PowerPoint PPT Presentation

  • Uploaded on

Parallel and Distributed Computing for Neuroinformatics. Allen D. Malony University of Oregon. Professor Department of Computer and Information Science. Director NeuroInformatics Center Computational Science Institute. Outline. Neuroinformatics Dynamic brain analysis problem

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Parallel and Distributed Computing for Neuroinformatics' - odele

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
parallel and distributed computing for neuroinformatics

Parallel and Distributed Computing for Neuroinformatics

Allen D. Malony

University of Oregon


Department of Computerand Information Science


NeuroInformatics Center

Computational Science Institute

  • Neuroinformatics
    • Dynamic brain analysis problem
    • NeuroInformatics Center (NIC) at UO
  • Neuroinformatics research at the NIC
    • Dense-array EEG analysis (APECS, HiPerSat, Mc)
    • Brain image segmentation
    • Computational head modeling
    • Ontologies and tool integration (NEMO, GEMINI)
    • Parallel and distributed computing emphasis
  • ICONIC Grid HPC system at UO
  • Cerebral Data Systems
  • Oregon E-science
neuroscience and neuroinformatics
Neuroscience and Neuroinformatics
  • Application of computer and information science to the understanding of brain organization and function
  • Integration of information across many levels
    • Physical and functional
    • Gene to behavior
    • Microscopic to macroscopic scales
  • Challenges in brain observation and modeling
    • Structure and organization (imaging)
    • Operational and functional dynamics (temporal/spatial)
    • Physical, functional, and cognitive operation (models)
  • How to create and maintain of integrated views of the brain for both scientific and clinical purposes?
human brain dynamics analysis problem
Human Brain Dynamics Analysis Problem
  • Understand functional operation of the human cortex
    • Dynamic cortex activation
    • Link to sensory/motor and cognitive activities
    • Multiple experimental paradigms and methods
    • Multiple research, clinical, and medical domains
  • Need for coupled/integrated modeling and analysis
    • Multi-modal observation (electromagnetic, MR, optical)
    • Physical brain models and theoretical cognitive models
  • Need for robust tools
    • Complex analysis of large multi-model data
    • Reasoning and interpretation of brain behavior
  • Problem solving environment for brain analysis
neuroimaging technology
Neuroimaging Technology
  • Alternative techniques for analyzing brain dynamics
  • Blood flow neuroimaging (PET, fMRI)
    • Good spatial resolution
      • allows functional brain mapping
    • Temporal limitations to tracking of dynamic activities
  • Electromagnetic measures (EEG/ERP, MEG)
    • msec temporal resolution
      • can distinguish fast neurological components
    • Spatial resolution sub-optimal
      • requires a mapping to cortical sources
  • Need integrated neuroimaging technology (How?)
    • Achieve both good spatial and temporal resolution
integrated dynamic brain analysis

good spatial

poor temporal

Cortical Activity

Knowledge Base

Head Analysis

Source Analysis

Structural /



spatial pattern





Activity Model





IndividualBrain Analysis


Response Model




Array EEG / MEG

temporal pattern


Signal Analysis

Response Analysis

Component Response

Knowledge Base

poor spatial

good temporal



Integrated Dynamic Brain Analysis
experimental methodology and tool integration
Experimental Methodology and Tool Integration

16x256bits permillisec







processed EEG

mesh generation

source localization constrained to cortical surface

Interpolator 3D



neuroinformatics center nic at uo
NeuroInformatics Center (NIC) at UO
  • Application of computational science methods to human neuroscience problems
    • Tools to help understand dynamic brain function
    • Tools to help diagnosis brain-related disorders
    • HPC simulation, large-scale data analysis, visualization
  • Integration of neuroimaging methods and technology
    • Need for coupled modeling (EEG/ERP, MR analysis)
    • Apply advanced statistical signal analysis (PCA, ICA)
    • Develop computational brain models (FDM, FEM)
    • Build source localization models (dipole, linear inverse)
    • Optimize temporal and spatial resolution
  • Internet-based capabilities for brain analysis services, data archiving, and data mining
nic organization
NIC Organization
  • Allen D. Malony, Director
  • Don M. Tucker, Associate Director
  • Sergei Turovets, Computational Physicist
  • Bob Frank, Mathematician
  • Dan Keith, Software Engineer (distributed systems grid)
  • Chris Hoge, Software Engineer (computational)
  • Ryan Martin / Brad Davidson, Systems administrators
  • Gwen Frishkoff, Research Associate, Univ. Pittsburgh
  • Kai Li, Ph.D. student (brain segmentation)
  • Adnan Salman, Ph.D. student (computational modeling)
  • Performance Research Lab (my other hat)
brainwave research 101
Brainwave Research 101
  • Electroencephalogram (EEG)
    • Electrodes (sensors) measure uV
  • EEG time series analysis
    • Event-related potentials (ERP)
      • link brain activity to sensory–motor, cognitive functions
      • statistical average to increase signal-to-noise ratio (SNR)
    • Signal cleaning and component decomposition
    • Localize (map) to neural sources
electrical geodesics inc egi
Electrical Geodesics Inc. (EGI)
  • EGI Geodesics Sensor Net
    • Dense-array sensor technology
      • 64/128/256 channels
    • 256-channel GSN
      • AgCl plastic electrodes
  • Net Station
    • Advanced EEG/ERP data analysisand visualization
  • Stereotactic EEG sensor registration
  • State of the art technology
  • Research / clinical products
  • EGI/CDS medical services
  • Epilepsy affects more than 5 million people yearly
    • U.S., Europe, and Japan
  • EEG in epilepsy diagnosis
    • Childhood and juvenile absence
    • Idiopathic (genetic)
    • Distinguish different types
  • EEG in presurgical planning
    • Localize seizure onset
    • Fast, safe, inexpensive
    • Dense array improves accuracy
    • Requires good source modeling
eeg time series progression of absence seizure
EEG Time Series - Progression of Absence Seizure

First full spike–wave


topographic mapping of spike wave progression
Topographic Mapping of Spike-Wave Progression
  • Palette scaled for wave-and-spike interval (~350ms)

-130 uV (dark blue)  75 uV (dark red)

  • 1 millisecond temporal resolution is required
  • Spatial density (256) to capture shifts in topography
neuroinformatic challenges
Neuroinformatic Challenges
  • Dense-array EEG signal analysis and decomposition
    • Artifact cleaning and component analysis
  • Automatic brain image segmentation
    • Brain tissue identification
    • Cortex extraction
  • Computational head modeling
    • Tissue conductivity estimation
    • Source localization
  • Statistical analysis to detect brain states
    • Discriminant analysis
    • Pattern recognition
  • Electromagnetic databases and ontologies
applying ica for eeg blink removal
Applying ICA for EEG Blink Removal
  • Component analysis is used to separate EEG signals
    • Independent component analysis (ICA)
  • Blinks are a major source of noise in EEG data
  • Blink signals are separable from cognitive responses







Identify blinks

and remove

  • Event info
  • Time markers
  • Blink events
  • Bad channel removal
  • Baseline correction
  • ICA algorithms - Infomax - FastICA - …
  • ICA components
  • Blink templates
  • Reconstitute EEG w/out blink data



independent component analysis
Independent Component Analysis

EEG waveform


sinusoids- raw EEG




tool for eeg data decomposition apecs
Tool for EEG Data Decomposition (APECS)
  • Automated Protocol for Electromagnetic Component Separation (APECS)
  • Motivation
    • EEG data cleaning (increases signal-to-noise (SNR))
    • EEG component separation (addresses superposition)
    • Data preprocessing prior to source localization
  • Distinctive Features
    • Implements different decomposition methods
    • Multiple metrics for component classification
    • Quantitative and qualitative criteria for evaluation
apecs evaluation qualitative criteria
APECS Evaluation: Qualitative Criteria

Spatio-temporal blink profiles






apecs evaluation quantitative criteria
APECS Evaluation: Quantitative Criteria

Covariance between “baseline” (blink-free) and ICA-filtered data: Infomax and FastICA.

Infomax gives consistently better results. FastICA results are more variable.

ICA decompositions most successful when only one spatial projector is strongly correlated with blink “template” (spatial filter).

parallelization of component analysis algorithms
Parallelization of Component Analysis Algorithms
  • Dense-array EEG increases analysis complexity
  • Long time measurements require more processing
  • ICA algorithms are computationally challenging
    • Processing time and memory requirements
  • Increase performance through ICA parallelization
  • High-Performance Signal Analysis Toolkit (HiPerSAT)
    • Matlab ICA algorithms implemented in C++
      • Infomax: Matlab (runica.m)  parallel (OpenMP)
      • FastICA: Matlab (fastica.m)  parallel (MPI)
    • Validate results with Matlab standard algorithms
    • Evaluate accuracy and compare speedup
hipersat parallel infomax
HiPerSAT Parallel Infomax
  • Requires multi-threading
  • Over 3 timesfaster thanMatlab
    • 3-fold increaseon 4 processors
  • Speedup falls after4 processors
    • Limits on parallelizationof loop matrix operations
    • May be able to improvewith larger blocking
hipersat parallel fastica
HiPerSAT Parallel FastICA
  • Linear speedup
  • Over 130 times fasterthan Matlab
    • 8-fold increaseon 32 processors
  • Performancemay allow finerprocessing
  • Tradeoff improvedaccuracy versus morecomplex processing andexecution time
matlab tool integration
Matlab Tool Integration
  • Many neuroimaging tools are based on Matlab
    • EEGLAB (UC San Diego)
    • BrainStorm (USC / Los Alamos National Lab)
    • SPM (University College of London)
  • Matlab is mostly a closed computational environment
  • EEG/MEG analysis can overwhelm Matlab
    • Limited to workstation processing resources
    • Memory requirements high due to Matlab workspace
  • Desire to use Matlab as a client in distributed systems
    • Matlab is not multi-threaded
    • Complicates building concurrent for external interfaces
mc matlab concurrent
Mc: Matlab Concurrent
  • Matlab is built on top of a JVM
    • Use for GUI and graphics
  • We can leverage JVM for concurrency and interaction
  • How do we create concurrent tasks in Matlab?
    • Matlab programming semantics issue
    • Create task abstraction
    • Provide Matlab package for constructing tasks
  • Concurrent tasks interface with runtime layer
    • Client task manager runs tasks on servers and monitors
    • Server task executor schedules tasks on resources
  • Voila!  Mc
mc task manager
Mc Task Manager
  • Single background thread running in the JVM
  • Coordinate several components:
    • File transfer
      • Ganymed SSH2 implementation for Java
    • Matlab interaction
      • MatlabControl class
    • Scripting with Jython
      • dynamic access to objects in the JVM
      • Matlab m-scripting through MatlabControl class
        • waits for free cycles when execution Matlab code
  • Responds to user or timer events
  • Manages execution of tasks and status reporting
hipersat apecs and mc
HiPerSAT, APECS, and Mc
  • Demonstrated APECS running HiPerSAT tasks
  • 20 simultaneous HiPerSAT servers on a cluster
  • Compare with sequential EEGLAB processing
    • ICA processing on 20 1-GB EEG files
  • Performance speedup
    • 5x for C++ optimization and 2-way parallelism
    • 10x speedup for 20 concurrent HiPerSAT tasks
  • Also demonstrate for multiple Matlab clients
  • Recently applied to EEG processing for imagery analysis
    • 100+ simultaneous tasks across 8 different platforms
  • K. Glass, G. Frishkoff, R. Frank, C. Davey, J. Dien, A. Malony, A Framework for Evaluating ICA Methods of Artifact Removal from Multichannel EEG, ICA Conference, Grenada, Spain, 2004.
  • R. Frank and G. Frishkoff, APECS: A Framework for Implementation and Evaluation of Blink Extraction from Multichannel EEG, Journal of Clinical Neurophysiology, to appear, 2006.
  • D. Keith, C. Hoge, A. Malony, and R. Frank, “Parallel ICA Methods for EEG Neuroimaging,” International Parallel and Distributed Processing Symposium (IPDPS 2006), May 2006.
  • C. Hoge, D. Keith, and A. Malony, “Client-side Task Support in Matlab for Concurrent Distributed Execution,” Austrian-Hungarian Workshop on Distributed and Parallel Systems (DAPSYS), September 2006.
animated topography of spike wave dynamics
Animated Topography of Spike–Wave Dynamics
  • Spatial and temporal dynamicsare important to observetogether
  • Linked cortical networks
    • Fronto-thalamic circuit

(executive control)

    • Limbic circuit

(episodic memory)

  • Problem of Superposition
    • How many sources?
    • Where are they located?
  • Can only infer locations using“scalp space”information
addressing superposition brain electrical fields
Addressing Superposition: Brain Electrical Fields
  • Brain electrical fields are dipolar
  • Volume conduction
    • Depth and location indeterminacy
    • Highly resistive skull (CSF: skull est. from 1:40 to 1:80)
    • Left-hemisphere scalp field may be generated by a right-hemisphere source
  • Multiple sources  superposition
    • Radial source Tangential sources
      • one and two sources  varying depths
source localization
Source Localization
  • Mapping of scalp potentials to cortical generators
    • Signal decomposition (addressing superposition)
    • Anatomical source modeling (localization)
  • Source modeling
    • Anatomical Constraints
      • Accurate head model and physics
      • Computational head model formulation
    • Mathematical Constraints
      • Criteria (e.g., “smoothness”) to constrain solution
  • Current solutions limited by
    • Simplistic geometry
    • Assumptions of conductivities
brain sources of epileptic seizure
Brain Sources of Epileptic Seizure
  • Single time point source solution
  • Need to identify sources for each msec time sample
  • Visualize dynamics in “source space”
dipole sources in the cortex
Dipole Sources in the Cortex
  • Scalp EEG is generated in the cortex
  • Interested in dipole location, orientation, and magnitude
    • Cortical sheet gives possible dipole locations
    • Orientation is normal to cortical surface
  • Need to capture convoluted geometry in 3D mesh
    • From segmented MRI/CT
  • Linear superposition
advanced image segmentation
Advanced Image Segmentation
  • Native MR gives high gray-to-white matter contrast
  • Image analysis techniques
    • Edge detection, edge merger, region growing
    • Level set methods and hybrid methods
    • Knowledge-based
  • After segmentation, color contrasts tissue type
  • Registered segmented MRI
network flow based skull stripping
Network Flow Based Skull Stripping
  • Graph construction with LoG
    • Nodes, edges, edge weights, source terminals, sink terminals
  • Identify non-brain terminals
    • Scalp, eyeballs, orbits
    • Fourth ventricle, pons
  • Identify brain terminals
  • Knowledge-based terminal detection
  • Min-cut (Max-flow)
    • Separate source from sink
    • Sum of weights is minimum
cortical surface extraction pipeline
Cortical Surface Extraction Pipeline
  • Initial segmentation of white matter
    • Scale-space based analysis of LoG
    • Incorporation of spatial knowledge
  • Cortical surface topology correction
    • Minimize cortical surface are increase
    • Based on network flow theory
    • Based on Cauchy-Crofton formula
  • Cortical surface geometry refinement
    • White matter shape knowledge
      • radius function constraints
      • sheet constraints
    • Medial surface (skeleton) manipulation
topology correction in cortex extraction
Topology Correction in Cortex Extraction

Extracted Cortex

Before topology correction

After topology correction

building computational brain models
Building Computational Brain Models
  • MRI segmentation of brain tissues
  • Conductivity model
    • Measure head tissue conductivity
    • Electrical impedance tomography
      • small currents are injectedbetween electrode pair
      • resulting potential measuredat remaining electrodes
    • Finite element forward solution
  • Source inverse modeling
    • Explicit and implicit methods
    • Bayesian methodology
conductivity modeling
Conductivity Modeling

Governing Equations ICS/BCS

Continuous Solutions



System of Algebraic Equations

Discrete Nodal Values

TridiagonalADISORGauss-SeidelGaussian elimination

Equation (Matrix) Solver

 (x,y,z,t)J (x,y,z,t)B (x,y,z,t)

Approximate Solution

conductivity optimization and parallelization
Conductivity Optimization and Parallelization
  • Design as a conductivity search problem
    • Master launches new inverse problems with guesses
    • Inverse solvers run iterative forward calculations
    • Compare solutions with measured results
  • Parallelization approach
    • Forward solver
      • Alterative Direction Implicit (ADI) method
    • Optimization search
      • Simplex or simulated annealing (Monte Carlo)
    • Hybrid
alternating direction implicit adi method
Alternating Direction Implicit (ADI) Method
  • Finite difference method
    • C++ with OpenMP for parallelization
    • LAPACK for matrix operations
conductivity optimization dynamics
Conductivity Optimization Dynamics
  • Attempting todetermine tissueconductivity values
    • Scalp
    • Skull
    • Brain
    • CSF
  • Simplex algorithm
  • Values convergeto minimal error
  • Increase number of features in future work
  • K. Li, A. Malony, and D. Tucker, “Automatic Brain MR Image Segmentation with Relative Thresholding and Morphological Image Analysis,” International Conference on Computer Vision Theory and Applications (VISAPP), Setúbal, Portugal, February 2006.
  • K. Ki, A. Malony, and D. Tucker, “A Multiscale Morphological Approach to Topology Correction of Cortical Surfaces,” International Workshop on Medical Imaging and Augmented Reality (MIAR 2006), Sanghai, China, August 2006.
  • A. Salman, S. Turovets, A. Malony, J. Eriksen, and D. Tucker, “Computational Modeling of Human Head Conductivity,” International Conference on Computational Science (ICCS), best paper, May 2005.
  • A. Salman, S. Turovets, A. Malony, V. Vasilov, “Multi-Cluster, Mixed-Mode Computational Modeling of Human Head Conductivity,” International Workshop on OpenMP (IWOMP), June 2005.
  • S. Turovets, A. Salman, A. Malony, P. Poolman, C. Davey, D. Tucker, “Anatomically Constrained Conductivity Estimation of the Human Head in Vivo: Computational Procedure and Preliminary Experiments,” Electrical Impedance Tomography (EIT), July 2006.
neural electromagnetic ontology nemo
Neural ElectroMagnetic Ontology (NEMO)
  • How can brain electromagnetic (EEG and MEG) data be compared and integrated across experiments and laboratories?
  • Need a system for representation, storage, mining, dissemination
  • Need standardization of methods for measure generation
    • Identification and labeling of components
    • Patterns of interest
  • General agreement on criteria for component identification
    • Patterns can be hard to identify
    • Variability in techniques for measure generation
  • NEMO will address issue by providing
    • Spatial and temporal ontology database
    • Use for large-scale data representation, mining, and meta-analysis
    • Components in average EEG and MEG (ERPs)
nemo architecture
NEMO Architecture
  • Composed of three modules
    • Database mining
    • Inference engine
    • Query (user) interface
  • Methods for measure generation
    • Spatial ontologies and temporal ontologies
    • Cognitive functional mapping
  • User interactions
    • Query formulation
    • Mapping-rule definitions
  • Scalable integration system
  • Online repository for storing metadata
    • Spatio-temporal ontologies, database schema, mappings
iconic grid

graphics workstations

interactive, immersive viz

other campus clusters

Internet 2

Gbit Campus Backbone













IBM p655



IBM p690



SGI Prism






Dell Pentium Xeon

112 total



Shared Storage System

40 Terabytes

computational integrated neuroimaging system






compute resources

Computational Integrated Neuroimaging System
cis computational and visualization laboratory
CIS Computational and Visualization Laboratory
  • Support interdisciplinary computer science
    • Informatics
    • Computational science
  • Resource development
    • Phase 1 (complete)
      • NSF MRI grant ($1M)
      • ICONIC Grid
    • Phase II (underway)
      • Visualization Lab ($100K)
        • rear projection
          • 3D stereo and 2x2 tiled
        • 3x4 tiled 24” LCD display
    • Phase III …
gemini project
GEMINI Project
  • Grid-enabled Methods for Integrated NeuroImaging
  • Dynamic neuroimaging algorithms and visualization
  • Grid-based integration (processing and data sharing)
  • High-end tool integration and environments
  • Neuroinformatics data ontologies
  • GEMINI is
    • A distributable set of programs and configuration files
    • A set of web services and WSRF-accessible resources
    • An internet-accessible service executing on systems
    • Based on open technologies
gemini model
  • Zones
    • GEMINI resourceservers
  • Tasks
    • Job representation
  • Agents
    • Execution context
  • Build on endpointreferences
  • Supports workflow
leveraging internet hpc and grid computing
Leveraging Internet, HPC, and Grid Computing
  • Telemedicine imaging and neurology
    • Distributed EEG and MRI measurement and analysis
    • Neurological medical services
    • Shared brain data repositories
    • Remote and rural imaging capabilities
  • Build on emerging web services and grid technology
  • Leverage HPC compute and data centers
  • Create institutional and industry partnerships
    • Electrical Geodesics, Inc.
    • Cerebral Data Systems (UO partnership with EGI)
    • Other industrial partnerships
cerebral data systems
Cerebral Data Systems
  • Partnership between EGI and University of Oregon
  • Develop and market neuroinformatics services
    • Neurological medical data transfer, storage, and analysis
    • High-performance, expert EEG and MR analysis
    • Telemedicine and distributed collaboration
    • Shared brain repositories
  • Target markets
    • Research and clinical
    • Epilepsy diagnosis and pre-surgical planning
    • MR image segmentation
  • Technology integration
    • Internet, computional grids, and HPC
oregon e science grid
Oregon E-Science Grid

Internet 2 /National LambdaRail

Region 2

Region 1

Regional networks

Region 5

Region 4

Institutional partners


Regional points of presence

Region 3