1 / 26

Applications - PowerPoint PPT Presentation

  • Uploaded on

Applications. BioSim. Mahantesh Halappanavar, Ashutosh Mishra, Ravindra Joshi, Mike Sachon. SURAgrid “All Hands” Meeting, Washington DC March 14 – 16, 2007. BioSim: Bio-electric Simulator for Whole Body Tissues.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Applications' - ramona

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript



Mahantesh Halappanavar,

Ashutosh Mishra, Ravindra Joshi,

Mike Sachon

SURAgrid “All Hands” Meeting, Washington DC

March 14 – 16, 2007

Biosim bio electric simulator for whole body tissues
BioSim: Bio-electric Simulator for Whole Body Tissues

  • Numerical simulations for electrostimulation of tissues and whole-body biomodels

  • Predicts spatial and time dependent currents and voltages in part or whole-body biomodels

  • Numerous diagnostic and therapeutic applications, e.g., neurogenesis, cancer treatment, etc.

  • Fast parallelized computational approach

Simulation models
Simulation Models

  • Whole-body discretized within a cubic space simulation volume

  • From electrical standpoint, tissues are characterized as conductivities and permittivities

  • Cartesian grid of points along the three axes. Thus, at most a total of six nearest neighbors

* Dimensions in millimeters

Numerical models
Numerical Models

  • Kirchhoff’s node analysis

  • Recast to compute matrix only once

  • For large models, matrix inversion is intractable

  • LU decomposition of the matrix

Numerical models1
Numerical Models


  • Voltage: User-specified time-dependent waveform

  • Impose boundary conditions locally

  • Actual data for conductivity and permittivity

  • Results in extremely sparse (asymmetric) matrix

Red: Total elements in the matrix

Blue: Nonzero Values

Why focus on solvers
Why Focus on Solvers?

  • Scaling: (Source: David Keys, NIA Nov 2006)

    • “Science” phase scales as:

    • “Solver” phase scales as

    • Computation will be almost all solver after several doublings

    • Optimal solver saves computation cycles for physics

The landscape of sparse ax b solvers


A = LU


y’ = Ay

More General






More Robust

More Robust

Less Storage

The Landscape of Sparse Ax=b Solvers

Source: John Gilbert, Sparse Matrix Days in MIT 18.337

Lu decomposition
LU Decomposition

Source: Florin Dobrian

Lu decomposition1
LU Decomposition

Source: Florin Dobrian

Computational complexity
Computational Complexity

  • 100 X 100 X 10 nodes: ~75 GB of memory (8-B floating precision)

  • Sparse data structure: ~ 6 MB (in our case)

  • Sparse direct solver: SuperLU-DIST

    • Xiaoye S. Li and James W. Dimmel, “SuperLU-DIST: A Scalable Distributed-Memory Sparse Direct Solver for Unsymmetric Linear Systems”, ACM Trans. Mathematical Software, June 2003, Volume 29, Number 2, Pages 110-140.

  • Fill reducing orderings with Metis

    • G. Karypis and V. Kumar, “A fast and high quality multilevel scheme for partitioning irregular graphs”, SIAM Journal on Scientific Computing, 1999, Volume 20, Number 1.

Performance on compute clusters
Performance on compute clusters

144,000-node Rat Model

Blue: Average iteration time

Cyan: Factorization time

Output visualization with matlab
Output: Visualization with MATLAB

Potential Profile at a depth of 12mm

Output visualization with matlab1
Output: Visualization with MATLAB

  • Simulated Potential Evolution

  • Along the Entire 51-mm Width of the Rat Model

Deployment on
Deployment on

  • Mileva: 4-node cluster dedicated for SURAgrid purposes

  • Authentication

    • ODU Root CA

    • Cross certification with SURA Bridge

    • Compatibility of accounts for ODU users

  • Authorization

  • Initial Goals:

    • Develop larger whole-body models with greater resolution

    • Scalability tests

Grid workflow
Grid Workflow

  • Establish user accounts for ODU users

    • SURAgrid Central User Authentication and Authorization System

    • Off-line/Customized (e.g., USC, LSU)

  • Manually launch jobs based on remote resource

    • SSH/GSISSH/SURAgrid Portal


  • Transfer files

    • SCP/GSISCP/SURAgrid Portal

Recent efforts in grid enabling
Recent Efforts in grid-enabling:

  • Porting to 100% open source tools (GCC/GFORTRAN)

  • SURAgrid Sites:

    • Texas A&M University: Calclab

    • University of Virginia: Grid04 and Grid11

  • Experiments with MUMPS 4

    • Symmetric matrices and out-of-core

  • Acknowledgements:

    • Jim Jokl, Steve Losen, Steve Johnson, Brain Brooks, Kate Barzee and Mary Fran Yafchak

News february 14 2007
News: (February 14, 2007)


  • Science:

    • Electrostimulation has variety of diagnostic and therapeutic applications

    • While numerical simulations provide many advantages over real experiments, they can be very arduous

  • Grid enabling:

    • New possibilities with grid computing

    • Grid-enabling an application is complex and time consuming

    • Security is nontrivial

Future steps
Future Steps

  • Grid-enable BioSim

    • Explore alternatives for grid enabling BioSim

    • Explore funding opportunities

    • Load Balancing

    • Establish new collaborations

    • Scalability experiments with large compute clusters accessible via SURAgrid

  • Future applications:

    • Molecular and Cellular Dynamics

    • Computational Nano-Electronics

    • Tools: Gromacs, DL-POLY, NAMD

References and contacts
References and Contacts

  • A Mishra, R Joshi, K Schoenbach and C Clark, “A Fast Parallelized Computational Approach Based on Sparse LU Factorization for Predictions of Spatial and Time-Dependent Currents and Voltages in Full-Body Biomodels”, IEEE Trans. Plasma Science, August 2006, Volume 34, Number 4.


  • Ravindra Joshi, Ashutosh Mishra, Mike Sachon, Mahantesh Halappanavar

    • (rjoshi, amishra, msachon, mhalappa)

Teaching initiative

Teaching Initiative

CS775/875: Distributed Computing

Ravi Mukkamala

Professor, Department of Computer Science


  • Graduate course with ~15 students

  • Guest lecture

  • Followed by a homework

    • Familiarize with grid computing concepts

    • Hands-on approach

    • Experiment with Globus services & commands

  • Acknowledgements:

    • Jim Jokl, Steve Losen, Steve Johnson, Brain Brooks, Nicole Geiger, Kate Barzee and Mary Fran Yafchak


  • Laboratory for testing the concepts

  • Potential to attract students

  • For SURAgrid

    • Large number of short-lived certificates

    • Cleanup … (CRLs?/home drives/…)

    • Centralized account creation (Still painful )

    • Short term funding/internships for grad/under-grad students?