Interactive Problem Solving: The Polder Meta Computing Inititiative - PowerPoint PPT Presentation

Interactive problem solving the polder meta computing inititiative l.jpg
Download
1 / 71

Interactive Problem Solving: The Polder Meta Computing Inititiative Peter Sloot Computational Science University of Amsterdam, The Netherlands Ariadne’s Red-Rope From PSE to Virtual Laboratory and Motivation Architecture Infrastructure Job Level: Hierarchical Scheduling

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Interactive Problem Solving: The Polder Meta Computing Inititiative

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Interactive problem solving the polder meta computing inititiative l.jpg

Interactive Problem Solving:The Polder Meta Computing Inititiative

Peter Sloot

Computational Science

University of Amsterdam, The Netherlands


Ariadne s red rope l.jpg

Ariadne’s Red-Rope

  • From PSE to Virtual Laboratory and Motivation

  • Architecture

    • Infrastructure

    • Job Level: Hierarchical Scheduling

    • Resource management: Task-migration

  • Interaction && Case implementation

  • Interactive Algorithms


Slide3 l.jpg

Virtual Laboratory Environment

Advanced Scientific Domains

Computational Physics

System Engineering

Computational Bio-medicine

Local User

Local User

Virtual Simulation & Exploration Environment (ViSE)

Communication & collaboration (ComCol)

Virtual-lab Information Management for Cooperation (VIMCO)

Physical apparatus

Distributed Computing & Gigabit Local Area Network

ViSE

Net Client

App. User

MRI/CT

Internet 2 Wide Area Network


Interactive computing why l.jpg

Interactive Computing: Why?

  • Goal: From Data, via Information to Knowledge

  • Complexity: Huge data-sets, complex processes

  • Approach: Parametric exploration and sensitivity analyses:

    • Combine raw (sensory) data with simulation

    • Person in the loop:

      • Sensory interaction

      • Intelligent short-cuts


Intro case study from biomedicine l.jpg

Intro: Case study from biomedicine...


Changing the paradigm l.jpg

In Vitro

In Vivo

In Silico

Changing the Paradigm


Changing the paradigm7 l.jpg

In Vitro

In Vivo

In Silico

Changing the Paradigm


Changing the paradigm8 l.jpg

In Vitro

In Vivo

In Silico

Changing the Paradigm


Current situation l.jpg

Diagnosis & Planning

Treatment

Observation

Current Situation


New possibilities in the vl l.jpg

Fast, High-throughput

Low Latency

Internet

High Performance

Super Computing

New Possibilities in the VL

  • Time and Space Independence

  • 3D Information

  • Simulation based planning

  • Surgeon ‘in the loop’


Experimental set up l.jpg

Experimental set-up


Architecture l.jpg

Architecture


Architecture continued hybrid system l.jpg

Cave

Origine 2000

9

10

11

12

13

14

8

15

7

16

6

17

5

18

4

19

ATM

3

20

2

1

0

23

22

21

GRAPE1

GRAPE0

Architecture Continued: Hybrid system

  • Host: The DAS

    • 24 node parallel cluster in a 200 node wide area machine

    • 200 MHz Pentium Pro

    • Myrinet 150MB/s

    • ATM wide-area interconnect between clusters


Immersive environments l.jpg

Immersive Environments


3d information and interaction l.jpg

3D Information and Interaction


Problem curse of dynamics l.jpg

Problem: Curse of dynamics:

Static task load

Dynamic task load

Static task allocation

Predictable reallocation

Dynamical reallocation

Static resource load

Dynamic resource load


Solution to curse l.jpg

Solution To Curse

  • Performance of a parallel program usuallydictated by slowest task

    • Task resource requirements and available resources both vary dynamically

    • Therefore, optimal task allocation changes

    • Gain must exceed cost of migration

  • Resources used by long-running programs may be reclaimed by owner


Dynamite initial state l.jpg

Node A

Node B

PVMtask 1

PVMD

A

PVMD

B

Node C

PVMtask 2

PVMD

C

Dynamite Initial State

Two PVM tasks communicating through a network of daemons

Migrate task 2 to node B


Prepare for migration l.jpg

Node A

Node B

Newcontext

PVMtask 1

PVMD

A

PVMD

B

Node C

Program

PVM

Ckpt

PVMD

C

Prepare for Migration

Create new context for task 2

Tell PVM daemon B to expect messages for task 2

Update routing tables in daemons (first B, then A, later C)


Checkpointing l.jpg

Checkpointing

Node A

Node B

Newcontext

PVMtask 1

PVMD

A

PVMD

B

Node C

Program

PVM

Ckpt

PVMD

C

Send checkpoint signal to task 2

Flush connections

Checkpoint task to disk


Cross cluster checkpointing design l.jpg

Cross-cluster checkpointing(design)

Node A

Node B

Helper

task

PVMtask 1

PVMD

A

PVMD

B

Node C

Program

PVM

Ckpt

PVMD

C

Send checkpoint signal to task 2

Flush connections, close files

Checkpoint task to disk via helper task


Restart execution l.jpg

Restart Execution

Node A

Node B

NewPVM

task 2

PVMtask 1

PVMD

A

PVMD

B

Node C

PVMD

C

Restart checkpointed task 2 on node B

Resume communications

Re-open & re-position files


Special considerations l.jpg

Special considerations

  • Preserve communication

    • PVM should continue to run as if nothing happened

    • Use location independent addressing

  • Open files

    • Preserve open file state


Performance l.jpg

Performance

  • Migration speed largely dependent on the speed of shared file system

    • and that depends mostly on the network

  • NFS over 100 Mbps Ethernet

    • 0.4 s < Tmig < 15 s for 2 MB < sizeimg < 64 MB

  • Communication speed reduced due to added overhead

    • 25% for 1 byte direct messages

    • 2% for 100 KB indirect messages


Current status dynamite part l.jpg

Current status: Dynamite Part

  • Checkpointer operational under

    • Solaris 2.5.1 and higher (UltraSparc, 32 bit)

    • Linux/i386 2.0 and 2.2 (libc5 and glibc 2.0)

  • PVM 3.3.x applications supported and tested

    • Pam-Crash (ESI) - car crash simulations

    • CEM3D (ESI) - electro-magnetics code

    • Grail (UvA) - large, simple FEM code

    • NAS parallel benchmarks

    • BloodFlow

  • MPI and socket (Univ. of Krakow) libraries available

  • Scheduling not yet satisfactory


  • Architecture revisited l.jpg

    Architecture: Revisited


    Design considerations l.jpg

    Design Considerations

    • High Quality presentation

    • High Frame rate

    • Intuitive interaction

    • Real-time response

    • Interactive Algorithms

    • High performance computing and networking...


    Problem time time what has become of us l.jpg

    Problem: Time, time what has become of us?


    Solution asynchronicity l.jpg

    Solution: Asynchronicity


    A police officer to guide the asynchronous processes l.jpg

    A police officer to guide the asynchronous processes


    Runtime support l.jpg

    Runtime Support

    • Need generic framework to support modalities

    • Need interoperability

    • High Level Architecture (HLA):

      • data distribution across heterogeneous platforms

      • flexible attribute and ownership mechanisms

      • advanced time management


    Provoking a bit l.jpg

    Provoking a bit…

    Progress in natural sciences comes from taking things apart ...

    Progress in computer science comes from bringing things together...


    Proof is in the pudding l.jpg

    Proof is in the pudding...

    • Diagnostic Findings

      • Occluded right iliac artery

      • 75% stenosis in left iliac artery

      • Occluded left SFA

      • Diffuse disease in right SFA


    Problem from image to simulation l.jpg

    Problem: From Image to Simulation

    MR Scan of Abdomen

    MR Scan of Legs


    Solution 3dmanual initialization l.jpg

    Solution: 3DManual initialization

    Place start point

    Place one or more end points

    Wave propagates from start- to end point

    Backtrack = first estimation of the centerline

    Wave propagates from ‘centerline’  vessel wall

    Distance Transform from vessel wall to center  centerline


    Wavefront propagation l.jpg

    Wavefront Propagation

    Place start point

    Place one or more end points

    Wave propagates from start- to end point

    Backtrack = first estimation of the centerline

    Wave propagates from ‘centerline’  vessel wall

    Distance Transform from vessel wall to center  centerline


    Mra backtrack l.jpg

    MRA: Backtrack

    Place start point

    Place one or more end points

    Wave propagates from start- to end point

    Backtrack = first estimation of the centerline

    Wave propagates from ‘centerline’  vessel wall

    Distance Transform from vessel wall to center  centerline


    Mra wavefront propagation l.jpg

    MRA: Wavefront Propagation

    Place start point

    Place one or more end points

    Wave propagates from start- to end point

    Backtrack = first estimation of the centerline

    Wave propagates from ‘centerline’  vessel wall

    Distance Transform from vessel wall to center  centerline


    Mra distance transform l.jpg

    MRA: Distance Transform

    Place start point

    Place one or more end points

    Wave propagates from start- to end point

    Backtrack = first estimation of the centerline

    Wave propagates from ‘centerline’  vessel wall

    Distance Transform from vessel wall to center  centerline


    3 d selection of region of interest l.jpg

    3-D selection of region of interest


    Tracking the vessels l.jpg

    Tracking the vessels


    Building the geometric models l.jpg

    Building the Geometric Models


    Vr interaction l.jpg

    VR-Interaction


    Alternate treatments l.jpg

    Alternate Treatments

    Preop

    AFB w/

    E-S Prox.Anast.

    AFB w/

    E-E Prox.Anast.

    Angio w/Fem-Fem

    Angio w/ Fem-Fem &

    Fem-Pop


    Problem flow through complex geometry l.jpg

    Problem: Flow through complex geometry

    • After determining the vascular structure simulate the blood-flow and pressure drop…

    • Conventional CFD methods might fail:

      • Complex geometry

      • Numerical instability wrt interaction

      • Inefficient shear-stress calculation


    Solution to interactive flow simulation l.jpg

    Solution to interactive flow simulation

    • Use Cellular Automata as a mesoscopic model system:

      • Simple local interaction

      • Support for real physics and heuristics

      • Computational efficient


    Mesoscopic fluid model l.jpg

    Mesoscopic Fluid Model

    • Fluid model with Cellular Automata rules

    • Collision: particles reshuffle velocities

    • Imposed Constraints

      • Conservation of mass

      • Conservation of momentum

      • Isotropy

    Details...


    Equivalence with ns l.jpg

    ...Equivalence with NS

    • For lattice with enough symmetry: equivalent to the continuous incompressible Navier-Stokes equations:

    Implicit parallel and complex geometry support.


    Efficient calculation of shear stress l.jpg

    Efficient Calculation of Shear-Stress

    Perpendicular momentum transfer:

    AND the momentum stress tensor P thatis linearly related to the shear stresses sab

    From LBE scheme:


    Velocity magnitude l.jpg

    10 cm/sec

    0 cm/sec

    Velocity Magnitude


    Peak systolic pressures rest l.jpg

    Peak Systolic Pressures - Rest

    150 mmHg

    50 mmHg

    Preop

    AFB w/

    E-S Prox.Anast.

    AFB w/

    E-E Prox.Anast.

    Angio w/Fem-Fem

    Angio w/ Fem-Fem &

    Fem-Pop


    Last slides l.jpg

    … last slides...


    Other virtual laboratory applications @ uva l.jpg

    Other Virtual Laboratory Applications @ UvA

    Computing in Physics

    Computing in

    Engineering

    Computing in Engineering

    Bio-medical

    Computation

    Bio- informatics

    Environment

    Cultural Inheritance Environment

    VL for Material Science

    Traffic Payment for mobility

    Apply VL in non-quality of service environment

    Study of blood flow through veins

    DNA

    Research

    Art objects

    preservation

    restoration

    Meta data

    Integration

    Combining problem solving & data intensive environments

    Modeling VL in non-QoS situation environment

    Integration of simulation & visualization by man in the loop

    Combing data

    mining & intelligent

    data bases

    Collaborative data

    integration

    User

    User

    Central-part

    Central-part

    Virtual Laboratory

    Virtual Laboratory

    ViSEComColVIMCO

    ViSEComColVIMCO

    Physical Apparatus

    Internet and Web Software

    Internet and Web Software

    Distributed Computer infrastructure

    Distributed Computer infrastructure


    Acknowledgements l.jpg

    Acknowledgements

    RUL/AZL:

    H. Reiber, PhD.

    Bloem, PhD, M.D.

    SARA:

    A. de Koning, PhD.

    Arcobel:

    S. ten Den

    IBM:

    J. Geise

    Stanford:

    Charley Taylor, PhD.

    Christopher K. Zarins, PhD. M.D.

    UvA:

    Robert Belleman

    Alfons Hoekstra, PhD

    Dick van Albada, PhD

    Benno Overeinder, PhD

    Krakow

    Marian Bubak, PhD

    Kamil Iskra


    Support l.jpg

    Support

    IBM

    SARA

    SGI

    Platform HPCN

    ICES-KIS-1

    ICES-KIS-2

    KNAW

    NWO/FOM


    Slide56 l.jpg

    http://science.uva.nl/~sloot

    sloot@science.uva.nl


    Slide58 l.jpg

    MFlop/s

    ?

    ASCI-Blue

    1000.000

    ASCI-Red

    Structural Biology

    CM-5

    100.000

    Pharmaceutical

    10.000

    72 hr Weather

    Cray Y-MP

    48 hr Weather

    1000

    Cray X-MP

    2D Plasma

    100

    Oil reservoir

    10

    CDC 6600

    1

    IBM 704

    0.1

    1955

    1965

    1975

    1985

    1995

    2005


    Results mean flow rates ml min rest l.jpg

    Results - Mean Flow Rates (ml/min) - Rest


    Cellular automata l.jpg

    Cellular Automata

    • 1966 Introduced by John von Neumann

    • 1985 Stephen Wolfram suggested CA are capable of Universal Computation

    • 1990 Lindgren et al., proved UC in 1D CA


    Slide61 l.jpg

    Productie Regel 110

    t=0

    0

    1

    0

    1

    0

    0

    1

    1

    0

    t=1

    1

    1

    1

    0

    1

    1

    1

    0

    1

    100

    0

    111

    0

    110

    1

    101

    1

    011

    1

    010

    1

    001

    1

    000

    0


    Slide62 l.jpg

    Time Evolution of 1D Cellular Automata 110

    Back to Mesoscopic Models


    The lattice gas model l.jpg

    The Lattice Gas model

    • Fluid model with Cellular Automata rules

    • Collision: particles reshuffle velocities

    • Imposed Constraints

      • Conservation of mass

      • Conservation of momentum

      • Isotropy


    The hexagonal lattice l.jpg

    The Hexagonal Lattice


    Collision rules examples l.jpg

    Collision rules examples

    Two body collision

    N1 AND N4 => N2 AND N5 && N3 AND N6

    Three body collision

    N2 AND N4 AND N6 => N1 AND N3 AND N5


    Streaming and colliding l.jpg

    Streaming and Colliding


    From lga to lbm l.jpg

    From LGA to LBM

    • Average LGA equation to get continuous values instead of boolean values

    • Boltzmann molecular chaos assumption to factorize products in collision operator:

      => Iterate:


    From micro dynamics to macro dynamics 1 l.jpg

    From Micro Dynamics to Macro Dynamics (1)

    • Taylor expansion to get continuous differential operators:


    From micro dynamics to macro dynamics 2 l.jpg

    From Micro Dynamics to Macro Dynamics (2)

    • Chapman Enskog expansion of equilibrium Distribution Function:

    • With imposed constraints:


    From micro dynamics to macro dynamics 3 l.jpg

    From Micro Dynamics to Macro Dynamics (3)

    • Multi-scale expansion of time and space derivatives:

    • Solve collision/flow equation for different order of 


    Slide71 l.jpg

    Back to mesoscopic models


  • Login