The amp backplane
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

The AMP Backplane PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

The AMP Backplane. Discreet Management of Numerical Libraries and Multiphysics Data. Bill Cochran Oak Ridge National Laboratory [email protected] The AMP Backplane. Developers:. Collaborators:. Advisors:. Oak Ridge National Lab. Argonne National Lab. Oak Ridge National Lab.

Download Presentation

The AMP Backplane

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The amp backplane

The AMP Backplane

Discreet Management of Numerical Libraries and Multiphysics Data

Bill Cochran

Oak Ridge National Laboratory

[email protected]


The amp backplane1

The AMP Backplane

Developers:

Collaborators:

Advisors:

Oak Ridge National Lab

Argonne National Lab

Oak Ridge National Lab

Los Alamos National Lab

Oak Ridge National Lab

Argonne National Lab

Idaho National Lab

Los AlamosNational Lab

AbdellatifYacout

Marius Stan

Larry Ott

John Turner

Mike Rogers

Kevin Clarno

Bobby Philip

Bill Cochran

SrdjanSimunovic

RahulSampath

SrikanthAllu

GokanYesilyurt

Jung Ho Lee

James Banfield

PallabBarai

SreekanthPannala

PhaniNukala

Larry Ott

Jay Billings

Richard Martineau

Glen Hansen

SametKadioglu

Ray Berry

Cetin Unal

Steven Lee

Los Alamos National Lab

Gary Dilts

BogdanMihaila

Discreet Management of Numerical Libraries and Multiphysics Data


The amp backplane

The AMP Backplane

Vectors

Epetra_Vectorx;

Vecy;

N_Vectorz;

VecAXPBYPCZ ( z , alpha , 1 , 0 , x , y );


The amp backplane

Why So Many Libraries?

AMP uses:

Contact

Preconditioning

Moertel and ML

SNES and KSP

JFNK

IDA

Time integration


The amp backplane

The AMP Backplane

Mechanics

Vectors

Temperature

Matrices

Oxygen Diffusion

Burn Up

Neutronics

Etc.

Epetra_CrsMatrix P;

Mat A;

N_Vectorx, y, z;

P.Multiply ( false , x , y );

MatMult ( A , y , z );

stk::mesh::EntitycurElement;

libMesh::FE integrator;

integrator.reinit ( &curElement );

Epetra_Vectorx;

Vecy;

N_Vectorz;

VecAXPBYPCZ ( z , alpha , 1 , 0 , x , y );

Meshes

VecmultiPhysicsSolution;

VectempPellet, displacementPellet;

VecthermoMechanicsPellet;

SolveThermoMechanics ( thermoMechanicsPellet );


The amp backplane

How Does It Work?

Virtual methods

TheLessYouKnow

Polymorphism

Templates

Iterators

Standard template library


The amp backplane

How Do I Use It?

Master six classes

Linear combinations,

Norms,

Get/Set

etc.

Matrix-Vector products

Scaling

etc.

Multiple domains

Parallel management

I/O

Space allocation

etc.

Entity iteration

Boundary conditions

Memory management

Vector indexing

etc.

Mapping mesh

entities to indices in

vectors and matrices

Describe desired

memory layout

Index individual physics

AMP::Vector

AMP::Matrix

AMP::MeshManager

AMP::MeshManager::Adapter

AMP::Variable

AMP::DOFMap


The amp backplane

?

In Parallel

How Do I Use It

Step 1: makeConsistent()

Step 2: ???

Step 3: Profit!

Multi-multicore

Multicore


The amp backplane

How Discreet Is It?

AMP::Vector::shared_ptrsundialsView;

sundialsView = AMP::SundialsVector::view ( vector );

N_VectorsundialsVec;

sundialsVec =

sundialsView->castTo<AMP::SundialsVector>().getNVector();

AMP::Vector::shared_ptrepetraView;

epetraView = AMP::EpetraVector::view ( vector );

Epetra_Vector &epetraVec =

epetraView->castTo<AMP::EpetraVector>().getEpetra_Vector();

AMP::Vector::shared_ptrthermalResidual;

AMP::Vector::shared_ptrthermalSolution;

thermalResidual =

residual->subsetVectorForVariable ( temperatureVar );

thermalSolution =

solution->subsetVectorForVariable ( temperatureVar );

AMP::Vector::shared_ptrpetscView;

petscView = AMP::PetscVector::view ( vector );

VecpetscVec;

petscVec = petscView->castTo<AMP::PetscVector>().getVec();

  • Single domain/single physics

  • Default linear algebra engine

  • Hopefully, limitation eased by Tpetra

  • Variables describe

    • Memory layout

    • Physics

    • Discretization

  • Most vector functionality

  • Enough matrix functionality

  • Works with SNES and KSP

  • Most vector functionality

  • Works with IDA


The amp backplane

What About Performance?

C++

Clever compiler optimizations

2) Virtual methods

Iterative access:

FORTRAN-esque speed

L2Norm(), dot(), min(), axpy(), scale(), …

Non-iterative access:

for ( i = 0 ; i != numElems ; i++ )

for ( j = 0 ; j != 8 ; j++ )

vector->addValue ( elem[8*i+j] , phi );


The amp backplane

Digression

Time to perform dot product 2 vectors:

0.05 secs

Virtual method penalty:

50%

Time to perform tight loop virtual method

dot product:

0.075 secs

Dot product # floating point ops:

2n-1

40n-20

Dot product FLOPS (FORTRAN style):

Similar sized matvecw.r.t. FLOPS:

24n-12

matvec cache penalty:

40%


The amp backplane

What About Performance?

C++

Clever compiler optimizations

2) Virtual methods

Iterative access:

FORTRAN-esque speed

Non-iterative access:

for ( i = 0 ; i != numElems ; i++ )

vector->addValues ( 8 , elem + 8*i , phi );

for ( i = 0 ; i != numElems ; i++ )

for ( j = 0 ; j != 8 ; j++ )

vector->addValue ( elem[8*i+j] , phi[j] );


The amp backplane

Does it work?

100,000+ unit tests:

AMP interface

AMP interface vsPETSc

AMP interface vsEpetra

PETSc wrappers

SUNDIALS wrappers

EpetravsPETSc

Various bugs found in development

Single physics, single domain

Multiple physics, single domain

Single physics, multiple domains

Multiple physics, multiple domains

Multiple linear algebra engines

AMP vectors

PETSc views

Sundials views

Serial

Parallel

Views

Clones

Clones of views

Views of clones


The amp backplane

What Can It Do?

SUNDIALS IDA time integration

PETSc SNES JFNK

quasi-static

Trilinos ML

preconditioning


The amp backplane

What Can It Do?

Reading Meshes

21.5k Elements/core

10.75k Elements/core

43k Elements/core

Time (s)

128 domains

(88M Elements)

Superscaling

32 domains

(22M Elements)

Number of cores


The amp backplane

What Can It Do?

Multiphysics

Multidomain

Multicore


The amp backplane

What’s On The Horizon?

“PMPIO” check pointing and restart

Hot swap linear algebra engines

Rudimentary contact search

On-the-fly d.o.f. extraction

Better interface for multi* data


The amp backplane

What’s Left To Do?

Performance testing and tuning

More libraries

Generalized discretizations

Bringing everything together


  • Login