use petsc and slepc package to solve large scale linear system quick start n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Use PETSC and SLEPC package to solve large scale linear system ---quick start PowerPoint Presentation
Download Presentation
Use PETSC and SLEPC package to solve large scale linear system ---quick start

Loading in 2 Seconds...

play fullscreen
1 / 70

Use PETSC and SLEPC package to solve large scale linear system ---quick start - PowerPoint PPT Presentation


  • 383 Views
  • Uploaded on

Use PETSC and SLEPC package to solve large scale linear system ---quick start. Sheng Yi Wang , Department of Mathematics, National Taiwan University 2011/07/29. Outline. PETSC ---Vector ---Matrix ---linear system solver SLEPC --- eigenvalue problem solver PBS .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Use PETSC and SLEPC package to solve large scale linear system ---quick start' - veradis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
use petsc and slepc package to solve large scale linear system quick start

Use PETSC and SLEPC package to solve large scale linear system---quick start

Sheng Yi Wang, Department of Mathematics, National Taiwan University2011/07/29

outline
Outline
  • PETSC

---Vector

---Matrix

---linear system solver

  • SLEPC

---eigenvalue problem solver

  • PBS

PETSC & SLEPC

what is petsc
What is PETSC ??
  • PETSC = Portable Extensible Toolkit for Scientific Computation
  • PETSC is intended for use in large-scale application projects.
  • PETSC support MPI, that can parallel execute.
  • PETSC can interfaces many external software

ex: Matlab, Mathematica, MUMPS…etc

PETSC & SLEPC

what can petsc do
What can PETSC do??
  • Vector operation
  • Matrix operation
  • Linear system ( sparse or dense) Ax=b
  • Nonlinear solver
  • ODE & PDE solve

( steady state or time dependent)

PETSC & SLEPC

user s background
User’s background
  • Some knowledge of parallel (MPI command)
  • You are familiar with C/Fortran language.
  • You are familiar with Linux environment.

PS: PETSC can be installed in Windows

(but you have to install many other packages)

PETSC & SLEPC

how install
How install??
  • Step1 : Set environment variables PETSC_DIR

ex: PETSC_DIR=/home/sywang/petsc-3.1-p8; export PETSC_DIR

  • Step 2. Configure (to check your environment)

---BLAS, C compiler, (MPI) must be installed

ex1: ./configure PETSC_ARCH=intel-64-complex --with-cc=icc --with-fc=ifort --with-debugging=0 --download-c-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex

PETSC & SLEPC

slide7

Ex2: ./configure PETSC_ARCH=intel-64-O3-real --with-cc=gcc --with-fc=gfortran --with-debugging=1 --with-blas-lapack-dir=/opt/intel/mkl/10.2.5.035 --with-mpi-dir=/opt/mpich2 --with-scalar-type=real--download-mumps=1

PETSC & SLEPC

slide8

Step 2: make all

  • Step 3: make test
  • Set PETSC_DIR and PETSC_ARCH to ~/.bashrc
  • If you install PETSC successfully, install SLEPC is very easy

PETSC & SLEPC

slide9

Don’t be afraid, install the PETSC is the most difficult process, after that , all you need to do is the following:

1. Call function

2. Set parameter

3. Show the output and explain the results

PETSC & SLEPC

start using petsc
Start using PETSC

PETSC & SLEPC

declare variables
Declare Variables

Vec sol, rhs;

MatMtx_A;

KSPksp;

PetscInt ii= 10;

PetscScalarvalue[3];

PetscScalarval_rhs;

PetscErrorCodeierr;

PETSC & SLEPC

rough coding
Rough coding

PetscInitialize();

ObjCreate(MPI_comm,&obj);

ObjSetType(obj, );

ObjSetFromOptions(obj, );

ObjSolve(obj, );

ObjGetxxx(obj, );

ObjDestroy(obj);

PetscFinalize()

PETSC & SLEPC

slide14

Vector

PETSC & SLEPC

i vector
I. Vector
  • Vec a
  • VecCreate(MPI_Commcomm,Vec* x)
    • VecCreate(PETSC_COMM_WORLD, &a)
  • VecSetSizes(Vec v, PetscInt n, PetscInt N)
    • VecSetSizes(a,PETSC_DECIDE,20);

PETSC & SLEPC

i vector1
I. Vector
  • VecSet(Vecx,PetscScalar alpha)
    • VecSet(a,1.0)
  • VecSetValues(Vecx,PetscIntni,constPetscInt ix[],const PetscScalar y[],InsertModeiora)
    • VecSetValues(a,1,0,-3, INSERT_VALUES)

InsertMode : INSERT_VALUES, ADD_VALUES

PETSC & SLEPC

i vector2
I. Vector
  • VecSetRandom(Vecx,PetscRandomrctx)
    • VecSetRandom(b,r)
  • VecSetFromOptions(Vecvec)
    • VecSetFromOptions(a)
  • VecDuplicate(Vecv,Vec *newv)
    • VecDuplicate(a,&b)

PETSC & SLEPC

i vector3
I. Vector
  • VecView(a,PETSC_VIEWER_STDOUT_WORLD);
  • VecAssemblyBegin(a);
  • VecAssemblyEnd(a);
  • VecDestroy(a);

PETSC & SLEPC

example presentation
Example presentation
  • Example 1:

Create Vec and use some basic vector operation

PETSC & SLEPC

slide21

Matrix

PETSC & SLEPC

2 matrix
2. Matrix
  • Mat mtx_a
  • MatCreate(MPI_Commcomm,Mat* A)
    • MatCreate(PETSC_SOMM_WORLD,&mtx_a)
  • MatSetSizes(Mat A,intm,intn,intM,int N)
    • MatSetSizes(mtx_a,PETSC_DECIDE, PETSC_DECIDE,10,10)
  • MatSetFromOptions(Mat A)
    • MatSetFromOptions(mtx_a)

PETSC & SLEPC

2 matrix1
2. Matrix
  • MatSetValues(Mat A,PetscIntm,const

PetscIntidxm[],PetscIntn,constPetscInt

idxn[],const PetscScalar v[],InsertMode

addv)

    • MatSetValues(mtx_a,1,1,1,1,2.0,INSERT_VALUE)

PETSC & SLEPC

example
Example
  • I have a small matrix s_a [1 2 ;3 4]

but I want to insert s_a into global matrix mtx_a and the index is (1,2)

MatSetValues(mtx_a,2,1,2,2,s_a,INSERT_VALUE)

0 0 0 0 0

0 0 1 2 0

0 0 3 4 0

0 0 0 0 0

PETSC & SLEPC

2 matrix2
2. Matrix
  • MatAssemblyBegin(Mat, MAT_FINAL_ASSEMBLY)
  • MatAssemblyEnd(Mat ,MAT_FINAL_ASSEMBLY)
  • MatDestroy(Mat )

PETSC & SLEPC

2 matrix matrix free
**2. Matrix: Matrix-Free
  • If you don’t want to create all matrix ( the matrix is too large),PETSC allow users write their own matrix operator
  • extern intmult(Mat,Vec,Vec);

MatCreateShell(comm,m,n,M,N,ctx,&mat); MatShellSetOperation(mat,MATOP_MULT,(void(*)(void))mult);

MatDestroy(mat);

PETSC & SLEPC

2 read matrix from file
2. Read matrix from file
  • In many case, someone has created the matrix, so we don’t rewrite the matrix, we just want to read the matrix and solve them quickly, how to do that??
  • If you have matrix which is ASCII format(just store the nonzero element , i , j, aij)

we can call PETSC function to transfer ASCII to binary file ( PETSC can read them very quickly)

PETSC & SLEPC

binary file
Binary file
  • PetscViewerBinaryOpen(MPI_Commcomm,const char name[],PetscViewerFileTypetype,PetscViewer *binv)
    • PetscViewerBinaryOpen(PETSC_COMM_WORLD,fileout,FILE_MODE_WRITE,&view)

PETSC & SLEPC

example presentation1
Example presentation
  • Example 2:

Read matrix in ASCII format and transfer them to PETSC binary file

  • Example 3:

Read matrix from PETSC binary file and create a vector and use a basic matrix operation

PETSC & SLEPC

slide31

Linear System (KSP)

PETSC & SLEPC

3 linear system solver ksp
3.Linear system solver : KSP
  • Linear system problem: give matrix A and vector b  solve Ax=b
  • The dimension is too large to find inverse, and the matrix is sparse, so we need to use iterative method to solve them.
  • The basic method to solve linear system is Krylov subspace methods.

PETSC & SLEPC

3 linear system solver ksp1
3. Linear system solver : KSP

First of all, set the matrix A and rhs vector b

Declare Variables KSP ksp

  • KSPCreate(MPI_Commcomm,KSP *ksp)
    • KSPCreate(PETSC_COMM_WORLD,&ksp)
  • KSPSetOperators(KSP ksp,MatAmat,MatPmat,MatStructure flag)
    • KSPSetOperators(ksp,A,A, SAME_NONZERO_PATTERN)

PETSC & SLEPC

3 linear system solver ksp2
3. Linear system solver : KSP
  • KSPSetType(KSP ksp, const KSPType type)
    • KSPSetType(ksp,KSPGMRES) -ksp_type
  • KSPSetFromOptions(KSP ksp)
    • KSPSetFromOptions(ksp)
  • KSPSolve(ksp,Vecb,Vec x)
    • KSPSolve(ksp,b,x)
  • KSPGetIterationNumber(KSP ksp,PetscInt *its)
    • KSPGetIterationNumber(ksp,&it)

PETSC & SLEPC

preconditioner
Preconditioner
  • In many case, the matrix A has large condition number, so we need use a proconditoner to reduce condition number
  • PETSC provide many preconditioner, and some of them can use MPI to parallel.

PETSC & SLEPC

3 linear system solver ksp3
3. Linear system solver : KSP
  • KSPSetFromOptions(KSP ksp)
    • KSPSetFromOptions(ksp)
  • KSPGetPC(KSP ksp,PC *pc)
    • KSPGetPC(ksp,&pc)
  • PCSetType(PC pc, PCType type)
    • PCSetType(pc,PCBJACOBI) -pc_type

PETSC & SLEPC

how to parallel
How to Parallel ??
  • MPI- Message Passing Interface

PETSC and SLEPC support MPI and user don’t have to call MPI function

    • mpiexec -np 4 yourprogram
    • mpirun -np 8 -machinefile mf yourprogram
  • PETSC_COMM_SELF PETSC_COMM_WORLD
  • PETSC and SLEPC initial will include mpi.h, so if you want to use MPI function,you can use them, too

PETSC & SLEPC

a ssign parameters at run time
Assign parameters at run time
  • Serial :
    • ./ex4 -ksp_typebcgs -pc_typelu -ksp_rtol 1e-4
    • ./ex4 -ksp_typebcgs -pc_typelu -ksp_rtol 1e-4
    • ./ex4 -ksp_typegmres -pc_typeasm -ksp_rtol 1e-8 -ksp_max_it 20
  • Parallel:

In one node :

    • mpiexec –np 4 ./ex4 -ksp_type cg -pc_typesor - pc_sor_omega 1.8 -ksp_rtol 1e-4 -ksp_view

In multi nodes:

    • mpiexec –np 16 –machinefile mf ./ex4 -ksp_type cg -pc_typeasm -ksp_rtol 1e-4 -ksp_view -ksp_monitor_true_residual

PETSC & SLEPC

example presentation2
Example presentation
  • Example 4 :

Read matrix from a binary file, and call PETSC

function to solve linear system. And show how

to assign parameters at run time

PETSC & SLEPC

makefile
makefile

include ${PETSC_DIR}/conf/base

linearsym : linearsym.ochkopts

-${CLINKER} –o linearsymlinearsym.o{PETSC_LIB}

${RM} linearsym.o

PETSC & SLEPC

bash script
Bash script

#!/bin/bash

for((i=10;i<=30;i=i+5))

do

./linearsym -n $i -ksp_typegmres -pc_typejacobi

-ksp_max_it 100 -ksp_view > result_gmres_$i

done

for((i=10;i<=30;i=i+5))

do

./linearsym -n $i -ksp_type cg -pc_typejacobi

-ksp_max_it 100 -ksp_view > result_cg_$i

done

PETSC & SLEPC

perl bash script like
Perl (bash script like)
  • #!/usr/bin/perl

for ($i=1; $i<=199; $i++) {

$sor_omega=0.01*$i;

system("~/program/openmpi2/bin/mpiexec -np $i -machinefile mf10 ex4 -fin m22103 –ksp_type cg –pc_typesor –pc_sor_omega$sor_omega-log_summary > m22103_cg_sor_omega$sor_omega");

system("echo $i");

sleep(3);}

PETSC & SLEPC

slide45

Eigenvalue (EPS)

PETSC & SLEPC

4 eigenvalue problem solver slepc
4.Eigenvalue Problem Solver: SLEPC
  • SLEPC the Scalable Library for Eigenvalue Problem Computations
  • Standard Eigenvalue problem :

give matrix A, want to find unknown vector x and value k  Ax=kx

  • General Eigenvalue problem :

give matrix A and matrix B, want to find unknown vector x and value k  Ax=kBx

  • Still , the matrix is very large and sparse.

PETSC & SLEPC

how to install slepc
How to install SLEPC
  • Export SLEPC_DIR =/home/sywang/petsc-3.1-p6;
  • ./configure (they will follow the PETSC environmental setting)
  • Make all
  • Make test

PETSC & SLEPC

declare variables1
Declare Variables

EPS eps;

Mat A,B ;

PetscIntii,nn = 10,col[3];

PetscScalar value[3];

PetscScalarkr,ki;

PetscErrorCodeierr;

PETSC & SLEPC

4 eigenvalue problem solver slepc1
4.Eigenvalue Problem Solver: SLEPC
  • EPSCreate(MPI_Commcomm,EPS *eps)
    • EPSCreate(PETSC_COMM_WORLD,eps)
  • EPSSetOperators(EPS eps,MatA,Mat B)
    • EPSSetOperators(eps,A,B)  Ax=kBx
    • EPSSetOperators(eps,A,PETSC_NULL)  Ax=kx
  • EPSSetProblemType(EPS eps,EPSProblemType type)
    • EPSSetProblemType(eps,EPS_HEP) -eps_hermitian
    • EPSSetProblemType(eps,EPS_NHEP) -eps_non_hermitian

PETSC & SLEPC

4 eigenvalue problem solver slepc2
4.Eigenvalue Problem Solver: SLEPC
  • EPSSetType(EPS eps,constEPSType type)
    • EPSSetType(eps,EPSJD) -eps_typejd
  • EPSSetTolerances(EPS eps,PetscRealtol,PetscIntmaxits)
    • EPSSetTolerances(eps,1e-8,200)

-eps_tol 1e-8 –eps_max_it 200

PETSC & SLEPC

eps type
EPS Type

PETSC & SLEPC

4 eigenvalue problem solver slepc3
4.Eigenvalue Problem Solver: SLEPC
  • EPSSetDimensions(EPS eps,PetscIntnev,PetscIntncv)
    • EPSSetDimensions(eps,2,18) -eps_nev 2 -eps_ncv 18
  • EPSCreate(MPI_Commcomm,EPS *eps)
    • EPSSetWhichEigenpairs(eps, EPS_SMALLEST_REAL)

-eps_smallest_real

PETSC & SLEPC

eps which
EPS which

PETSC & SLEPC

4 eigenvalue problem solver slepc4
4.Eigenvalue Problem Solver: SLEPC
  • EPSSolve(eps)
  • EPSView(eps,PETSC_VIEWER_STDOUT_WORLD)

-eps_view

  • EPSGetIterationNumber(eps, &its)
  • EPSGetOperationCounters(EPS eps,PetscInt* ops,PetscInt* dots,PetscInt* lits)
    • EPSGetOperationCounters(eps,PETSC_NULL,PETSC_NULL,&lits)

PETSC & SLEPC

4 eigenvalue problem solver slepc5
4.Eigenvalue Problem Solver: SLEPC
  • EPSGetType(eps,&type)
  • EPSGetDimensions(eps,&nev,&ncv)
  • EPSGetTolerances(eps,&tol,&maxit)
  • EPSGetConverged(eps,&nconv)

PETSC & SLEPC

4 eigenvalue problem solver slepc6
4.Eigenvalue Problem Solver: SLEPC
  • EPSGetEigenpair(EPS eps, PetscInti, PetscScalar *eigr, PetscScalar *eigi, VecVr, Vec Vi)
    • EPSGetEigenpair(eps,i,&kr,&ki,PETSC_NULL,PETSC_NULL)
  • EPSComputeRelativeError(eps,i,&error)
  • EPSDestroy(eps)

PETSC & SLEPC

spectral transformation shift invert
Spectral Transformationshift & invert
  • In many case, the eigenvalue we want is the smallet (but nonzero),if we don’t use shift and invert, it takes many time to find eigenvalue
  • But when you use invert, you will need to solve linear system, they are done by calling PETSC KSP solver

PETSC & SLEPC

spectral transformation
Spectral Transformation
  • EPSCreate(MPI_Commcomm,EPS *eps)
    • EPSCreate(PETSC_COMM_WORLD,eps)
  • ST st;
  • PetscScalar shift = 2.0;
  • EPSGetST(EPS eps,ST* st);
  • STSetShift(ST st,PetscScalar shift);
    • STSetShift(st,2.0); -st_shift
  • STSetType(ST st,STType type);
    • STSetType(st, STSHIFT) -st_type shift

PETSC & SLEPC

spectral transformation1
Spectral Transformation
  • Shift-invert need to solve linear system. We need to call PETSC function
  • In general we, set parameter in command line

ex:

-st_typesinvert -st_ksp_type cg -st_pc_typeasm -st_ksp_rtol 1e-10

-st_typesinvert -st_ksp_typebcgs -st_pc_typesor -st_pc_sor_omega 1.8 -eps_monitor_draw

PETSC & SLEPC

makefile1
makefile
  • all: eigensym

include ${SLEPC_DIR}/conf/slepc_common

eigensym : eigensym.ochkopts

-${CLINKER} -o eigensymeigensym.o{SLEPC_LIB}

${RM} eigensym.o

PETSC & SLEPC

how to get info from petsc and slepc
How to get info from PETSC and SLEPC
  • ./linearsym -ksp_view
  • ./eigensym -eps_view
  • ./linearsym -ksp_monitor
  • ./eigensym -eps_monitor
  • ./eigensym -eps_monitor_draw_all
  • ./linearsym -log_summary

PETSC & SLEPC

example1
Example
  • Example5 :

Read a matrix from a binary file, and find its

eigenvalue

PETSC & SLEPC

slide66
PBS
  • PBS = Portable Batch System
  • PBS is first developed by NASA
  • You need to write a bash script and submit job to the cluster

PETSC & SLEPC

script
Script
  • #PBS -N Job_name
  • #PBS -q queue_name (according to cluster)
  • #PBS -o result_file
  • #PBS -e error_message
  • #PBS -l node=2:ppn=4

#PBS -l nodes=node01:ppn=4+node02:ppn=4

PETSC & SLEPC

slide68
info
  • Submit job : qsubyour_sheell
  • Check job : qstat

qstat -n

  • Check node : pbsmodes

pbsnodes –l free

PETSC & SLEPC

reference
Reference
  • Hands-On Exercises for SLEPC

http://www.grycap.upv.es/slepc/handson/

  • PETSC home page :

http://www.mcs.anl.gov/petsc/petsc-as/

user guide:

http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manual.pdf

  • SLEPC home page :

http://www.grycap.upv.es/slepc/

user guide:

http://www.grycap.upv.es/slepc/documentation/slepc.pdf

  • Matrix market:

http://math.nist.gov/MatrixMarket/

PETSC & SLEPC

slide70

Thank You

PETSC & SLEPC