1 / 55

Basic Concepts of FEM Framework & API

Basic Concepts of FEM Framework & API. Gunavardhan Kakulapati ( kakulapa@cs.uiuc.edu ). Introduction FEM Framework overview Using the framework API Installation Conversion Example. Motivation. Finite Element Method Used extensively Computationally intensive Do it in parallel !!.

damara
Download Presentation

Basic Concepts of FEM Framework & API

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Basic Concepts of FEM Framework & API Gunavardhan Kakulapati (kakulapa@cs.uiuc.edu)

  2. Introduction • FEM Framework overview • Using the framework API • Installation • Conversion Example

  3. Motivation • Finite Element Method • Used extensively • Computationally intensive • Do it in parallel !!

  4. FEM Programs • FEM programs manipulate elements and nodes • Element is a portion of problem domain, surrounded by nodes • Element computation: based on fields of surrounding nodes • Elements contribute to fields of surrounding nodes

  5. FEM Mesh

  6. FEM Program Structure read mesh, connectivity, boundary conditions time loop           element loop- Element deformation applies forces to surrounding nodes           node loop- Forces and boundary conditions change node positions end time loop write out mesh data for postprocessing

  7. Parallelization • Partition the FEM Mesh into multiple chunks • Distribute elements, replicate shared nodes • Shared nodes = Communication !! • Partition so that communication is minimized

  8. Partitioning

  9. Parallel FEM Program read/get chunk mesh data, connectivity, shared nodes chunk time loop           element loop- Element deformation applies forces to surrounding nodes <update forces on shared nodes>          node loop- Forces and boundary conditions change node positions end time loop write chunk mesh data for postprocessing >>

  10. FEM Framework: Goals • Separate parallel implementation from numerical algorithms • Parallel version to closely resemble the serial program • Allow features like load balancing, visualization.

  11. FEM Framework: Responsibilities FEM Application (Initialize, Registration of Nodal Attributes, Loops Over Elements, Finalize) FEM Framework (Update of Nodal properties, Reductions over nodes or partitions) Partitioner Combiner METIS Charm++ (Dynamic Load Balancing, Communication) I/O

  12. FEM Framework Program • May contain user-written, library-called subroutines: • init • driver • mesh_updated (more on this later…) • init andmesh_updated are called on processor 0 • driver is called on every chunk

  13. Structure of an FEM Application init() Update driver Update driver Update driver Shared Nodes Shared Nodes

  14. init subroutine init          read the serial mesh and configuration data inform the framework about the mesh end subroutine

  15. driver subroutine driver          get local mesh chunk           time loop                FEM computations update shared node fields               more FEM computations           end time loop end subroutine

  16. Data output • Parallel output at the end of driver • Possible to visualize the data rather than printing it out (netfem). • Framework calls like FEM_Mesh_updated. (more on this later …) >>

  17. Framework Calls • FEM_Set_* • Called from initialization to set the serial mesh • Framework partitions mesh into chunks • FEM_Create_field • Registers a node data field with the framework, supports user data types • FEM_Update_field • Updates node data field across all processors • Handles all parallel communication • Other parallel calls (Reductions, etc.)

  18. Framework Calls: Mesh • FEM_Set_* • From init, these routines describe the mesh to the FEM Framework • Framework calls the partitioner • FEM_Get_* • From each chunk, the local portion of the mesh is obtained by the driver

  19. FEM_*_elem FEM_Set_elem(int elType,int nEl, int doublePerEl,int nodePerEl); subroutine FEM_Set_elem(elType,nEl,doublePerEl,nodePerEl) integer, intent(in) :: elType,nEl,doublePerEl,nodePerEl FEM_Get_elem(int elType,int* nEl, int* doublePerEl,int* nodePerEl); subroutine FEM_Get_elem(elType,nEl,doublePerEl,nodePerEl) integer, intent(in) :: elType integer, intent(out) :: nEl,doublePerEl,nodePerEl

  20. FEM_*_node subroutine FEM_Set_node(nNode,doublePerNode) integer, intent(in) :: nNode,doublePerNode subroutine FEM_Get_node(nNode,doublePerNode) integer, intent(out) :: nNode,doublePerNode

  21. Element Connectivity subroutine FEM_Set_Elem_Conn_r(elType,conn) integer, intent(in) :: elType integer, intent(in), dimension(nodePerEl,nEl) :: conn subroutine FEM_Get_Elem_Conn_r(elType,conn) integer, intent(in) :: elType integer, intent(out), dimension(nodePerEl,nEl) :: conn subroutine FEM_Set_Elem_Conn_c(elType,conn) integer, intent(in) :: elType integer, intent(in), dimension(nEl,nodePerEl) :: conn subroutine FEM_Get_Elem_Conn_c(elType,conn) integer, intent(in) :: elType integer, intent(out), dimension(nEl,nodePerEl) :: conn

  22. Additional Data for Nodes and Elements subroutine FEM_Set_node_data_r(data) REAL*8, intent(in), dimension(doublePerNode,nNode) :: data subroutine FEM_Get_node_data_r(data) REAL*8, intent(out), dimension(doublePerNode,nNode) :: data subroutine FEM_Set_elem_data_r(data) REAL*8, intent(in), dimension(doublePerElem,nElem) :: data subroutine FEM_Get_elem_data_r(data) REAL*8, intent(out), dimension(doublePerElem,nElem) :: data

  23. Node Fields • Framework handles combining data for shared nodes and keeps them in sync • Framework does not understand meaning of node fields, only their location and types • Framework needs to be informed of locations and types of fields • Create_field once, Update_field every timestep

  24. FEM_Create_field • To handle the updating of shared node values. • Tell the framework where the shared data items of each node are. • Creates a “field” and pass the field ID for updating shared nodal values.

  25. FEM_Create_simple_field function integer :: FEM_Create_simple_field( base_type, vec_len) integer, intent(in) :: base_type, vec_len • Base_type • FEM_BYTE- INTEGER*1, or CHARACTER*1 • FEM_INT- INTEGER*4 • FEM_REAL- REAL*4 • FEM_DOUBLE- DOUBLE PRECISION, or REAL*8

  26. Create_simple_field Example ! 3D Force for each node ! stored as 3*n real*8 array REAL*8 ALLOCATABLE, DIMENSION(:) :: nodeForce INTEGER :: fid ... allocate nodeForce as 3*n_nodes... fid = FEM_Create_simple_field(FEM_DOUBLE,3)

  27. FEM_Create_field function integer :: FEM_Create_Field(base_type, vec_len, offset, dist) integer, intent(in) :: base_type, vec_len, offset, dist

  28. Node Fields

  29. Create_field Example ! 3D force is contained as fXYZ variable ! in a user-defined type node_type TYPE(node_type), ALLOCATABLE, DIMENSION(:) :: nodes INTEGER :: fid ...allocate nodes array as n_nodes... fid = FEM_Create_Field(FEM_DOUBLE,3, offsetof(nodes(1), nodes(1)%fXYZ), offsetof(nodes(1), nodes(2)) )

  30. Update and Reduce Field • subroutine FEM_Update_Field(fid,nodes) integer, intent(in) :: fid varies, intent(inout) :: nodes • subroutine FEM_Reduce_Field(fid,nodes,outVal,op) integer, intent(in) :: fid,op varies, intent(in) :: nodes varies, intent(out) :: outVal • op is • FEM_SUM • FEM_MIN • FEM_MAX

  31. Utility function integer :: FEM_Num_Partitions() function integer :: FEM_My_Partition() function double precision :: FEM_Timer() subroutine FEM_Print_Partition() subroutine FEM_Print(str) character*, intent(in) :: str

  32. Advanced FEM calls • FEM_Update_mesh • Reassembles chunks of the mesh • FEM_Add_node • Adds a new node into the mesh

  33. FEM_Update_mesh • Reassembles all the chunks • Can be called only from driver • Must be called from all chunks • Useful scenarios • Giving out data as the simulation runs • Repartition the mesh • Serial output when simulation finishes

  34. FEM_Update_mesh C call: void FEM_Update_mesh(int callMeshUpdated, int doWhat) Fortran call: subroutine FEM_Update_mesh(callMeshUpdated,doWhat) integer, intent(in) :: callMeshUpdated, doWhat

  35. FEM_Update_mesh parameters • callMeshUpdated is non-zero => call mesh_updated (callMeshUpdated) • doWhat:

  36. FEM_Update_mesh examples • FEM_Update_mesh(k,0) • Call mesh_updated(k) on assembled mesh, while the driver continues • FEM_Update_mesh(k,1) • Repartition after mesh_updated • FEM_Update_mesh(k,2) • Block driver routines till mesh_updated(k)

  37. Adding Nodes • One can add new nodes, and update connectivity in driver • Use FEM_Set_* subroutines • New nodes are considered private • Framework can repartition the mesh • Optionally calls user’s mesh_updated subroutine

  38. FEM_Add_node C call: void FEM_Add_node(int localIdx, int nBetween, int * betweenNodes) Fortran call: subroutine FEM_Add_node(localIdx,nBetween,betweenNodes) integer, intent(in) :: localIdx,nBetween integer, intent(in) :: betweenNodes(nBetween) >>

  39. Installing FEM Framework

  40. Where to Get It ? FEM Framework is included in Charm++ distribution, available under CVS CSH: setenv CVSROOT ":pserver:checkout@charm.cs.uiuc.edu:/cvsroot" Or BASH: export CVSROOT=":pserver:checkout@charm.cs.uiuc.edu:/cvsroot" You should now be able to do a > cvs login(no password needed, just type [Enter] at prompt) and then > cvs co -P charm to get the entire Charm++ source.

  41. How to Build It ? > cd charm and do > ./build FEM net-linux -O This will make a net-linux directory, with bin, include, lib etc subdirectories. Platforms: net-sol, mpi-origin, mpi-linux etc.

  42. How to Write Programs ? • Write from scratch • Concepts and API discussed earlier • Read the FEM-Framework Manual http://charm.cs.uiuc.edu/manuals/fem • Convert existing program • To be covered in next section

  43. How to Compile & Link ? • Use “charmc”: available under bin • a multi-lingual compiler driver, understands f90 • Knows where modules and libraries are • Portable across machines and compilers • Linking • use “-language femf” : for F90 • Use “–language fem” : for C/C++ • See example Makefiles • pgms/charm++/fem/…

  44. How to Run ? • Just run it! (net- versions only) • Serial, but nice for debugging/testing • Use Charmrun • A portable parallel job execution script • Specify number of processors: +pN • Special “nodelist” file for net-* versions • Multiple chunks per processor: use +vpM

  45. Charmrun Example Nodelist File: $(HOME)/.nodelist group main host tur0001.cs.uiuc.edu host tur0002.cs.uiuc.edu host tur0003.cs.uiuc.edu etc… >> ./charmrun pgm +p4 +vp8

  46. FEM Framework Conversion example

  47. A Serial Program Processing Input Output

  48. A Parallel Framework Program Parallel Processing Input Output Parallel Infrastructure

  49. Real Names of Pieces Driver Init Finalize or Mesh_updated Charm++ FEM Framework

  50. Serial Example Program • F90 example • Reads input mesh in “Triangle” format • Does simple explicit mechanics computation (CST triangles) • Writes Tecplot output • Not a toy example (by Philippe Geubelle) • Reads a parameter file • Applies boundary conditions • Uses real mechanics

More Related