COMPUTATIONAL FLUID DYNAMICS: MODULAR ELEMENTS FOR CACTUS?

1. COMPUTATIONAL FLUID DYNAMICS: MODULAR ELEMENTS FOR CACTUS? Sumanta Acharya L. R. Daniel Professor & Director Turbine Innovation & Energy Research Center College of Engineering Acknowledgements Frank Muldoon, Mayank Tyagi NASA, ONR, CCT, BOR

2. Motivation • Fluid flow controls all aspects of our lives!! • Human body • Drug manufacture & delivery, DNA Analysis • Transportation • Entertainment & Sports • Energy • Chemical, petrochemical, refining • Defense • Many others!! • Research in fluid dynamics is driven by a need to survive on EARTH!

3. Physics, Mathematics and Simulations of Fluid Turbulence • Fluid flow is governed by the Navier-Stokes Equations. So we can compute? • Phenomenology of Turbulence • Stochastic (Randomness) • Non-linear • Dissipative • Enhanced Diffusion

4. SIMULATIONS OF FLUID TURBULENCE Simulation of turbulent flows • Direct Numerical Simulation (DNS) • Resolves all the scales of turbulent flows • Extremely computational intensive • Large eddy simulation (LES) • Resolves large energy containing scales and models small scales • Less computationally intensive as compared to DNS • Reynolds-averaged Navier-stokes (RANS) models • Solves ensemble averaged equations and models all the scales • Most widely used in industries Scales in spectrum of turbulent flow field DNS: Resolved up to kd LES: Resolved up to kr and filters the scales larger than kc RANS: Models the entire spectrum

5. Direct Numerical Simulations of Turbulent Flows • No modeling needed (Needs better schemes though) • Computationally prohibitively expensive (Degrees of freedom active in a turbulent flow at Reynolds number Re are approximately Re9/4 (upper bound)) • Motivation for Grid-computing & more efficient algorithms. Fastest machine too slow; Largest cluster too small. For flow around a car the smallest eddies are 10-5 meters in size Direct numerical simulation of this flow 9×10¹¹ years of computing time& 2×10¹¹Gbytes of storage For this reason most flows of industrial interest cannot be solved using Navier-Stokes equations

6. Turbulence Modeling using RANS Closure • Mathematically sound but lacks physics (Hierarchy of moment equations is NOT closed and one must invoke closure hypothesis always at some agreeable level) • Not-universal (All kinds of approximations are made that may not be valid in different scenarios) • Will be widely used in industry for foreseeable future (Computationally affordable for really complex problems of industrial interests) DNS or LES Best RANS? Muldoon & Acharya, 2004

7. Navier-Stokes Equations DNS (months) RANS (hrs) LES (days) Need faster machines, more numbers of them, and better algorithms to move from a world of “blur” to a world of “clear vision”.

8. Elements of a CFD Algorithm • Models: Differential Equation L(u)=f • Grid Generation: Cartesian, block body-fitted, adaptive mesh refinement, Unstructured; Immersed Boundary Methods; Adaptive time-stepping • Discretization: Finite differences, finite elements, spectral schemes; AU=b; Higher order schemes, Compact schemes, Upwind, TVD, Adaptive discretization • Boundary Conditions:Dirichelet, Neumann, Periodic, Convective • Solvers: Linearized system of coupled algebraic equations-sparse matrices AU=b. Explicit schemes, Implicit Elliptic solvers: Direct, Iterative (GMRES, CG, Line-relaxation); Multi-Grid • Pressure-velocityCoupling:Poisson Equations, (Pseudo)-Compressible Formulations, Matrix pre-conditioning • Parallelization Issues: Domain Decomposition, Ghost Cells, Message Passing • I/O • Data Processing & Visualization

9. DISCRETIZATION Mass Conservation Momentum Conservation Note for Cactus: Interpolation stencils & order Convection Term Diffusion Term Note for Cactus: Felxiblity in choosing order and stencil Staggered Grid vs. Collocated Grid Note for Cactus: Array indexing

10. Note for Cactus: Felxiblity in choosing order and stencil Design code for higher order stencils Implementation-specify stencil locations and order

11. Remarks on Fornberg Algorithm-Module • Recursive relation for the finite-difference weights on arbitrary grids • The Fornberg Module can be used for interpolation as well as nth- derivatives for desired order of accuracy (provided that sufficient data is given) • Simple and Ingenious

12. Large Eddy Simulations (LES) Homogeneously filtered incompressible Navier-Stokes equations • Subgrid Scale Modeling • Filtering

13. Examples of some filters • Compact or Pade filters • Fourier cut-off (or use some transfer function to do convolution in fourier space) • Averaging (cell-volume) • Quadrature rules (Tensorial product in multi-dimensions for Simpson’s or Trapezoidal Rule) Typical filters used in finite-difference calculations A volume-averaging 2Δ width filter Cactus: Filtering leads to weighted averaging or interpolation. Use interpolation thorns

14. Remarks on Filter-module • Need explicit filters for finite-difference calculations • Need filters for Large-eddy simulations methodology • Filters are subjected to several physical and mathematical constraints

15. Pressure-Poisson Equation: Basic Details of Fractional-Step Scheme Fractional Step Scheme Convection, Diffusion Terms Continuity Constraint Pressure-Poisson Equation

16. Pressure Poisson Solver: Assembly of Thorns Expand Poisson’s Equation Take Fourier Transform in z-direction Write differential operators in Matrix form Absorb wavenumbers in one of the operators (say Y) Introduce Eigenvalue Decomposition Simple matrix manipulations by pre/post multiplying the appropriate operators Take Inverse Fourier transform of the last step to get the solution

17. Remarks on Pressure-Poisson equation-Module • Parallel direct sparse solver (PSPASES) • Parallel (fast and efficient) elliptic solvers of any kind • Parallelization using domain decomposition (Influence matrix approach) Remarks on Pressure-Velocity Coupling Strategies • Poisson Equation-Direct Solver • Poisson Equation-Iterative Solver • Pseudo-Compressibility Approach • SCGS, SCGS-PP, SCGS-PPV • SIMPLE, SIMPLER, SIMPLEC, SIMPLEM (Moukalled) • Note to Cactus: Thorns for different solution strategies; specify which during compilation

18. Parallelization • Straightfoward Domain decomposition is to simply break up the Cartesian grid into blocks of approximately the same size. Need thorn for domain decomposition. • Give each process ghost cells which are filled using information communicated from the neighbors. • The size of these ghost cells (and therefore the size of the data to be communicated) depends on the order of the schemes used. Need thorn for automatically assigning ghost cells based on the order of scheme. • Use calls to the MPI library to explicitly send and receive data.

19. Domain decomposition

20. Adaptive Mesh Strategies • Concept of Springs with stiffness parameters (Harvey and Acharya) • Concept of truncation error estimate defining region to be flagged for further refinement. Flagged region is then re-gridded leading to embedded grids with potential multi-grid strategies (Moukalled and Acharya) • Concept of Adaptive Differencing schemes (Rhodes and Acharya) • Error estimator • Surface Mesh • Metrics • Poisson Solver Harvey & Acharya, 1991 Moukalled & Acharya, 1980’s

21. Block Body Fitted Grids

22. Immersed Boundary Method • Poisson Equation Thorn • Interpolation Thorn

23. Different Interpolation Strategies for Immersed Boundary forcing Vim = (V1/h1+V2/h2)/(1/h1+1/h2) Vm : Computed Velocity influencing the immersed point Vim : Immersed Velocity hm : Distance between immersed point and calculated point Vint = Vd [2]/[ 2 – ] – Vc []/[ 2 – ] Vext = Vd []/[ 2 – ] + Vc [ – ]]/[ 2 – ]

24. Flow past heated circular cylinder in a channel at ReD = 100 Validation of Immersed Boundary Method Temperature Vorticity Table 1: Comparison of experimental and computed values of drag coefficients, Nusselt number and Strouhal number (a: White (1990), b: Incropera and Dewitt (1990)).

25. Applications of LES in Turbomachinery Stator-Rotor Interaction Film Cooling Internal Cooling

26. 95 Cylinders array (Staggered Arrangement) Handling Complex Geometries With IBM Velocity Vectors Pressure Contours

27. Biological Application: Suspension Feeding by a Mussel

28. Algorithm to spread influence of body force (Lagrangian) on grid nodes (Eulerian) • Define a curve through the lagrangian points • Find intersections with grid lines and interpolate the flow fields at these intersection points • Calculate the weights for spreading operation between the grid nodes and the intersection points Frontal/Lateral Cilia Latero-Frontal Cirri

29. Simpler Problem: Flow Field around Cilia Row Contours of Streamwise Component of velocity Free Slip Free Slip Free Slip No Slip Free Slip

30. Chemical Industry Application: Mixing in Stirred-Tank Reactors

31. Turbomachinery Application: Unsteady Stator-Rotor Interactions Vorticity Pressure

32. Remarks on Immersed Boundary Method-Module • Need Immersed boundary forcing scheme to solve moving geometries • Need systematic way to introduce immersed boundary points • Applications: Biological -> Chemical Industry -> Turbomachinery