high fidelity terascale simulations of turbulent combustion
Download
Skip this Video
Download Presentation
High-Fidelity Terascale Simulations of Turbulent Combustion

Loading in 2 Seconds...

play fullscreen
1 / 40

High-Fidelity Terascale Simulations of Turbulent Combustion - PowerPoint PPT Presentation


  • 117 Views
  • Uploaded on

High-Fidelity Terascale Simulations of Turbulent Combustion. Jacqueline H. Chen, Evatt R. Hawkes, Ramanan Sankaran, James C. Sutherland, and Joseph C. Oefelein Combustion Research Facility Sandia National Laboratories Livermore CA Supported by

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'High-Fidelity Terascale Simulations of Turbulent Combustion' - ninon


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
high fidelity terascale simulations of turbulent combustion
High-Fidelity Terascale Simulations of Turbulent Combustion

Jacqueline H. Chen, Evatt R. Hawkes, Ramanan Sankaran, James C. Sutherland, and Joseph C. Oefelein

Combustion Research Facility

Sandia National Laboratories

Livermore CA

Supported by

Division of Chemical Sciences, Geosciences, and Biosciences,

Office of Basic Energy Sciences, and DOE SciDAC

Computing: LBNL NERSC, ORNL CCS/NLCF, PNNL MSCF,

SNL HPCNP Infiniband test-bed,

SNL CRF BES Opteron cluster

Visualization: Kwan-Liu Ma and Hiroshi Akiba UC Davis

outline
Outline
  • DNS of turbulent combustion – challenges and opportunities
  • Sandia S3D DNS capability – terascale simulations on Office of Science platforms
  • New combustion science to advance predictive models:
    • 3D simulations of turbulent jet flames with detailed chemistry.
    • Layering Large Eddy Simulation and DNS approaches.

Vorticity fields in DNS of a turbulent jet flame

(volume rendering by

Kwan-Liu Ma and Hiroshi Akiba)

turbulent combustion is a grand challenge
Turbulent combustion is a grand challenge!
  • Stiffness : wide range of length and time scales
    • turbulence
    • flame reaction zone
  • Chemical complexity
    • large number of species and reactions (100’s of species, thousands of reactions)
  • Multi-Physics complexity
    • multiphase (liquid spray, gas phase, soot, surface)
    • thermal radiation
    • acoustics ...
  • All these are tightly coupled

Diesel Engine Autoignition, Soot Incandescence

Chuck Mueller, Sandia National Laboratories

several decades of relevant scales
O(4)

Range

Continuum

O(4)

Several decades of relevant scales
  • Typical range of spatial scales
    • Scale of combustor: 10 – 100 cm
    • Energy containing eddies: 1 – 10 cm
    • Small-scale mixing of eddies: 0.1 – 10 mm
    • Diffusive-scales, flame thickness: 10 – 100 m
    • Molecular interactions, chemical reactions: 1 – 10 nm
  • Spatial and temporal dynamics inherently coupled
  • All scales are relevant and must be resolved or modeled

Terascale computing:

~3 decades in scales

(cold flow)

combustion cfd approaches to tackle different length scale ranges
Increasing cost

and resolution for

fixed physical problem

Combustion CFD Approaches to tackledifferent length scale ranges
  • Reynolds–Averaged Navier–Stokes (RANS)
    • Coarse meshes, full range of dynamic scales modeled
    • Empirical closure, bulk approximation, current engineering CFD
  • Large Eddy Simulation (LES)
    • Energetic scales resolved, sub-grid scale dynamics modeled
    • combustion closure required, captures large-scale unsteadiness
  • Direct Numerical Simulation (DNS)
    • No sub-grid modeling required but limited on range of scales
    • Building-block configurations, research tool
role of direct numerical simulation dns
Oxidizer

Fuel

HO2

CH4

CH3O

O

Role of Direct Numerical Simulation (DNS)
  • A tool for fundamental studies of the micro-physics of turbulent reacting flows
  • A tool for the development and validation of reduced model descriptions used in macro-scale simulations of engineering-level systems
  • Physical insight into chemistry turbulence interactions
  • Full access to time resolved fields

DNS

Engineering-level

CFD codes

(RANS and

future LES)

Physical

Models

DNS

Piston Engines

s3d mpp dns capability at sandia
S3D code characteristics:

Solves compressible reacting Navier-Stokes

F90/F77, MPI, domain decomposition.

Highly scalable and portable

8th order finite-difference spatial

4th order explicit RK integrator

hierarchy of molecular transport models

detailed chemistry

multi-physics (sprays, radiation and soot) from SciDAC “Terascale Simulation of Turbulent Combustion”

70% parallel efficiency on 4096 processors on NERSC; 90% parallel efficiency on 5120 CrayXT3 processors; 75% parallel efficiency on 1000 CrayX1E processors

Terascale computations => need scalar optimization customized to architecture

CCS, NERSC consultants

S3D MPP DNS capability at Sandia

S3D is a state-of-the-art DNS code developed with

13 years of BES sponsorship.

S3D scales up to 1000s of processors… and beyond?

s3d performance improvements on seaborg
S3D performance improvements on Seaborg

Application Software Case Studies in FY05 for MICS (K. Roche)

Using Xprofiler, IPM and working with David Skinner:

  • 8% improvement in performance by minimizing conversion between dimensional and nondimensional units for interfacing with Chemkin
  • interface to Mathematical Acceleration Subsystem (MASS) library for exponential and logarithmic functions resulted in 10% improvement. Another 5% due to using vectorized exp function in VMASS
  • Tabulating Gibbs energy as a function of temperature resulted in 8% savings
  • Remove excessive dynamic allocation calls resulted in 12% savings.
  • Eliminate MPI derived data types for communicating noncontiguous boundary data with neighbors

45% efficiency improvement

outline1
Outline
  • DNS of turbulent combustion – challenges and opportunities
  • Sandia S3D DNS capability – terascale simulations on Office of Science platforms
  • New combustion science to advance predictive models:
    • 3D simulations of turbulent jet flames with detailed chemistry
    • Layering Large Eddy Simulation and DNS approaches

Vorticity fields in DNS of a turbulent jet flame

(volume rendering by

Kwan-Liu Ma and Hiroshi Akiba)

slide10
FY05 INCITE award: Direct simulation of a 3D turbulent nonpremixed CO/H2/air jet flame with detailed chemistry (350 million grids, 12 species)
  • Objectives:
  • Study fine-grained coupling between turbulent mixing and finite-rate chemical effects
    • Extinction-reignition dynamics
    • Mixing time scales
    • Reynolds number effect
    • Preferential diffusion
    • Flow unsteadiness
  • Test assumptions in subgrid mixing and reaction models

z

x

y

x

INCITE run test case (1/2 resolved) 350 million grid points, Re=5000

Isoscalar surface of scalar dissipation rate (local mixing rate)

slide11

The effect of chemistry-turbulence interactions in turbulent nonpremixed jet flames: mixing of passive and reacting scalars

Evatt Hawkes, Ramanan Sankaran,

James Sutherland, and Jacqueline Chen

Computational Platforms: NERSC Seaborg, ORNL CCS Phoenix Cray X1E, XT3 and PNNL MPP2

description of runs temporally evolving non premixed plane jet flames
Air

Mixing,

Reaction

Fuel

Mixing,

Reaction

Air

Description of runs- Temporally evolving non-premixed plane jet flames
  • A canonical flow with shear-driven turbulence

x

y

Heat release iso-contours

dns data sets of turbulent nonpremixed h 2 co flames mixing extinction reignition
Large computing allocations are enabling new science runs

INCITE at NERSC, capability computing at NLCF ORNL, ERCAP at PNNL, NERSC

Detailed H2/CO chemistry (17 d.o.f., Li et al. 2005)

Parameters selected to maximize Re

Case A:

Re 6000

40 million grid points

480 processors on PNNL MPP2

~ 1 TB data

Case B:

Re 8000

100 million grid points

1728 processors on Seaborg

192/240 processors on ORNL CCS Cray X1/E

~ 2.5 TB data

INCITE calculation:

Currently running on Seaborg

Re 5000

350 million grid points

~4096 Seaborg processors

3 million hours total

~ 9TB raw data

Two additional calculations, parametric study in Re

Re 2500, 150 million grid points

Re 10,000, 500 million grid points

DNS data-sets of turbulent nonpremixed H2/CO flames (mixing, extinction/reignition)
community data sets
Community data sets
  • New opportunities for numerical benchmarks – highly resolved LES and DNS
  • TNF workshop (1996-2004): International Collaboration of Experimental and Computational Researchers
    • Framework for detailedcomparison of measured and modeled results (http://www.ca.sandia.gov/TNF/abstract.html)
non premixed combustion concepts
Non-premixed combustion concepts
  • Mixture fraction Z: the amount of fluid from the fuel stream in the mixture
  • Z is a conserved (passive) scalar – (no reactive source term)
  • Scalar dissipation, a measure of local molecular mixing rate:
example development of the jet 40 million grid pnnl run
Example development of the jet:40 million grid PNNL run
  • Left: Vorticity. Right: Simultaneous volume rendering of mixture fraction, scalar dissipation and OH radical.

(Rendering by Hiroshi Akiba and Kwan-Liu Ma, UC Davis)

mixing timescales
Mixing timescales
  • Models for molecular mixing are required in the PDF approach to turbulent combustion (Pope 1985), a sub-grid model used in RANS and LES approaches.
  • TNF workshop – CFD predictions are dependent on mixing timescale choice.
  • Models assume that scalar mixing timescales are identical for all scalars and determined by the turbulence timescale.
    • scalars with different diffusivities?
    • reactive scalars?
definitions
is assumed to be order unity in most models

is assumed to be the same for all scalars

Definitions
  • Mechanical time-scale:
  • Scalar time-scale:
  • Time-scale ratio:
mixture fraction to mechanical timescale ratio
Mixture fraction to mechanical timescale ratio
  • Confirmation that mixture fraction to mechanical time scale ratio is order unity.
  • Average value about 1.5, similar to values reported by experiments, simple chemistry DNS, and used successfully in models.

Timescale Ratio rZ

effect of diffusivity
Effect of diffusivity

Increasing

diffusivity

  • Smaller, more highly diffusive species do have faster mixing timescales
finite rate chemistry effects on mixing
Finite-rate chemistry effects on mixing

  • HO2 and H2O2 have faster mixing times in the middle of the simulation, while OH and O are lower
  • Diffusivity trend does not appear to hold for HO2 and H2O2 versus O and OH.
  • What is going on?
radical production and destruction in high dissipation regions
Radical production and destruction in high dissipation regions

Color scale: mass fraction (blue is low; red is high)

White iso-contours: scalar dissipation rate

OH

  • OH is consumed while HO2 is produced in high dissipation regions

HO2

dissipation of passive and reactive scalars
Dissipation of passive and reactive scalars
  • Blue: Z, Green: OH, Red: HO2
  • Dissipation fields of Z and HO2 are co-incident and aligned with principal strain directions
  • OH dissipation occurs elsewhere, more in the centre of the jet
  • OH is reduced in the high dissipation regions, leading to longer mixing times, opposite for HO2
  • At later times, mixing rates relax, OH returns and HO2 decreases
100 million grid run on seaborg cray x1
100 million grid run on Seaborg / Cray X1
  • We need to establish any Re dependence
  • Need parametrics at larger Re
  • Run in progress… but seems to be showing same effects
conclusions mixing timescales
Conclusions - mixing timescales
  • New finding: detailed transport and chemistry effects can alter the observed mixing timescales
  • Models may need to incorporate these effects
    • a poor mixing model could lead to incorrectly predicting a stable flame when actually extinction occurs
  • This type of information cannot be determined any other way at present
    • ambiguities in a-posteriori model tests
    • too difficult to measure
    • need 3D and detailed chemistry to see this
knowledge discovery from terascale datasets
Knowledge Discovery From Terascale Datasets
  • Challenge:Need interactive knowledge discovery software for terascale high-fidelity scientific data sets where computing pipeline is distributed geographically.
  • Solution:Intelligent visualization pipeline
    • Simultaneous cognitive visualization
    • Intelligent feature extraction/tracking
    • Scalable transparent data sharing and parallel I/O across platforms

INCITE Award: DNS of a 3D turbulent flame (9 TB raw data - 350 million grids, 17 variables, 100,000 steps)

advanced post processing for extinction re ignition
Advanced post-processing for extinction/re-ignition
  • Extinction viewed as radical or reaction-rate “hole” in a mixture-fraction isosurface
  • Stems from fast chemistry flame-sheet “flamelet” models of turbulent combustion
  • Holes can be expanding or contracting
  • Speed of the edge can be monitored – need algorithm for edge detection
advanced post processing for extinction re ignition1
Advanced post-processing for extinction/re-ignition
  • Holes could be tracked to determine any history effects
    • need parallel feature tracking
outline2
Outline
  • DNS of turbulent combustion – challenges and opportunities
  • Sandia S3D DNS capability – terascale simulations on Office of Science platforms
  • New combustion science to advance predictive models:
    • 2D DNS of HCCI combustion
    • 3D simulations of turbulent jet flames with detailed chemistry
    • Layering Large Eddy Simulation and DNS approaches

Vorticity fields in DNS of a turbulent jet flame

(volume rendering by

Kwan-Liu Ma and Hiroshi Akiba)

layering of les and dns approaches the future

Layering of LES and DNS approaches(the future...)

Joseph C. Oefelein, Evatt R. Hawkes, Jacqueline H. Chen

rans les dns at high re
DNSRANS-LES-DNS at High Re

Swirl-Stabilized Premixed Combustion

Azimuthal Velocity Component

  • LES gets lots more than RANS
  • High Re is too expensive for DNS (at present)
    • can afford 1000s now

useful but…

    • would like 100000s and more

LES

RANS

why les
Why LES?
  • LES treats the large scales directly
  • Large scales are geometry dependent
  • LES can couple directly with high Re experiments
  • But – LES needs sub-grid models
layering les and dns
ApplicationsLayering LES and DNS
  • Better BCs, ICs
  • Identify modelling questions
  • Identify relevant parameter space
  • Moderate Re:
  • Canonical problems
  • Cleverly designed experiments

LES

DNS

High Re

Experiments

  • Sub-grid models
multiphysics capabilities and algorithmic framework oefelein
Multiphysics capabilities and algorithmic framework – Oefelein
  • Theoretical framework (LES  DNS)
    • Fully-coupled conservation equations
    • Compressible, chemically reacting system
    • Real-fluid EOS, thermodynamics, transport
    • Detailed chemistry, multiphase flow, sprays
    • Dynamic subgrid-scale modeling
  • Numerical framework
    • Dual-time, all-Mach-number formulation
    • Generalized preconditioning methodology
    • Complex geometry, generalized coordinates
    • Massively-parallel (MPI), highly-scalable
    • Ported to all major platforms
  • Over 10 years development
  • Fine-grain scalability (NERSC IBM SP – Seaborg)
  • Grid size fixed, number of processors increased
magnitude of vorticity normalized
Magnitude of vorticity (normalized)

DNS

(all scales)

40 million grid, 120,000 hours

LES

(large scales)

0.6 million grid, 600 hours

summary
Summary
  • S3D is a state-of-the-art DNS capability for turbulent combustion simulations that scales to thousands of processors and is ported to all Office of Science platforms.
  • DOE supercomputing facilities are enabling new combustion science.
  • 3D DNS of detailed finite-rate chemistry effects in turbulent jets provides new insights and data for combustion modeling:
    • mixing of reactive scalars can be very different from conserved scalars
  • Layering LES and DNS approaches will open new avenues of exploration with great potential for the future
ad