A Validation Study for a Hypersonic Flow Model

1. A Validation Study for a Hypersonic Flow Model Brian Carnes, Derek Dinzl, Micah Howard, Sarah Kieweg, Jaideep Ray, William Rider, Tom Smith, Greg Weirs ASME VVS2018-9414

2. Outline • Motivation • Scope of V&V and UQ study • Experimental data and implications for validation • Inferred freestream conditions • Concluding Remarks

3. Motivation: SPARC simulation code SPARC is a new code under development at Sandia National Laboratories for the simulation of atmospheric reentry vehicles Current effort – mature methods and models: • TVD-class methods and implicit time advancement, for steady state flowfields • Established models for thermal nonequilibrium and chemical kinetics • RANS turbulence models (Spalart-Allmaras, SST) Development roadmap: • High-order discretizations, including shock-capturing • LES and DNS • More extensive physics Significant development effort on high performance on next generation architectures

4. Hypersonic Reentry Simulation Unsteady, turbulent flow Surface ablation & in-depth decomposition Flowfield radiation Gas-phase thermochemicalnon-equilibrium Atmospheric variations Maneuvering RVs: Shock/shock & shock/boundary layer interaction Laminar/transitional/turbulent boundary layer Random vibrational loading Gas-surface chemistry

5. Scope: End Application and QOIs • Atmospheric reentry of flight vehicles • Compressible flow from subsonic to hypersonic • Turbulent • Thermochemical nonequilibrium • Vehicle performance predictions require estimates of: • Aerodynamic forces • Heat transfer • Quantities of Interest (QOIs) from experiments and simulations: • Heat flux at surface • Pressure on surface • Separation length, separation point, and reattachment point Detached shock • Double cone quantities of interest Impingement point M>>1 Separation point Attached shock Impingement Point Separation Point

6. Project Scope: PCMM-driven V&V and UQ activities CVER Numerical Uncertainty (SVER) Experimental conditions and uncertainty Experimental measurements and uncertainty Uncertainty Quantification Validation Metrics SensitivityAnalysis • Large, diverse V&V and UQ team working on an array of SPARC assessments • We have used PIRTs and PCMM to aid our communication and collaboration, within the team and to our stakeholders • Some of our verification work is presented in a companion talk (VVS2018-9406) Validation Assessment

7. CUBRC Double Cone Validation Experiments • LENS I shock tunnel (~2000-2007) • Single element (N2 or O2) • Mild thermochemical nonequilibrium • LENS XX expansion tunnel (2013) • Air mixture • Strong thermochemical nonequilibrium • For both: • Same geometry: 25-55 double cone • Laminar flow • Surface diagnostics only Schematics: Holden, MacLean, Wadhams, and Dufrene, AIAA paper 2013-2837

8. Double Cone Experiments Double cone Pros: • Experiments are mature, with successive improvements addressing difficult aspects of the flowfield. • LENS-I and LENS-XX are complementary • LENS-XX covers a large range of free stream enthalpies, and several densities and temperatures. • Consistent measurements for surface heat flux, pressure: strong signal for separation and other flow field features as a function of free stream conditions Double cone Challenges: • Desire to use measurements to assess thermochemical nonequilibrium flow models, but no experimental diagnostics directly measure any aspects of flowfield composition or vibrational excitation • No replicate experiments • Uncertainties not well characterized, esp. free stream mean conditions, free stream spatial variability, surface heat flux and pressure measurement errors and uncertainties • LENS-I dataset required several years to “unfold”; LENS-XX data might also

9. Uncertainty Quantification: Experimental Data Challenges • Free stream conditions provided with “% error”: • Interpreted as bounds of uniform distribution • Heat flux and pressure measurements have “% error” • Interpreted as bounds of a uniform distribution • LENS I: Nompelis et al. – nozzle simulations to obtain free stream conditions. • LENS XX: unresolved questions about free stream conditions LENS I, Run 35: Free stream and wall condition (I. Nompelis) LENS XX: Case 1 and Case 4

10. LENS XX (Case 4): Initial Comparison Without UQ • Compare SPARC to US3D and Experiment • SPARC simulations on1024x512 mesh • Experimental error bars: heat flux +/- 7 %, pressure +/- 5% per CUBRC.

11. LENS XX (Case 4): Uncertainty Quantification and Validation Forward UQ on SPARC simulation  parametric uncertainty  need to calibrate • SPARC simulations on coarse 256x128 mesh • UQ ensemble from: 7, 3, and 3% r, U, T per CUBRC; Tv = T

12. Calibration and Validation • Calibration Goals • Develop an inference technique to estimate free stream (r, U, T, Tv) and uncertainties from measurements of heat flux, pressure, total enthalpy and pitot pressure • Calibrate using attached region • Predict over entire flow regime • Demonstrate with LENS-I Run 35 • Perform on LENS-XX Case 4, and possibly Case 1

13. LENS I (Run 35): Uncertainty Quantification I. Nompelis et al.: Obtained free stream conditions from nozzle simulations Separated region Attached region Second Cone • SPARC simulations on 1024x512 mesh • Experimental error bars: heat flux +/- 5 %, pressure +/- 3% per CUBRC.

14. Inferring free stream conditions: LENS-I run 35 • Heat transfer and pressure on the attached flow region of the first cone are well resolved even on coarse meshes – numerical error estimates are very small • LENS-I run 35 is well studied • Nompellis determined free stream conditions by simulating the diverging part of the shock tunnel • This run can be predicted well using a perfect gas model (so estimate r, U, and T) • Estimate free stream conditions from 3 values of P, 11 values of q, h0, Ppitot(freestream) • If the approach works for LENS-I run 35, we will try to infer the free stream conditions for LENS-XX cases

15. Inferring free stream conditions: methodology • Bayesian inference asks to to model the data – model mismatch. • Our model: log(Yobs) = log(Msurrogate(r, U, T)) + e, e ~ N(0, s2) • Yobs = {p(1:3), q(1:11), h0, Ppitot} • Estimate: P(r, U, T, s | Yobs) • Prior belief: Uniform distribution, +/- 10% around Nompelis’s values • QoI (surface) measurement uncertainties are not included in the inference process at this time • Steps: • Run SPARC 30 times, sampling priors for freestream (r, U, T); PCE on level 2 sparse grid. (Later repeated on level 3 sparse grid, same results) • Sample PCE (surrogate) across (r, U, T) space using Markov Chain Monte Carlo, comparing (simulation) observations to (experimental) measurements and updating priors to produce posteriors

16. Calibration: LENS I (Run35) test case Compare: • CUBRC’s original freestream conditions (not always in range of plots) • Nompelis’ non-eq freestream conditions • Our calibrated freestream conditions (MAP) Our calibrated values have better agreement with Nompelis – and we reproduce observations PDFs of Calibrated Values Priors correspond to x-axis plot limits

17. Comparing prior and posterior predictions – surface pressure • Tightens up pressure prediction – assimilating experimental data helps

18. Comparing prior and posterior predictions – heat flux • Tightens up heat flux prediction – assimilating experimental data helps

19. Comparing prior and posterior predictions – h0 & Ppitot • Under-predicting h0 • Over-predicting Ppitot slightly

20. Forward UQ: Calibrated LENS I (Run35) free stream conditions reproduce observations reasonably well • Calibration used only attached region’s experimental probe points. Prediction over the entire flow field using inferred conditions and uncertainties. • 5th/95th percentiles contain measurements (attached region; 11 probes) • SPARC simulations on 1024x512 mesh

21. Remarks on free stream calibration • Inclusion of measurement error • Relative weighting of measurements – in some ways the heat flux and pressure measurements are “overweighted” because there are multiple measurements • Calibration using only forecone data is not ideal… • Forecone pressure and heat flux are not very sensitive to free stream density or temperature • In LENS-XX cases, the foreconeflowfield is weakly sensitive to thermal nonequilbrium models – Tv will probably be insensitive as well • …but calibration using all the surface data removes the basis for validation statements

22. Concluding Remarks Validation assessment should be scoped, and expectations set, by the quality and quantity of the experimental data • What conclusions can be supported by the data and to what degree? LENS-XX example: • Experimental data is limited and somewhat suspect • If the only issue is that the reported free stream conditions are inaccurate, our calibration process might be able to correct that, and some conclusions about thermochemical nonequilibium models based on the downstream surface measurements might be possible At this time, SPARC credibility is based on whether SPARC simulations match other simulations that use the same models – much weaker than we would like, but what our evidence supports.

23. Backup Slides

24. Credibility context • Verification and Validation assessments have been compared to court cases: • A code or model cannot be proven to be error free • Verification and validation can provide evidence that errors have not been detected, and credibility is based on the preponderance of evidence • The jury of peers is comprised of subject matter experts • Unlike a trial, verification and validation results should not be summarized as an innocent-guilty (pass-fail) decision • V&V “evidence” provides a much richer credibility environment; almost all the useful information is outside the answers to the pass-fail questions “Is the code verified?” or “Is the code validated?” • V&V consumers are usually better served by identifying the boundary of credibility, or the relative confidence or risk associated with finer-grained questions • This talk will identify these issues in a case study

25. Scope: Establishing V&V and UQ Context • PIRT: Phenomenology Identification and Ranking Table • What are the relevant physics, how good are the respective models, and how important is each for this application? • Focuses information sharing between physics domain experts, code developers, and V&V and UQ experts • PCMM: Predictive Capability Maturity Model • For each contributor to simulation fidelity, how well are we assessing errors and uncertainties? • PIRTs and PCMM are communication tools • Coordinate effort of large team with a diverse set of perspectives • Convey a consistent, high-level view for project stakeholders (especially managers) • “Before” and “After” PIRTs and PCMM measure progress

26. Scope: Validation Hierarchy Atmospheric reentry: turbulent, thermochemical NEQ HIFiRE-I: Turbulent, single species Double Cone (LENS XX): 5-species, vibrational NEQ Double Cone (LENS I): 1-, 2-, or 5-species, vibrational NEQ NS + RANS NS NS + evib MSNS + evib MSNS + w PDEs Calorically perfect thermally perfect mixture of thermally perfects EOSs Sutherland, Pr=constant mass diffusion and mixing rules transport

27. Experimental Data: Implications for Validation • Experimentalists report free stream and surface measurement uncertainties as “+/- X%” and incorporate a lot of conservatism. • Interpret measurements as bounds based on expert judgement Lack of confidence in uncertainty information, and history of simulation community’s inability to predict experimental results • Community simulation results do not match LENS-XX experimental results; discrepancies have not bee resolved • Credibility evidence is focused on whether SPARC simulations match other other simulations that use the same models, and on inferring experimental free stream conditions