GoldSim User Conference San Francisco, CA October 25-26, 2007

1. GoldSim Tutorial and Discussion: Techniques for Testing, Ensuring Quality and Validating GoldSim Models GoldSim User Conference San Francisco, CA October 25-26, 2007

2. Overview • Best practices for building models that are not error-prone • Testing and verifying models • Verification tests whether the model does what you think it does • e.g., it accurately solves the equations • Validating models • Validation tests whether the model is an appropriate representation of reality • It solves the right equation • Most of this is common sense

3. Best Practices for Building Models that are Not Error-Prone • Document your assumptions in the model as you build your model • Notes, Text boxes • Use descriptive names

5. Best Practices for Building Models that are Not Error-Prone • Document your assumptions in the model as you build your model • Notes, Text boxes • Use descriptive names • Break complex calculations into separate elements • Do no bury details in input fields

8. Best Practices for Building Models that are Not Error-Prone • Document your assumptions in the model as you build your model • Notes, Text boxes • Use descriptive names • Break complex calculations into separate elements • Do no bury details in input fields • Use arrays (vectors or matrices) or cloned Containers for identical parallel calculations

9. Using Cloned Containers Localized Localized A B A B Clone2 Clone1 X = f(A,B) X = f(A,B)

10. Best Practices for Building Models that are Not Error-Prone • Don’t put unit conversions into your model

12. Best Practices for Building Models that are Not Error-Prone • Don’t put unit conversions into your model • Don’t use an equality test

14. Best Practices for Building Models that are Not Error-Prone • Don’t put unit conversions into your model • Don’t use an equality test • Any other suggestions or comments?

15. Testing and Verifying Models • To the extent possible, break your model into modules (e.g., functions or subroutines) that can be testing separately • Test over a range of variables • Compare to independent calculations (spreadsheet, analytical solution) • Use Interrupt elements to check for invalid behavior and bad logic

17. Testing and Verifying Models • To the extent possible, break your model into modules (e.g., functions or subroutines) that can be testing separately • Test over a range of variables • Compare to independent calculations (spreadsheet, analytical solution) • Use Interrupt elements to check for invalid behavior and bad logic • Other ideas or suggestions?

18. Validating Models • If possible, compare predicted results to historical observations • Can use GoldSim’s Optimization features to calibrate predicted results to observed results (typically entered in a Time Series)

20. Final Words • The time spent ensuring quality will be repaid many times over • You must be thinking about this from the beginning of the project – don’t wait until the end