1 / 17

 Emulation and sensitivity analysis in the Model Web

 Emulation and sensitivity analysis in the Model Web. Richard Jones Computer Science, Aston University, Birmingham, United Kingdom UncertWeb workshop, 10 September 2012, IfGI. Introducing sensitivity analysis.

keran
Download Presentation

 Emulation and sensitivity analysis in the Model Web

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1.  Emulation and sensitivity analysis in the Model Web Richard Jones Computer Science, Aston University, Birmingham, United Kingdom UncertWeb workshop, 10 September 2012, IfGI

  2. Introducing sensitivity analysis • Quantifies contributions of different uncertainty sources to the overall output uncertainty. • This can help to gain a better understanding of your (or someone else's) model. • Decide which inputs to focus on getting accurate data for.

  3. Introducing uncertainty analysis • Input uncertainty must be propagated through a model. • This is critical for using the model output in decision making. • Uncertainty analysis is commonly performed using Monte Carlo methods. • Random sampling from input distributions. • Multiple model runs using these samples.

  4. Implications • Sensitivity and uncertainty analysis require thousands of simulator runs. • Scales with the number of inputs, and outputs. • Unfeasible for slow models. • Changing parameters will require another set of runs. • Computational constraints have in the past limited the application of sensitivity and uncertainty analysis.

  5. Introducing emulators • An emulator is a surrogate statistical model. • Typically based on Gaussian process regression. • A mean and covariance function with parameters. • Trained on a series of simulator runs.

  6. Emulator benefits • Very fast to evaluate. • Can be passed a multi-point design to evaluate. • Rather than requiring multiple runs. • Is aware of its own prediction accuracy.

  7. Emulator implications • Building an emulator is complex. • Consists of several iterative stages, requiring input at each. • The tools for sensitivity analysis and emulation may not be available in your preferred language. • Input and outputs may be in formats difficult to read. • Conversion needed to use with tools.

  8. How the Model Web helps • Standardised interfaces and data. • No longer need to write specific code to execute each model. • Increased availability of services. • On a single machine, but networked and accessible everywhere. • Leverage to create a system to help users build emulators and perform sensitivity analysis.

  9. The emulation tools system • The system is based on a API backend, with Web frontend. • Backend uses MATLAB for sampling and GP, R for sensitivity analysis. • Web interface hides this implementation detail. • Frontend based on Ruby on Rails. • Long running jobs are performed in the background.

  10. API backend • Stateless and JSON based. • Provides access to a number of methods for: • Gathering process and I/O metadata from Web services. • Executing a process on Web service against a sample. • Training and validating an emulator. • Performing sensitivity analysis.

  11. Web frontend • Has two step-by-step tools for building an emulator and performing sensitivity analysis. • Building an emulator supports the complete emulator lifecycle. • Performing sensitivity analysis doesn’t require an emulator, but one can be used. • Both allow many parameter adjustments and provide visualisations to aid decision making.

  12. A note on Web services • The OGC define the WPS standard for exposing geospatial models/processes on the Web. • We have found some problems when using the standard: • Implementation complexity. • Lack of concrete message descriptions. • Full support for SOAP/WSDL and JSON missing.

  13. Processing Service framework • These shortcomings encouraged us to develop our own framework for processing services. • Concrete SOAP/WSDL support. • Full JSON interface. • Java based and extensible. • The emulator system is compatible with models exposed using our framework and WPS.

  14. Emulators as processes • We can use our emulator with the sensitivity analysis tool provided by the system. • Once your emulator is validated, it can be used without any further configuration. • It is also possible to use them as you would any other process. • Use within client software. • Integrate in new or existing workflows.

  15. Emulators as processes • The framework was extended to create a service that supports emulators. • Emulators can be uploaded to this service where they will be available like any other process. • Monte Carlo runs can be executed with a single call. • Reduces additional communication overheads.

  16. Summary • Sensitivity and uncertainty analysis require thousands of model runs. • Emulators can be used as surrogate simulators for large speed improvements. • Our system leverages the Model Web to make emulator and SA methods more accessible.

  17. Workbook • Navigate to: http://uncertws.aston.ac.uk/files/workbook.pdf

More Related