1 / 17

# Emulation and sensitivity analysis in the Model Web - PowerPoint PPT Presentation

Emulation and sensitivity analysis in the Model Web. Richard Jones Computer Science, Aston University, Birmingham, United Kingdom UncertWeb workshop, 10 September 2012, IfGI. Introducing sensitivity analysis.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about '  Emulation and sensitivity analysis in the Model Web' - keran

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Emulation and sensitivity analysis in the Model Web

Richard Jones

Computer Science, Aston University, Birmingham, United Kingdom

UncertWeb workshop, 10 September 2012, IfGI

• Quantifies contributions of different uncertainty sources to the overall output uncertainty.

• This can help to gain a better understanding of your (or someone else's) model.

• Decide which inputs to focus on getting accurate data for.

• Input uncertainty must be propagated through a model.

• This is critical for using the model output in decision making.

• Uncertainty analysis is commonly performed using Monte Carlo methods.

• Random sampling from input distributions.

• Multiple model runs using these samples.

• Sensitivity and uncertainty analysis require thousands of simulator runs.

• Scales with the number of inputs, and outputs.

• Unfeasible for slow models.

• Changing parameters will require another set of runs.

• Computational constraints have in the past limited the application of sensitivity and uncertainty analysis.

• An emulator is a surrogate statistical model.

• Typically based on Gaussian process regression.

• A mean and covariance function with parameters.

• Trained on a series of simulator runs.

• Very fast to evaluate.

• Can be passed a multi-point design to evaluate.

• Rather than requiring multiple runs.

• Is aware of its own prediction accuracy.

• Building an emulator is complex.

• Consists of several iterative stages, requiring input at each.

• The tools for sensitivity analysis and emulation may not be available in your preferred language.

• Input and outputs may be in formats difficult to read.

• Conversion needed to use with tools.

• Standardised interfaces and data.

• No longer need to write specific code to execute each model.

• Increased availability of services.

• On a single machine, but networked and accessible everywhere.

• Leverage to create a system to help users build emulators and perform sensitivity analysis.

• The system is based on a API backend, with Web frontend.

• Backend uses MATLAB for sampling and GP, R for sensitivity analysis.

• Web interface hides this implementation detail.

• Frontend based on Ruby on Rails.

• Long running jobs are performed in the background.

• Stateless and JSON based.

• Gathering process and I/O metadata from Web services.

• Executing a process on Web service against a sample.

• Training and validating an emulator.

• Performing sensitivity analysis.

• Has two step-by-step tools for building an emulator and performing sensitivity analysis.

• Building an emulator supports the complete emulator lifecycle.

• Performing sensitivity analysis doesn’t require an emulator, but one can be used.

• Both allow many parameter adjustments and provide visualisations to aid decision making.

• The OGC define the WPS standard for exposing geospatial models/processes on the Web.

• We have found some problems when using the standard:

• Implementation complexity.

• Lack of concrete message descriptions.

• Full support for SOAP/WSDL and JSON missing.

• These shortcomings encouraged us to develop our own framework for processing services.

• Concrete SOAP/WSDL support.

• Full JSON interface.

• Java based and extensible.

• The emulator system is compatible with models exposed using our framework and WPS.

• We can use our emulator with the sensitivity analysis tool provided by the system.

• Once your emulator is validated, it can be used without any further configuration.

• It is also possible to use them as you would any other process.

• Use within client software.

• Integrate in new or existing workflows.

• The framework was extended to create a service that supports emulators.

• Emulators can be uploaded to this service where they will be available like any other process.

• Monte Carlo runs can be executed with a single call.