Emulation and sensitivity analysis in the model web
This presentation is the property of its rightful owner.
Sponsored Links
1 / 17

 Emulation and sensitivity analysis in the Model Web PowerPoint PPT Presentation


  • 53 Views
  • Uploaded on
  • Presentation posted in: General

 Emulation and sensitivity analysis in the Model Web. Richard Jones Computer Science, Aston University, Birmingham, United Kingdom UncertWeb workshop, 10 September 2012, IfGI. Introducing sensitivity analysis.

Download Presentation

 Emulation and sensitivity analysis in the Model Web

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Emulation and sensitivity analysis in the model web

 Emulation and sensitivity analysis in the Model Web

Richard Jones

Computer Science, Aston University, Birmingham, United Kingdom

UncertWeb workshop, 10 September 2012, IfGI


Introducing sensitivity analysis

Introducing sensitivity analysis

  • Quantifies contributions of different uncertainty sources to the overall output uncertainty.

  • This can help to gain a better understanding of your (or someone else's) model.

    • Decide which inputs to focus on getting accurate data for.


Introducing uncertainty analysis

Introducing uncertainty analysis

  • Input uncertainty must be propagated through a model.

    • This is critical for using the model output in decision making.

  • Uncertainty analysis is commonly performed using Monte Carlo methods.

    • Random sampling from input distributions.

    • Multiple model runs using these samples.


Implications

Implications

  • Sensitivity and uncertainty analysis require thousands of simulator runs.

    • Scales with the number of inputs, and outputs.

    • Unfeasible for slow models.

      • Changing parameters will require another set of runs.

  • Computational constraints have in the past limited the application of sensitivity and uncertainty analysis.


Introducing emulators

Introducing emulators

  • An emulator is a surrogate statistical model.

    • Typically based on Gaussian process regression.

      • A mean and covariance function with parameters.

    • Trained on a series of simulator runs.


Emulator benefits

Emulator benefits

  • Very fast to evaluate.

  • Can be passed a multi-point design to evaluate.

    • Rather than requiring multiple runs.

  • Is aware of its own prediction accuracy.


Emulator implications

Emulator implications

  • Building an emulator is complex.

    • Consists of several iterative stages, requiring input at each.

  • The tools for sensitivity analysis and emulation may not be available in your preferred language.

  • Input and outputs may be in formats difficult to read.

    • Conversion needed to use with tools.


How the model web helps

How the Model Web helps

  • Standardised interfaces and data.

    • No longer need to write specific code to execute each model.

  • Increased availability of services.

    • On a single machine, but networked and accessible everywhere.

  • Leverage to create a system to help users build emulators and perform sensitivity analysis.


The emulation tools system

The emulation tools system

  • The system is based on a API backend, with Web frontend.

  • Backend uses MATLAB for sampling and GP, R for sensitivity analysis.

    • Web interface hides this implementation detail.

  • Frontend based on Ruby on Rails.

    • Long running jobs are performed in the background.


Api backend

API backend

  • Stateless and JSON based.

  • Provides access to a number of methods for:

    • Gathering process and I/O metadata from Web services.

    • Executing a process on Web service against a sample.

    • Training and validating an emulator.

    • Performing sensitivity analysis.


Web frontend

Web frontend

  • Has two step-by-step tools for building an emulator and performing sensitivity analysis.

    • Building an emulator supports the complete emulator lifecycle.

    • Performing sensitivity analysis doesn’t require an emulator, but one can be used.

  • Both allow many parameter adjustments and provide visualisations to aid decision making.


A note on web services

A note on Web services

  • The OGC define the WPS standard for exposing geospatial models/processes on the Web.

  • We have found some problems when using the standard:

    • Implementation complexity.

    • Lack of concrete message descriptions.

    • Full support for SOAP/WSDL and JSON missing.


Processing service framework

Processing Service framework

  • These shortcomings encouraged us to develop our own framework for processing services.

    • Concrete SOAP/WSDL support.

    • Full JSON interface.

    • Java based and extensible.

  • The emulator system is compatible with models exposed using our framework and WPS.


Emulators as processes

Emulators as processes

  • We can use our emulator with the sensitivity analysis tool provided by the system.

    • Once your emulator is validated, it can be used without any further configuration.

  • It is also possible to use them as you would any other process.

    • Use within client software.

    • Integrate in new or existing workflows.


Emulators as processes1

Emulators as processes

  • The framework was extended to create a service that supports emulators.

  • Emulators can be uploaded to this service where they will be available like any other process.

  • Monte Carlo runs can be executed with a single call.

    • Reduces additional communication overheads.


Summary

Summary

  • Sensitivity and uncertainty analysis require thousands of model runs.

  • Emulators can be used as surrogate simulators for large speed improvements.

  • Our system leverages the Model Web to make emulator and SA methods more accessible.


Workbook

Workbook

  • Navigate to:

http://uncertws.aston.ac.uk/files/workbook.pdf


  • Login