Evolving inversion methods in Geophysics with Cloud Computing – a case study of an eScience collab...
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Prof J Craig Mudge FTSE University of Adelaide Australia PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on
  • Presentation posted in: General

Evolving inversion methods in Geophysics with Cloud Computing – a case study of an eScience collaboration Mudge, Chandrasekhar, Heinson , Thiel. Prof J Craig Mudge FTSE University of Adelaide Australia School of Computer Science/ School of Earth Sceinces

Download Presentation

Prof J Craig Mudge FTSE University of Adelaide Australia

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Prof j craig mudge ftse university of adelaide australia

Evolving inversion methods in Geophysics with Cloud Computing – a case study of an eScience collaboration Mudge, Chandrasekhar, Heinson, Thiel

Prof J Craig Mudge FTSE

University of Adelaide

Australia

School of Computer Science/ School of Earth Sceinces

7th IEEE eScience Conference, Stockholm, December 2011


Two south australian successes in geology

Two South Australian successes in geology

  • Hot rocks for geo-thermal energy - 95% investment is in South Australia

  • Olympic Dam  - BHP Billiton

    -- world's fourth largest copper deposit, fifth largest gold deposit and the largest uranium deposit.

[email protected] IEEE eScience 2011


Outline

Outline

Cloud computing

Collaborative Cloud Computing Lab (C3L)

Inversion in magnetotelluric processing

Geothermal – EGS in South Australia

Results and Lessons learned

Future work


Prof j craig mudge ftse university of adelaide australia

Cloud service provider owns and operates the infrastructure and innovates to keep technology leading edge, handle software upgrades, and steadily reduce energy costs

Google, Dalles Oregon Microsoft Azure, Chicago


Massive scale of data centres delivers 4 7x cost reduction and energy efficiency

Massive scale of data centres delivers 4 – 7X cost reduction and energy efficiency

Air flow


A no machines lab

A no-machines Lab

eScience enabled by

cloud computing

machines

Seed funding from

-- Department of Mines www.pir.sa.gov.au

-- MSFT Research Jim Gray Seed Grant

Started June 2010

[email protected] IEEE eScience 2011


Our three cloud service providers

Our three cloud service providers

Amazon Web Services

Microsoft Azure

Now adding government funded eResearch clouds which will run Open Stack (NASA and Rackspace)

[email protected] IEEE eScience 2011


Magnetotelluric mt imaging

Magnetotelluric (MT) imaging

(Heinson and Mudge, 2010)

CO2 in depleted gas field

  • Using the magnetic and electric fields of the earth, MT imaging determines the resistivity structure of a sub-surface area of interest.

  • It goes deeper (hundred or so Km) than seismic (<2 Km) but does not have the same resolution

  • Applications

    • mineral exploration,

    • water management in mining,

    • geothermal exploration,

    • carbon storage,

    • aquifer research and management

    • earthquake and volcano studies.


Electrical resistivity

Electrical resistivity


Electromagnetic methods

Electromagnetic methods


Data logging by university of adelaide geophysics on a geothermal site paralana sa australia

Data logging by University of Adelaide Geophysics, on a geothermal site – Paralana, SA, Australia


Mt processing steps

MT Processing steps

Inversion

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

start

Searching the solution space

compute sensitivity

matrix

compute model’s

MT response

Inversion iterations:

Compute model response,

compare with observed data

locally improve

model misfit

compare model responseto observed data

yes

requiredmisfit?

no

no

can locally improve misfit?

> max iterations?

yes

can locally improve smoothness?

yes

locally improve

model smoothness

smoothenough?

no

yes

no

no

yes

finish

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

[email protected] IEEE eScience 2011


Setting up a new inversion part 1

Setting up a new inversion – part 1

[email protected] IEEE eScience 2011


Setting up a new inversion part 2

Setting up a new inversion – part 2

[email protected] IEEE eScience 2011


Dashboard

Dashboard

[email protected] IEEE eScience 2011


Results and lessons learned

Results and Lessons learned

[email protected] IEEE eScience 2011


Speedup

Speedup

Sequential

Parallel

[email protected] IEEE eScience 2011


Performance analysis beyond speedup

Performance analysis beyond speedup

Sequential

Parallel

Examples of recent performance analysis

Effect of FORTRAN compiler with different optimisations has been worth exploring. A factor of

3X speed up from the Intel Visual Fortran Composer XE 2011 for Windows.

2. “Steal time” - time lost due to hypervisor’s management of a virtual machine – Netflix have

analysed their Amazon experience extensively

[email protected] IEEE eScience 2011


Results and learnings

Results and learnings

“No-machines” works

Speedup has led to 100% adoption in MT research

First results of monitoring fluid injection in EGS Reservoirs using magnetotellurics (MT) – promising since seismic does not indicate fluid flow, and MT is low cost

Taking chunks of FORTRAN is achievable in a timely manner

Capability building – a true eScience partnership

Our Web Services user interactions took same amount of programming effort as parallelising

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

eScience in the cloud- observations of a veteran of the computer industry (but not my co-authors in this eScience paper)

Web Services (giving interoperability between disparate services of historic proportion) could have been adopted faster in eScience

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

(Mudge, 2002)

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

(Mudge, 2002)

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

eScience in the cloud- observations of a veteran of the computer industry (but not my co-authors in this eScience paper)

Web Services (giving interoperability between disparate services of historic proportion) could have been adopted faster in eScience

Cloud computing will speed up the use of web services , because cloud makes it natural to interact using web services (service orientation, discovery, interoperability)

[email protected] IEEE eScience 2011


Lessons learned hpc programming

Lessons learned – HPC programming

MapReduce (Hadoop) is the programming model that best matches data centre as the computer. However, because it requires rewrite of existing programs, the first wave of benefits come from simpler parallelism – parameter sweeps, Monte Carlo simulation, job-level parallelism, etc.

Second wave of benefits will be new algorithms and rewrites using MapReduce

Nevertheless, the first wave in cloud-based bioinformatics (matching short reads against reference genome) did use MapReduce

[email protected] IEEE eScience 2011


Lessons learned azure

Lessons learned - Azure

  • Why was Azure much harder to migrate to than predicted?

    Answer:

    - We came from a non .Net environment

    - Azure younger than Amazon (2 years)

    • Virtual Machine in Beta

    • Deployment times 20 minutes vs 20 seconds slows debugging

  • Azure designed for long running applications, e.g., ecommerce, more than for scientific

    2. However, we persist.

    - Warehouse-sized data centre – operating system is robust and rich, e.g., hot swap of patches

    - Benefits of PaaS

[email protected] IEEE eScience 2011


Future work

Future work

[email protected] IEEE eScience 2011


Future work 1 of 2

Future work 1 of 2

  • Inversion on demand, available to colleagues and explorers world-wide, wrapped in workflow (persistence, provenance, partial runs, ...)

  • National/international collaboration building on a national Geophysics Virtual Lab

    - access to disparate data (seismic, borehole images, gravity, magnetic, ...) built by Auscope using results of GeoSciML Interoperability Working Group

[email protected] IEEE eScience 2011


Prof j craig mudge ftse university of adelaide australia

Societal Need

Sustainable Energy Policy

Environment

Virtual Laboratory

Energy Exploration Integrated Virtual Laboratory

Integrated Virtual Labs

Virtual Laboratories

Virtual Geophysical Laboratory

National Borehole Laboratory

Virtual Geodesy Laboratory

Virtual Earth Observation

Laboratory

Virtual Oceans Laboratory

Modelling & analytic tools

Processing Services

Processing Services

Processing Services

Processing Services

Processing Services

Middleware

Middleware

Middleware

Middleware

Middleware

Virtual Libraries

Data

Data

Data

Data

Data

Land cover

Marine

Geophysics

Borehole

Geodesy

[email protected] IEEE eScience 2011

Dr Robert Woodcock and Dr Lesley Wyborn


Future work 2 of 2

Future work 2 of 2

Explore statistical machine learning to detect interesting patterns

Exploring solution space using Evolutionary Algorithms implemented on thousands of processors in the cloud (Brad Alexander)

Promulgate security best practices

Following the success of speedup, model size has become the limiter for our geophysicists

[email protected] IEEE eScience 2011


Acknowledgements

Acknowledgements

Brad Alexander

Gordon Bell

Pinaki Chandrasekhar

Dennis Gannon

Graham Heinson

Tony Hey

Ed Lazowska

Stephan Thiel

[email protected] IEEE eScience 2011


Summary

Summary

Cloud computing

Collaborative Cloud Computing Lab (C3L)

Inversion in magnetotelluric processing

Geothermal – EGS in South Australia

Lessons learned

Future work


Prof j craig mudge ftse university of adelaide australia

Thanks and questions [email protected]+61 417 679 266+1 650 224 2111

[email protected] IEEE eScience 2011


Security best practices

Security best practices

Certifications

Physical security

Secure services

Data privacy via encryption

Backups

Constant monitoring

External review

Compare yours with Google, Amazon, Azure

[email protected] IEEE eScience 2011


  • Login