1 / 36

Prof J Craig Mudge FTSE University of Adelaide Australia

Evolving inversion methods in Geophysics with Cloud Computing – a case study of an eScience collaboration Mudge, Chandrasekhar, Heinson , Thiel. Prof J Craig Mudge FTSE University of Adelaide Australia School of Computer Science/ School of Earth Sceinces

swann
Download Presentation

Prof J Craig Mudge FTSE University of Adelaide Australia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evolving inversion methods in Geophysics with Cloud Computing – a case study of an eScience collaboration Mudge, Chandrasekhar, Heinson, Thiel Prof J Craig Mudge FTSE University of Adelaide Australia School of Computer Science/ School of Earth Sceinces 7th IEEE eScience Conference, Stockholm, December 2011

  2. Two South Australian successes in geology • Hot rocks for geo-thermal energy - 95% investment is in South Australia • Olympic Dam  - BHP Billiton -- world's fourth largest copper deposit, fifth largest gold deposit and the largest uranium deposit. craig.mudge@adelaide.edu.au IEEE eScience 2011

  3. Outline Cloud computing Collaborative Cloud Computing Lab (C3L) Inversion in magnetotelluric processing Geothermal – EGS in South Australia Results and Lessons learned Future work

  4. Cloud service provider owns and operates the infrastructure and innovates to keep technology leading edge, handle software upgrades, and steadily reduce energy costs Google, Dalles Oregon Microsoft Azure, Chicago

  5. Massive scale of data centres delivers 4 – 7X cost reduction and energy efficiency Air flow

  6. A no-machines Lab eScience enabled by cloud computing machines Seed funding from -- Department of Mines www.pir.sa.gov.au -- MSFT Research Jim Gray Seed Grant Started June 2010 craig.mudge@adelaide.edu.au IEEE eScience 2011

  7. Our three cloud service providers Amazon Web Services Microsoft Azure Now adding government funded eResearch clouds which will run Open Stack (NASA and Rackspace) craig.mudge@adelaide.edu.au IEEE eScience 2011

  8. Magnetotelluric (MT) imaging (Heinson and Mudge, 2010) CO2 in depleted gas field • Using the magnetic and electric fields of the earth, MT imaging determines the resistivity structure of a sub-surface area of interest. • It goes deeper (hundred or so Km) than seismic (<2 Km) but does not have the same resolution • Applications • mineral exploration, • water management in mining, • geothermal exploration, • carbon storage, • aquifer research and management • earthquake and volcano studies.

  9. Electrical resistivity

  10. Electromagnetic methods

  11. Data logging by University of Adelaide Geophysics, on a geothermal site – Paralana, SA, Australia

  12. MT Processing steps Inversion craig.mudge@adelaide.edu.au IEEE eScience 2011

  13. start Searching the solution space compute sensitivity matrix compute model’s MT response Inversion iterations: Compute model response, compare with observed data locally improve model misfit compare model responseto observed data yes requiredmisfit? no no can locally improve misfit? > max iterations? yes can locally improve smoothness? yes locally improve model smoothness smoothenough? no yes no no yes finish craig.mudge@adelaide.edu.au IEEE eScience 2011

  14. craig.mudge@adelaide.edu.au IEEE eScience 2011

  15. craig.mudge@adelaide.edu.au IEEE eScience 2011

  16. Setting up a new inversion – part 1 craig.mudge@adelaide.edu.au IEEE eScience 2011

  17. Setting up a new inversion – part 2 craig.mudge@adelaide.edu.au IEEE eScience 2011

  18. Dashboard craig.mudge@adelaide.edu.au IEEE eScience 2011

  19. Results and Lessons learned craig.mudge@adelaide.edu.au IEEE eScience 2011

  20. Speedup Sequential Parallel craig.mudge@adelaide.edu.au IEEE eScience 2011

  21. Performance analysis beyond speedup Sequential Parallel Examples of recent performance analysis Effect of FORTRAN compiler with different optimisations has been worth exploring. A factor of 3X speed up from the Intel Visual Fortran Composer XE 2011 for Windows. 2. “Steal time” - time lost due to hypervisor’s management of a virtual machine – Netflix have analysed their Amazon experience extensively craig.mudge@adelaide.edu.au IEEE eScience 2011

  22. Results and learnings “No-machines” works Speedup has led to 100% adoption in MT research First results of monitoring fluid injection in EGS Reservoirs using magnetotellurics (MT) – promising since seismic does not indicate fluid flow, and MT is low cost Taking chunks of FORTRAN is achievable in a timely manner Capability building – a true eScience partnership Our Web Services user interactions took same amount of programming effort as parallelising craig.mudge@adelaide.edu.au IEEE eScience 2011

  23. eScience in the cloud- observations of a veteran of the computer industry (but not my co-authors in this eScience paper) Web Services (giving interoperability between disparate services of historic proportion) could have been adopted faster in eScience craig.mudge@adelaide.edu.au IEEE eScience 2011

  24. (Mudge, 2002) craig.mudge@adelaide.edu.au IEEE eScience 2011

  25. (Mudge, 2002) craig.mudge@adelaide.edu.au IEEE eScience 2011

  26. eScience in the cloud- observations of a veteran of the computer industry (but not my co-authors in this eScience paper) Web Services (giving interoperability between disparate services of historic proportion) could have been adopted faster in eScience Cloud computing will speed up the use of web services , because cloud makes it natural to interact using web services (service orientation, discovery, interoperability) craig.mudge@adelaide.edu.au IEEE eScience 2011

  27. Lessons learned – HPC programming MapReduce (Hadoop) is the programming model that best matches data centre as the computer. However, because it requires rewrite of existing programs, the first wave of benefits come from simpler parallelism – parameter sweeps, Monte Carlo simulation, job-level parallelism, etc. Second wave of benefits will be new algorithms and rewrites using MapReduce Nevertheless, the first wave in cloud-based bioinformatics (matching short reads against reference genome) did use MapReduce craig.mudge@adelaide.edu.au IEEE eScience 2011

  28. Lessons learned - Azure • Why was Azure much harder to migrate to than predicted? Answer: - We came from a non .Net environment - Azure younger than Amazon (2 years) • Virtual Machine in Beta • Deployment times 20 minutes vs 20 seconds slows debugging • Azure designed for long running applications, e.g., ecommerce, more than for scientific 2. However, we persist. - Warehouse-sized data centre – operating system is robust and rich, e.g., hot swap of patches - Benefits of PaaS craig.mudge@adelaide.edu.au IEEE eScience 2011

  29. Future work craig.mudge@adelaide.edu.au IEEE eScience 2011

  30. Future work 1 of 2 • Inversion on demand, available to colleagues and explorers world-wide, wrapped in workflow (persistence, provenance, partial runs, ...) • National/international collaboration building on a national Geophysics Virtual Lab - access to disparate data (seismic, borehole images, gravity, magnetic, ...) built by Auscope using results of GeoSciML Interoperability Working Group craig.mudge@adelaide.edu.au IEEE eScience 2011

  31. Societal Need Sustainable Energy Policy Environment Virtual Laboratory Energy Exploration Integrated Virtual Laboratory Integrated Virtual Labs Virtual Laboratories Virtual Geophysical Laboratory National Borehole Laboratory Virtual Geodesy Laboratory Virtual Earth Observation Laboratory Virtual Oceans Laboratory Modelling & analytic tools Processing Services Processing Services Processing Services Processing Services Processing Services Middleware Middleware Middleware Middleware Middleware Virtual Libraries Data Data Data Data Data Land cover Marine Geophysics Borehole Geodesy craig.mudge@adelaide.edu.au IEEE eScience 2011 Dr Robert Woodcock and Dr Lesley Wyborn

  32. Future work 2 of 2 Explore statistical machine learning to detect interesting patterns Exploring solution space using Evolutionary Algorithms implemented on thousands of processors in the cloud (Brad Alexander) Promulgate security best practices Following the success of speedup, model size has become the limiter for our geophysicists craig.mudge@adelaide.edu.au IEEE eScience 2011

  33. Acknowledgements Brad Alexander Gordon Bell Pinaki Chandrasekhar Dennis Gannon Graham Heinson Tony Hey Ed Lazowska Stephan Thiel craig.mudge@adelaide.edu.au IEEE eScience 2011

  34. Summary Cloud computing Collaborative Cloud Computing Lab (C3L) Inversion in magnetotelluric processing Geothermal – EGS in South Australia Lessons learned Future work

  35. Thanks and questions craig.mudge@adelaide.edu.auwww.cloudinnovation.com.au+61 417 679 266+1 650 224 2111 craig.mudge@adelaide.edu.au IEEE eScience 2011

  36. Security best practices Certifications Physical security Secure services Data privacy via encryption Backups Constant monitoring External review Compare yours with Google, Amazon, Azure craig.mudge@adelaide.edu.au IEEE eScience 2011

More Related