computing needs assessment methodology and practice l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Computing Needs Assessment: Methodology and Practice PowerPoint Presentation
Download Presentation
Computing Needs Assessment: Methodology and Practice

Loading in 2 Seconds...

play fullscreen
1 / 26

Computing Needs Assessment: Methodology and Practice - PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on

Computing Needs Assessment: Methodology and Practice. Karen Petraska, NASA Office of the CIO James McCabe, Computing Consultant August 2011. Outline. Methodology Introduction Objectives Proving and Scaling the Process Preliminary Work Practice

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Computing Needs Assessment: Methodology and Practice' - brigham


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
computing needs assessment methodology and practice

Computing Needs Assessment:Methodology and Practice

Karen Petraska, NASA Office of the CIO

James McCabe, Computing Consultant

August 2011

outline
Outline

Methodology

  • Introduction
  • Objectives
  • Proving and Scaling the Process
  • Preliminary Work

Practice

  • Interviewed Scientists from the NASA ESM Community
  • Whiteboard Sessions
  • Use-Case Scenarios
  • Describing Workflows

Practice (Cont)

  • Process is Providing Valuable Insight
  • Found Varying Degrees of Complexity in ES Models
  • Generalizing Earth Science Modeling and Analysis
  • There are a Number of Interesting Topics that are Emerging
  • Future Work
  • Appendices
introduction
Introduction

Computing needs assessment is similar to requirements analysis process, and consists of:

  • Targeting NASA Mission Communities
  • Conducting in-depth interviews and whiteboard sessions with scientists and engineers
  • Learning how missions get done, on which IT assets, what is needed, when and where throughout scientists’ entire project lifecycles
  • Using these data to characterize mission IT workflows, developing use case scenarios and models to express general characteristics
  • Recommending refinements to NASA’s IT system to reflect evolving needs of missions for IT
objectives
Objectives
  • Ensure IT assets are available when needed
  • Optimize investments in IT assets for mission purposes
  • Balance overall performance in mission computing and analysis
  • Identify and address evolving needs of missions for IT
  • Develop processes and templates to assess computing needs throughout NASA
  • Go beyond “more, bigger, faster” to understand system-wide needs
  • Determine what is common across mission groups, as well as what is unique for each group

Our primary working assumption is that the most valuable resource is scientists’ and engineers’ time, whatever we can do to improve their effectiveness will pay off in improved mission results

proving and scaling the process
Proving and Scaling the Process
  • NASA Mission Communities are very large, highly unlikely a few individuals could interview them all
  • Therefore, we are working on interview questions and templates that could be used by others, to parallelize this process
  • In order to prove this process, we decided to start small with a focus group, the SMD Earth Science Modeling (ESM) Community
  • We began in June with preliminary work:
    • Development of draft interview questions
    • Defining important terms for science IT
    • Laying foundation to describe science workflows
  • Interviewing scientists throughout July and August

At this point we are prepared to discuss our methodology and some early findings, and deliver a full set of results at the December American Geophysical Union meeting

preliminary work draft interview questions
Preliminary WorkDraft Interview Questions

Categories of Topics Discussed with ESM Scientists:

  • Work Patterns and Behaviors
    • How, when, and where do scientists receive, generate, transport, analyze, store, and visualize data, how can science be done better
  • Computing Job Characteristics
    • What their computing and analysis jobs need in order to run, how could jobs be run more effectively, how can systems be better utilized by them
  • Environments and Support
  • Enhancements, Evolution, and Revolution
    • What can be done from an IT perspective to improve science
  • Computing, Analysis, Visualization Characterizations
    • What do the various systems that they use look like

Lists of questions associated with these topics are provided in Appendix A.

preliminary work defining important terms
Preliminary WorkDefining Important Terms

High-End Computing The application of specialized capabilities and large amounts of processing power (tens of thousands of processors) per job to solve the largest and most complex problems of that period (e.g. Grand Challenge class problems)

Mid-Range Computing Large computing clusters (on the order of thousands of nodes) that may have some of the specialized capabilities of high-end computing, but are generally smaller and less specialized. As such, they operate earlier in the project lifecycle, on subsets of Grand Challenge class problems

Low-End Computing Departmental, project, and individual computing resources, ranging from a few to tens of processors, with little to no specialization

preliminary work describing science modeling workflows
Preliminary WorkDescribing Science Modeling Workflows
  • Based on coding models from Biegel and Kepner
  • Used as a starting point to begin discussions with scientists
  • For example, model for running codes:

Initial Run

Scale and Optimize

Full Run

interviewed scientists from nasa earth science modeling community
Interviewed Scientists from NASA Earth Science Modeling Community

LaRC

  • Atmospheric Sciences

GSFC

  • Atmospheric Chemistry
  • Goddard Modeling and Assimilation Office (GMAO)
  • Global Modeling Initiative (GMI)
  • NASA Unified Weather Research and Forecasting (NU-WRF)

ARC

  • NASA Earth Exchange (NEX)

JPL

  • Ice Sheet System Model (ISSM)
  • Estimating the Circulation and Climate of the Ocean (ECCO)
  • Carbon Monitoring
  • Climate

A list of scientists interviewed to date is provided in Appendix B

whiteboard sessions
Whiteboard Sessions

Using our preliminary work, had whiteboard sessions with individuals and groups, in which they outlined their science, workflows, and IT assets used

Example: NASA Unified Weather Research and Forecasting (NU-WRF)

LIS: Land Information System

GOCART: Goddard Chemistry Aerosol Radiation and Transport

SDSU: Satellite Data Simulator Unit

RAD: Radiation

use case scenarios
Use-Case Scenarios

For each group interviewed, developed a Use-Case Scenario that describes their:

  • Work and science
  • Workflows
  • IT assets used
  • Issues
  • Desirables and potential impacts
  • Future IT needs
describing workflows
Describing Workflows

When possible, steps in the process of developing science data are captured, along with any IT-related information

Example: Cloud Modeling at LaRC

NCAR: National Center for Atmospheric Research

ASDC: Atmospheric Sciences Data Center

GrADS: Grid Analysis and Display System

C3M: CERES, CALIPSO, CloudSat, MODIS

process is providing valuable insight
Process is Providing Valuable Insight
  • Found varying degrees of complexity in Earth Science Models, and corresponding implications to IT
  • Generalizing Earth Science Modeling and Analysis
    • Characterizes computing, analysis, visualization
    • Identifies key interfaces within and between components, where transport services apply
    • These may be strategic locations from an IT architecture perspective
    • Also need to capture unique characteristics
  • There are a number of interesting topics that are emerging
    • Science data
    • Processing queues
    • Security
    • Support to scientists
    • Environments

Need a much larger sample size

found varying degrees of complexity in earth science models
Found Varying Degrees of Complexity in Earth Science Models

Increasingly complex science models stress the performance levels of

underlying IT infrastructure

LIS: Land Information System

GOCART: Goddard Chemistry Aerosol Radiation and Transport

SDSU: Satellite Data Simulator Unit

RAD: Radiation

generalizing earth science modeling and analysis
Generalizing Earth Science Modeling and Analysis

Earth Science Model Processing and Analysis

there are a number of interesting topics that are emerging
There are a Number of Interesting Topics that are Emerging
  • Science data
    • Growth in science data set sizes is outpacing our transport capabilities
    • Data set sizes approaching PB
    • Possible architectural options include: co-locating resources with data sets, improving transport, or replicating data sets
  • Processing queues
    • Queue wait times increase time to solution by up to 300%
  • Security
    • Security implementations reduce end-to-end performance, there is a need to better balance between the two
  • Support to scientists
    • Technical support across project lifecycle critical to scientists’ success
  • Environments (Processors, Storage, Communications, Libraries, Compilers)
    • As model complexities increase, its becoming harder for scientists to integrate across systems
future work
Future Work

Near-Term – Completing Assessment of Earth Science Modeling

  • Gather and derive more data to improve performance of NASA IT assets to science communities
  • Round out scope of NASA ESM Community
    • Other SMD ESM groups (Goddard Institute for Space Science)
    • ESM at other agencies (Los Alamos and Oak Ridge National Labs), for calibration purposes
  • Compare and contrast results

Results from this work will be presented at the American Geophysical Union meeting in December

future work20
Future Work

Longer Term – Expanding Scope to Other Missions

At end of August we will begin rolling out the assessment process throughout other directorates in NASA

This will consist of:

  • Identifying points of contact in each mission to lead this effort
  • Conducting workshops to introduce process templates and questions, and teach the assessment process
  • Working with leads to facilitate applying process to their subject groups
  • Assisting with the processing of data and synthesizing into agency results
appendix a list of interview questions
Appendix A – List of Interview Questions

Work Patterns and Behaviors:

  • Work breakdown – categorizing major work components
  • Time to solution (intermediate and final)
  • How, when, and where do they receive, generate, process, store, and display data
  • Interactions with computing facilities and personnel
  • Other, non-science skills needed GSFC

Job Characteristics:

  • Numerical schemes
  • Degree of parallelism
  • Interprocessor latency
  • Internal and external communications
  • Characterization of data sets
  • Levels of precision
  • Provenance and logging
  • Use of metadata
  • Locality of data
  • Level of data sensitivity
  • Mission-critical jobs
  • Time-critical jobs
appendix a list of interview questions23
Appendix A – List of Interview Questions

Resource Utilization:

  • Scheduling of computing resources
  • Degree of interactivity with resources
  • Billing units and usage
  • Issues with using computing resources

Enhancements and Evolution:

  • Possible ways to enhance work
  • Potential value of enhancements
  • Technologies of interest and why
  • Desired computing platforms and why

Software, Tools, and Utilities:

  • Languages and compilers
  • Tools and utilities
  • COTS software
  • Specialized software
  • Software engineering needed
  • Customizations to any of the above

Visualization Characterization:

  • Locality of visualization resources
  • Real-time visualization
appendix a list of interview questions24
Appendix A – List of Interview Questions

Computing Characterization:

  • Interconnect type
  • Interprocessor communications speed
  • Average/Peak processing loads
  • Current computing hardware types
  • Architectures: shared or single memory
  • Number of processors, cores
  • Types of processors
  • Per-processor and total memory
  • Local cache
  • Local storage size and type

Computing Characterization:

  • Locality of computing resources
  • Specialized hardware
  • Throughput
  • Latency/Delay
  • Reliability/Operational Availability
  • Job management

Storage Characterization:

  • Locality of storage resources
  • Quantity and types of storage
appendix b list of scientists interviewed to date
Appendix B – List of Scientists Interviewed to Date

LaRC

  • Atmospheric Sciences
    • Kuan-Man Xu
    • Anning Cheng

GSFC

  • NU-WRF
    • Robert Burns
    • Jim Geiger
    • Joe Santonello
    • Sujay Kumar
    • Toshi Matsui

GSFC

  • Atmospheric Chemistry
    • Qian Tan
  • GMAO
    • Michele Rienecker
    • Ron Gelaro
    • Arlindo de Silva
    • Bill Putman

ARC

  • NEX
    • Rama Nemani
appendix b list of scientists interviewed to date26
Appendix B – List of Scientists Interviewed to Date

JPL

  • ISSM
    • Eric Larour
    • Mathieu Morlighem
    • Helene Seroussi
  • ECCO
    • Ichiro Fukumori
    • Benny Cheng
    • Ou Wang

JPL

  • Carbon Monitoring
    • Kevin Bowman
    • Robert Ferraro
    • Frank Li