Beyond the os textbook getting started in os research
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Beyond the OS textbook - getting started in OS research PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on
  • Presentation posted in: General

Beyond the OS textbook - getting started in OS research. Kim Keeton HP Labs [email protected] Dilma Da Silva IBM Research [email protected] http://www.hpl.hp.com/personal/Kimberly_Keeton http://www.research.ibm.com/people/d/dilma/. Getting started - a steep uphill climb?.

Download Presentation

Beyond the OS textbook - getting started in OS research

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Beyond the os textbook getting started in os research

Beyond the OS textbook - getting started in OS research

Kim Keeton

HP Labs

[email protected]

Dilma Da Silva

IBM Research

[email protected]

http://www.hpl.hp.com/personal/Kimberly_Keeton

http://www.research.ibm.com/people/d/dilma/

2007 SOSP Women's Workshop


Getting started a steep uphill climb

Getting started - a steep uphill climb?

Completed

Research

Project

Textbook

Lessons

2007 SOSP Women's Workshop


A little bit about kim

A little bit about Kim

  • Education

    • PhD, Computer Science, UC Berkeley (1999)

    • MS, Computer Science, UC Berkeley (1994)

    • BS, Computer Engineering and Engineering and Public Policy, Carnegie Mellon (1991)

  • Career

    • HP Labs, 1999 - present

  • Research interests

    • Storage systems, storage management, dependability, information management

2007 SOSP Women's Workshop


A little more about kim

A little more about Kim

2007 SOSP Women's Workshop


A little bit about dilma

A little bit about Dilma

  • Education

    • PhD, Computer Science, Georgia Tech (1997)

    • MS, Computer Science, Univ Sao Paulo, Brazil (1990)

    • BS, Computer Science, Univ of Sao Paulo, Brazil (1986)

  • Career

    • IBM Research, 2000-present

    • Assistant Professor, University of Sao Paulo, 1996-2000

  • Research interests

    • Operating systems, distributed systems

      • Parallel computing, software engineering, mobile computing

2007 SOSP Women's Workshop


A little more about dilma

A little more about Dilma

  • Musical activities: embracing lack of “natural talent”

  • Passions: books , knitting

  • Navigating politics: large extended family 

2007 SOSP Women's Workshop


Outline

Outline

  • Getting prepared

  • Picking the “right” problem

  • Finding a creative solution

  • Quantitatively evaluating your solution

  • Communicating your results

  • The lessons along the way

2007 SOSP Women's Workshop


Getting prepared system building

Getting prepared: system building

  • Sharpening your system-building skills

    • Find teams and partners that are right for you

    • Learning by hand-on work, learning by example (Knuth’s literary programming)

    • Develop aesthetic for elegant, simple solutions; fight temptation (yours, others) to build Taj Mahals!

    • Your arsenal:

      • Tool chains, scripting, debugging, performance evaluation, tracing

      • Model checking

      • Simulation

2007 SOSP Women's Workshop


Getting prepared map the territory

Getting prepared: Map the territory

  • Read papers to learn … and to reflect on the approach taken

    • Identify the good, the bad, and the ugly

    • Reading groups can be useful

  • Be curious … with focus

  • Challenge your pride & prejudice tendencies

  • Common ineffective behavior:

    • Blindness to whole bodies of work

    • Drowning on papers

  • 2007 SOSP Women's Workshop


    Picking the right problem

    There are lots of right problems, not just one!

    Many ways to find a great problem to work on:

    Understand technology trends and changing usage patterns

    Run experiments to understand real problems

    Attend talks, including interesting ones in other areas

    Look for open questions in future work section

    Carve out niche in larger group project

    Do an internship to understand “real world” problems

    Solve a problem that impacts you personally

    Stumble across it while solving another problem

    Picking the “right” problem

    2007 SOSP Women's Workshop


    Picking the right problem1

    Picking the “right” problem

    • Do “real stuff”: solve problems that someone cares about

      • Exposure to industrial-strength real-world problems

    • Pick a small problem and let it grow

      • Course projects are a great starting point

    • Easier to make contributions if you arrive early to a field

      • Part of the contribution is framing the research question

      • “Good research is done with a shovel, not with tweezers.” -- Roger Needham

    • Things to remember:

      • It doesn’t need to be ACM-Dissertation-Award perfect

      • You’re not going to work on it for your entire career

      • Make sure you are excited enough to work on it for several years

    2007 SOSP Women's Workshop


    Finding a creative solution

    Keep things simple unless there’s a good reason not to

    Pick innovation points carefully; be compatible everywhere else

    Best results are obvious in retrospect

    Borrow techniques from other fields

    Machine learning

    Data mining

    Optimization

    Control theory

    User interfaces

    Finding a creative solution

    2007 SOSP Women's Workshop


    Quantitatively evaluating your solution

    Computer systems ideas must be quantitatively evaluated

    Apply the scientific method

    Use experiments to answer questions

    Use standard benchmarks

    Use measurements of real systems

    Understand your results

    Quantitatively evaluating your solution

    2007 SOSP Women's Workshop


    Apply the scientific method

    Apply the scientific method

    • Goal: (dis)prove hypothesis

      • Not to measure what’s easy to instrument

      • Think of it as minimizing the effort to prove your point

    • Useful exercise #1: back-of-the-envelope calculations to see potential of an idea

    • Useful exercise #2: draw graph you expect to see

    • If experimental results differ:

      • Flaw in intuition? Flaw in system? Flaw in measurement?

      • Uncovered interesting behavior?

    • Use intuition to ask questions, not to answer them

    2007 SOSP Women's Workshop


    Use experiments to answer questions

    Use experiments to answer questions

    • Microbenchmarks

      • Focus on a particular aspect of system

      • Easy to reason about

    • Macrobenchmarks

      • Do not focus on particular aspect of system

      • More realistic evaluations of idea

    2007 SOSP Women's Workshop


    Use standard benchmarks

    Use standard benchmarks

    • “For better or worse, benchmarks shape a field.” – Dave Patterson

    • Benefits of standard workloads:

      • Workload is well-defined, agreed upon by community

      • Provides comparability with other systems

      • May provide load generation tools

    • Transaction Processing Performance Council (TPC) (http://www.tpc.org)

    • Standard Performance Evaluation Corporation (SPEC) (http://www.spec.org)

    2007 SOSP Women's Workshop


    Use measurements of real systems

    Use measurements of real systems

    • Traces of actual system behavior will ground your work in reality

    • Storage Networking Industry Association (SNIA) IOTTA Repository (http://iotta.snia.org/repository/resources/)

      • System call traces: Coda, Drew Roselli's traces and others

      • Network file system traces: Berkeley Auspex, Harvard NFS

      • Block-level storage traces: HP Labs cello

    • Computer Failure Data Repository (CFDR) (http://cfdr.usenix.org/)

      • Node outages in more than 20 large HPC clusters

    • Other options:

      • Work with industrial partners

      • Measure your local systems

    2007 SOSP Women's Workshop


    Understand your results

    Understand your results

    • Use knowledge of statistics to evaluate results

      • Example: perform multiple iterations of experiment and calculate confidence intervals

    • Give yourself enough time to collect and understand the data

      • Things always take longer than you think

      • You won’t always get the measurements right the first time

      • You may decide you need measure something new to make the point

    2007 SOSP Women's Workshop


    Communicating your results

    Communicating your results

    • Report in sufficient detail for others to reproduce results

    • Deadlines: friend or foe?

    • Ask colleagues for feedback

      • Early drafts of papers

      • Practice talks

    • Benefits

      • You’ll get useful feedback

      • People will be more aware of what you’re doing

    2007 SOSP Women's Workshop


    Learning from mistakes

    Learning from mistakes

    • Innovation involves risk

      • “Would you like the formula for success?" TJ Watson asked. "Double your rate of failure."

    • Being smart: early failure detection

    • Inventory of “failures” and “successes”

      • We learn from both

    2007 SOSP Women's Workshop


    Summary

    Summary

    Completed

    Research

    Project

    • It’s a learning process

    • We can make the journey smoother:

      • Improve system building skills

      • Learn from past and current efforts in the research community

      • Learn about trends and problems in the field

      • Adhere to scientific methods

      • Practice careful evaluation

      • Communicate

    Textbook

    Lessons

    2007 SOSP Women's Workshop


    More importantly

    More importantly …

    HAVE FUN!

    2007 SOSP Women's Workshop


    Acknowledgments

    Eric Anderson

    Carla Ellis

    Manish Gupta

    Jim Gray

    Arif Merchant

    Jeff Mogul

    Brad Morrey

    Dave Patterson

    Sharon Perl

    Anna Povzner

    Craig Soules

    Alistair Veitch

    Janet Wiener

    John Wilkes

    Jay Wylie

    Acknowledgments

    2007 SOSP Women's Workshop


  • Login