methods for learning metrics n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Methods for Learning Metrics PowerPoint Presentation
Download Presentation
Methods for Learning Metrics

Loading in 2 Seconds...

play fullscreen
1 / 24

Methods for Learning Metrics - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

Methods for Learning Metrics. A workshop in conjunction with Metrics 2005 September 19, 2005 Como, Italy. Organized by: Carolyn Seaman Letizia Jaccheri. Rough Schedule for Today. 9am    Introductions, team-building, and preliminaries 10am    Break

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Methods for Learning Metrics' - ellis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
methods for learning metrics

Methods for Learning Metrics

A workshop in conjunction with

Metrics 2005

September 19, 2005

Como, Italy

Organized by: Carolyn Seaman

Letizia Jaccheri

rough schedule for today
Rough Schedule for Today

9am    Introductions, team-building, and preliminaries

10am    Break

10:30m   Presentations of position papers and discussion

12:30pm  Lunch

2pm    Teaching design exercises

3:30pm    Break

4:00pm Group photo

4:05pm Discussion

5:30pm Close

introductions
Introductions
  • Step 0: Name, institution, job title
  • Step 1:
    • Pair up
    • Share work details – research, teaching, management, development, consulting, etc.
    • Introduce your partner
introductions1
Introductions
  • Step 2
    • Pair up differently
    • Share who you are outside of work:
      • What’s important to you?
      • What do you enjoy?
      • What defines you?
    • Introduce your partner
introductions2
Introductions
  • Step 3
    • Pair up differently
    • Share what you have taught
      • Other than software engineering courses
      • Subject
      • Context
      • Audience
    • Introduce your partner
presentations
Presentations
  • Luigi Buglione, Alain Abran:
    • Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP
  • Linda M. Ott
    • Developing Healthy Skepticism not Disbelief - Problems in Teaching Software Metrics
  • Giuseppe Lami
    • Learning Metrics for Natural Language Requirements in an Under-Graduate Course
  • Letizia Jaccheri, Sandro Morasca
    • Understanding the role of software metrics in an Empirical Software Engineering Course for PhD students
    • overview of course
  • Maurizio Morisio, Marco Torchiano
    • TDE: a method and tool to teach OOP
teaching design exercise
Teaching Design Exercise
  • Split into 3 groups
  • Everyone does task 1 in their group
  • Each group presents their course design
  • Divide tasks 2, 3, and 4 among the three groups
  • Each group presents their work
  • Discussion
teaching design exercise task 1
Teaching Design ExerciseTask 1
  • Subject to be taught:

How to change money in a foreign country

  • Design a course, including:
    • Objectives and overall structure
    • Outline of material to be presented
    • Exercises
    • Assessments
task 1 cont
Task 1 (cont.)
  • Course should be based on collaborative exercises, e.g.
    • Producing a film or video as a group
    • Creating a piece of artwork
    • Collaboratively designing a poster on some topic
    • Deciding how something should be built (i.e. making design decisions)
  • … and student reflection and feedback, e.g.
    • Student presentations on reading material
    • Personal logs or journals written by students
    • Essays about lessons learnt
    • Oral examinations
    • Etc.
task 1 results
Task 1 Results
  • Group A
    • Objectives:
      • To provide the skills needed to end up with the needed currency in hand without breaking rules
    • Materials
      • Talks – money, value of money, different exchange rates, why it is a problem
      • Motivate importance of topic – why it will be helpful to them
      • Example scenario – going abroad to a conference
      • Converting cash and converting with credit card
    • Group exercises
      • Teams – one person on each team doesn’t know how to convert money, others teach him
    • Assessments
      • Short term – give same value of money in different currencies to different students and they have to convert it to dollars, then they have to write a log, which is evaluated against the objectives
      • Public exposition – so they can learn from mistakes and from other students
task 1 results1
Task 1 Results
  • Group B
    • Main objective – to teach someone how to make available local currency
    • Half day course
    • Present an IDEF0 Process model for explaining the conversion process – inputs to outputs and mechanisms to do that
    • Topics:
      • What is money – different forms, cash, credit cards, etc.
      • Cost of living in different countries
      • Fees, commissions
      • Exchange rates
      • Where can we change money?
      • Outputs – traveler’s checks, cash, receipt
      • Changing money back into original currency
      • Teller machines, bureaus de change
      • Flowchart to describe different options
    • Exercise – Excel spreadsheets with change rates and world events related to changes
    • Role playing – purchase of currency
    • Assessment – role play, ordinal scale (0-3), strengths, areas for improvement
    • SMART evaluation
    • Feedback – oral to group; written through evaluation forms; improvements and teaching style
task 1 results2
Task 1 Results
  • Group C
    • 8 sections: objectives, different ways to change, considerations, economy, safety and security, convenience, exercises, assessment
    • Objectives – assumed transactions are legal
      • Complete the transaction – end up with different currency than started with
      • Most economically and good exchange rate and good commission
      • Convenient and safe transaction
    • Different ways
      • Before the trip – bank or airport
      • During the trip – bank, airport, or hotel, ATM or change office
    • Considerations
      • Economy, safety, convenience
    • Ranked methods in decreasing order of:
      • Economy: Home bank, foreign bank, ATM, airport/hotel/change office
      • Safety and security: Bank, hotel, ATM, airport, change office
      • Convenience (depends on circumstances): Hotel, home bank
    • Exercises
      • Describe scenarios – circumstances of trip, situation
      • Students write what, how, why
    • Assessment
      • Groups of some size – critique each others’ responses and share ideas
    • Poster – Foreign Exchange 101
teaching design exercise task 2
Teaching Design ExerciseTask 2
  • Add to or modify the course you just designed to include a multi-faceted approach, including, e.g.
    • Psychological and emotional dimensions
    • Ethical concerns
    • Legal issues
task 2 results
Task 2 Results
  • Increased the number of constraints in the process model
  • Psychological and emotional – language barrier, style of writing (characters), may not know exchange rate, i.e. what to expect, security issues (older or not-computer-literate people), location of ATM, events you’re not aware of at an ATM
  • Ethical concerns – uneasy with government, not inclined to change a lot of cash, reduce expenditures if not comfortable with government
  • Legal constraints – constraints on amount of currency allowed in or out, types of money that can be exchanged (coins or bills), black market, merchants changing under the table
teaching design exercise task 3
Teaching Design ExerciseTask 3
  • Now consider the situation where you are teaching a course on changing money, but in the context of a more general course on travel
  • What are the learning objectives concerning changing money?
  • What learning objectives about travel could be illustrated by teaching how to change money?
  • How would your course be different?
task 3 results
Task 3 Results
  • Travel considerations – planning strategies (long term/prepaying vs. short term/more expensive/very flexible)
  • How to assess the need for money – local customs (credit cards accepted or not)
  • How to manage difficulties and travel emergencies – have multiple potential solutions
  • Travel planning illustrated by changing money
  • Differences in culture – orderly vs. chaotic, also affects changing money strategy
  • Different points of view
  • Urban vs. rural
  • Environment and government
  • Whether or not you need to change money – some places accept multiple currencies, esp. borders
teaching design exercise task 4
Teaching Design ExerciseTask 4
  • Now suppose that you are teaching your course, and some students say…

“Why do we need to know this? With the Euro, we’ll never need to change money again!”

  • How do you teach them that the concepts are still important?
task 4 results
Task 4 Results
  • Teach them that the concepts are still important
  • Use an experience that they might have, e.g. someone has to plan where they work in the next 5 years, have to compare wages in different countries
  • Traveling and buying goods – examples to show them that this will help them understand what’s going on
  • Experiment – go to EBay and buy Italian wine in Euros – how much will it cost?
discussion
Discussion
  • Can we apply what we’ve designed for teaching money-changing skills to teaching software metrics?
  • Task 1 – collaborative exercises and student feedback
  • Task 2 – balancing multiple perspectives
  • Task 3 – “sneaking in” metrics
  • Task 4 – dealing with “disbelievers”
applying ideas to metrics
Applying Ideas to Metrics

Teaching Currency Exchange:

  • Learning Objectives
    • To provide the skills needed to end up with the needed currency in hand without breaking rules
    • To teach someone how to make available local currency
    • Complete the transaction – end up with different currency than started with
    • Getting a good exchange rate and good commission
    • Convenient and safe transaction

Teaching Metrics:

  • Learning Objectives
    • To provide skills needed to develop metrics without violating personnel rules and rules of measurement theory
    • Goal-driven measurement (GQM) – how derive measures from local goals (multiple levels)
    • End up with a quantity that is meaningful and that is understood – measure what you intend to measure – and that is useful, provides some benefit
    • Designing efficient measures
    • Measuring efficiently
    • Cost of collecting metrics
    • Return on investment of measurement
    • Not accepting counterfeit information – knowing when the data is bogus
    • Audience of data – how to use, who should use it, security of data
applying ideas to metrics1
Applying Ideas to Metrics

Teaching Currency Exchange:

  • Material covered:
    • Why it is important
    • Why it’s a problem
    • Basic concepts - money, value of money, different exchange rates,
    • Different ways of converting
      • Before and after trip
      • Bank, ATM, bureau de change, airport/hotel
    • Different forms of money – cash, credit cards, travelers checks
    • Process
      • IDEF0 process model
      • Flowchart
      • Inputs and outputs
    • Cost of living in different countries
    • Fees, commissions
    • Exchange rates
    • Changing money back into original currency
    • Economy, safety, convenience

Teaching Metrics:

  • Material covered:
    • Motivation
    • Measurement foundations and theory
    • Operational definitions of metrics
    • Different analysis methods
    • Process vs. product vs. personnel metrics
    • Dangers in converting measurement units; different scales for effort, e.g. – needed for benchmarking
    • Different modes of data collection
    • Different measurement purposes; application of specific metrics
    • Different measurement scales – ordinal, interval, etc.
    • Measurement process
    • Comparing different processes
    • Variety of programmer salaries/costs
    • How measures work in different types of organizations
    • Use of normalized metrics
    • Costs of data collection/analysis
    • Auditing
    • Using measurement results for improvement
    • Showing progress through measurement results
    • Tie the results to process, not to people
    • Cost/benefit analysis of measurement
    • Evaluating/assessing intangibles
applying ideas to metrics2
Applying Ideas to Metrics

Teaching Currency Exchange:

  • Material covered (cont.):
    • Psychological and emotional – language barrier, style of writing (characters), may not know exchange rate, i.e. what to expect, security issues (older or not-computer-literate people), location of ATM, events you’re not aware of at an ATM
    • Ethical concerns – uneasy with government, not inclined to change a lot of cash, reduce expenditures if not comfortable with government
    • Legal constraints – constraints on amount of currency allowed in or out, types of money that can be exchanged (coins or bills), black market, merchants changing under the table
    • Strategy – for both travel and money changing
    • Travel planning illustrated by changing money
    • Differences in culture – orderly vs. chaotic, also affects changing money strategy
    • Urban vs. rural
    • Environment and government
    • Whether or not you need to change money – some places accept multiple currencies, esp. borders

Teaching Metrics:

  • Material covered:
    • Culture of quality
    • Measurement textbooks translated into different languages
    • Legal obligations of measurement
    • Acceptance of measurement programs by developers
    • Paranoia
    • Personal and professional vulnerability
    • Inappropriate uses of measurement
    • Distinguishing randomness from trends
    • Using measurement for building SLA’s and other types of contracts
    • Third party assessment – ethics and legalities
    • Subversion of metrics by changing behavior
    • Using different measures at different stages of project and converting between them
    • Influence of different programming languages
applying ideas to metrics3
Applying Ideas to Metrics

Teaching Currency Exchange:

  • Exercises:
    • one person on each team doesn’t know how to convert money, others teach him/her
    • Excel spreadsheets with change rates and world events related to changes
    • Role playing – purchase of currency
    • Oral feedback to group; written feedback through evaluation forms; improvements and teaching style
    • Describe scenarios – circumstances of trip, situation; students write what, how, why
    • Poster – Foreign Exchange 101
    • Comparing wages in different countries
    • Examples to show them how changing money will help them understand larger issues of economics and world events
    • Experiment – go to EBay and buy Italian wine in Euros – how much will it cost?

Teaching Metrics:

  • Exercises:
    • Group members teach one member about metrics; students create pedagogical materials
    • Project manager version of Monopoly
    • Process control charts – show real examples; relate to real events, e.g. vacations, reorgs
    • Refactoring exercise – take measurements, refactor according to metrics, re-measure
    • Evaluating the validity of metrics in a published study
    • Role playing manager and developer
    • Students look at own company’s data and apply estimation models and relative errors with actuals
    • Scenarios – discuss how expected values would be adjusted in that situation
    • Give a real scenario and set of measurement data, students make inferences and predictions about project
    • Differences between metrics on different types of software (e.g. OS, embedded, etc.)
    • Compare functionally equivalent software using a set of metrics to see differences
    • Measurement profiles of different teams of developers – what can you get with the same amount of XXX from different teams?
    • Design a metrics that demonstrates your value as a software engineer
    • Look at history of software industry through measurement snapshots
    • Outsourcing exercise
applying ideas to metrics4
Applying Ideas to Metrics

Teaching Currency Exchange:

  • Assessments:
    • Short term – give same value of money in different currencies to different students and they have to convert it to dollars, then they have to write a log, which is evaluated against the objectives
      • Public exposition – so they can learn from mistakes and from other students
    • role play, ordinal scale (0-3), strengths, areas for improvement
      • SMART evaluation
    • Groups of some size – critique each others’ written responses to set of questions and share ideas

Teaching Metrics:

  • Assessments:
    • Compare different students’ designs for the same measurement problem
    • Compare different students’ process assessments given same data and assessment process
    • Look at different metrics and how they relate to the underlying value of the software and how the metrics could be combined properly
    • Given a scenario, which metrics are most suitable and why?
    • Manager/developer role play – “why do I have to collect this?”
    • Role playing with appraisals – making sense of information given by a project team