1 / 31

Understanding Relative and Absolute Ratings in Scoring Tables for Effective Evaluation

This session focuses on mastering the distinctions between relative and absolute ratings in scoring tables—essential tools for making informed decisions in project evaluation. Participants will engage in practical exercises, utilizing various concepts related to NASA's probe project to develop skills in assessing widget designs based on defined criteria. The agenda includes an introduction to scoring tables, relative comparisons, and the impact of mapping on selection. By the end of the session, attendees will gain insights into which rating system best suits different evaluation scenarios.

xiang
Download Presentation

Understanding Relative and Absolute Ratings in Scoring Tables for Effective Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. As Class Convenes • Find your team • Sign attendance form • Insert any work due today and • Return folder to the front desk

  2. Session Agenda • Getting Started 5 min • UsingRelative15 minRanking in Scoring Tables • UsingAbsolute30 minRatings in Scoring Tables

  3. Learning Objective Achieve awareness of the differences between RelativeRatings and AbsoluteRatings in completing Scoring Tables (Matrices)

  4. Session Agenda • Getting Started 5 min • UsingRelative15 minRating in Scoring Tables • Using Absolute 30 min Ratings in Scoring Tables

  5. Sample Scoring Table

  6. Determining Ratings: The Text Book Says • Use relative comparisons • Use a simple rating scale 1 = much worse than reference to 5 = much better than reference

  7. Problems With Relative Rating • Scale Compression • if reference concept is the best relative to criterion 1, what rating values are available for criterion 1? • only 1 (much worse), 2 (worse) & maybe 3 (same) • Not rigorous for non experts

  8. In Some Cases We Can Do Better • Use absolute instead of relative rating of concepts • Use engineering science to predict the values of the criteria for the concepts

  9. A Class Exercise For a two criteria, three “concepts” problem, you will evaluate the “concepts” • using relative rating • using absolute values with mapping to a rating

  10. The Problem NASA wants to launch another probe to Mars. This probe requires a widget that TeamUSA has agreed to design and make. NASA wants a low mass, low costwidget. Three feasible concepts have been created; which one should be selected?

  11. Three Widget Concepts • anABS plasticcube4.8 cm on a side, $2000 • a Bakeliterod 5.0 cm long & 2.5 cm in diameter, $4000 • a glassrod 5.0 cm long & 2.8 cm in diameter, $6000

  12. Relative Rating Scale • Much Worse • Worse • About the Same • Better • Much Better light isbetterthan heavy

  13. Rate Concepts Using Relative Comparison Method Using Concept C as the reference, rate Concepts A & B 3 1 4

  14. Session Agenda • Getting Started 5 min • Using Relative 15 min Ranking in Scoring Tables • UsingAbsolute30 minRatings in Scoring Tables

  15. Using Absolute Ratings • Evaluate the criterion for each concept, i.e., calculate the mass of each concept

  16. Effect of Mapping on Selection

  17. Using Absolute Ratings • Why can't you use the calculated mass directly in the Scoring Table? • You can change the results in the table by just changing the units (metric tons vs micro grams)

  18. Using Absolute Ratings (cont’d) • You needto map the calculated masses into a dimensionless 1 to 5 rating scale • What is a map? A map is a function that converts one value (e.g., a criterion value) into another value (e.g., a rating)

  19. 112 37 80 Rate The Concepts Using Absolute Comparison Method Using your team’s map, rank Concepts A, B, & C

  20. Summary of Mapping Results 2 4.5 3 1.1 2 1.6 4 4.8 4.2

  21. Analysis of Scoring Tables • Does the mapping change the relative rankings? no, A is always the worst and B is always the best • What does change with different mappings? the value in the table, A ranges from 1 to 4

  22. Effect of Mapping on Selection

  23. Effect of Mapping on Selection

  24. Effect of Mapping on Selection

  25. Effect of Mapping on Selection

  26. Effect of Mapping on Selection

  27. Effect of Mapping on Selection

  28. Effect of Mapping on Selection

More Related