1 / 14

Using Evidence Effectively

Using Evidence Effectively. Outline . Principles for persuasive use of data STAR criteria Trusted sources Understanding of evidence Kinds Strategies for framing evidence Special strategies for statistical evidence Gathering evidence.

mccarver
Download Presentation

Using Evidence Effectively

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Evidence Effectively Mary Jean Harrold

  2. Outline • Principles for persuasive use of data • STAR criteria • Trusted sources • Understanding of evidence • Kinds • Strategies for framing evidence • Special strategies for statistical evidence • Gathering evidence Mary Jean Harrold

  3. General principles for the persuasive use of data (examples) Claim: Testing computer software is very expensive and can consume enormous resources. Evidence: It is estimated that the cost of developing the Boeing 777 is $10B. 35% of this cost is for software and half of those costs were for testing (published reports and Boeing Senior Research Fellow). Thus, the estimated cost of testing software in the Boeing 777 was $1.8B. Discuss with respect to general principles: STAR and trusted sources. Mary Jean Harrold

  4. General principles for the persuasive use of data (examples) Claim: Software maintenance is the most expensive activity of software development Evidence: Software maintenance can account for as much as 2/3 of the overall cost of software production (Pressman 1992, Schach 1994). Discuss with respect to general principles: STAR and trusted sources. Mary Jean Harrold

  5. General principles for the persuasive use of data (examples) Claim: My new regression testing technique can reduce the cost of regression testing by 50% Evidence: Regression testing (testing software after it has been modified) accounts for as much as 50% of the cost of maintenance (Beizer 1990, Leung and White 1993). I have developed a technique that reduces the cost of regression testing and have performed experiments to evaluate my technique. I performed my experiments on 7 C programs and versions: • Programs range in size from 150 LOC to 500 LOC • Each program had 15 versions, with each version containing one fault (error). • Each program 1000 test cases On average, across all programs, versions, and test cases, my technique provided a savings of 60% in regression testing time. Discuss with respect to general principles: STAR and trusted sources. Mary Jean Harrold

  6. Understanding of Evidence (examples) Claim: Testing computer software is very expensive and can consume enormous resources. Evidence: It is estimated that the cost of developing the Boeing 777 is $10B. 35% of this cost is for software and half of those costs were for testing (published reports and Boeing Senior Research Fellow). Thus, the estimated cost of testing software in the Boeing 777 was $1.8B. Discuss the kinds of evidence provided to support the claim. Mary Jean Harrold

  7. Understanding of Evidence (examples) Claim: Software maintenance is the most expensive activity of software development Evidence: Software maintenance can account for as much as 2/3 of the overall cost of software production (Pressman 1992, Schach 1994). Discuss the kinds of evidence provided to support the claim. Mary Jean Harrold

  8. Understanding of Evidence (examples) Claim: My new regression testing technique can reduce the cost of regression testing by 50% Evidence: Regression testing (testing software after it has been modified) accounts for as much as 50% of the cost of maintenance (Beizer 1990, Leung and White 1993). I have developed a technique that reduces the cost of regression testing and have performed experiments to evaluate my technique. I performed my experiments on 7 C programs and versions: • Programs range in size from 150 LOC to 500 LOC • Each program had 15 versions, with each version containing one fault (error). • Each program 1000 test cases On average, across all programs, versions, and test cases, my technique provided a savings of 60% in regression testing time. Discussthe kinds of evidence provided to support the claim. Mary Jean Harrold

  9. Understanding of Evidence (examples) Claim: Monitoring software after deployment to provide continuous analysis and improvement is needed to improve and maintain quality of today’s software systems. Evidence: Developing reliable software is difficult because of software’s inherent complexity and the limited availability of resources. However, time-to-market pressures, limited development resources, testing in the many running environments, difficulty in identifying security holes, etc. results, in the release of products with missing functionalities or errors. Thus, there is a need for techniques that monitor software’s behavior during its lifetime, and enable software producers to effectively find and fix problems after the software is deployed in the user’s environments. Discussthe kinds of evidence provided to support the claim. Mary Jean Harrold

  10. Understanding of Evidence (examples) Claim: Regression testing research has had a significant impact on the way that regression testing is performed in industry. Evidence: A survey of five commercial airplane software suppliers shows that all of these companies use techniques for regression testing developed between 1988 and 1999 by academic and industrial researchers and reported in the literature. Discussthe kinds of evidence provided to support the claim. Mary Jean Harrold

  11. Strategies for framing evidence • Controlling the space given to supporting versus contrary evidence • Emphasizing a detailed story versus presenting lots of facts and statistics • Providing contextual and interpretive comments when presenting data • Putting contrary evidence in subordinate positions • Choosing labels and names that guide the reader’s response to data • Using images to guide the reader’s response to data • Revealing the value system that determines the writer’s selection and framing of data Mary Jean Harrold

  12. Strategies for framing statistical evidence • Raw numbers versus percentages • Median versus mean • Unadjusted versus adjusted • Base point for statistical comparison Mary Jean Harrold

  13. Discussion A proposal to build a new ballpark in Seattle yielded a wide range of statistical arguments: • The ballpark would be paid for by raising the sales tax from 8.2% to 8.3% over a 20-year period. • The sales tax increase is 1/10 of 1%. • This increase represents $750 per 5-person family over the 20-year period of the tax. • For a family building a new home in the area, this tax will increase the building cost by $200. • This is a $250M tax increase for the residents of the Seattle area How would you describe the costs of the proposed ballpark if you • opposed the proposal? • supported the proposal? Mary Jean Harrold

  14. Creating a plan for gathering evidence What is your plan? • Personal experiences, details from your life or lives of friends, etc. • Observational studies • Persons to interview • Questions for a questionnaire or survey • Library research • Internet research Mary Jean Harrold

More Related