Chapter 8 Data Evaluation and Interpretation

1. Chapter 8Data Evaluation and Interpretation

2. Learning Objectives • Appreciate the role of critical thinking in evaluating research findings • Assess the dependability of information sources (print and digital) • Assess the quality of your evidence • Interpret your findings accurately and without bias • Understand that “certainty” in research is an elusive goal

3. Learning Objectives (continued) • Recognize common errors in reasoning and statistical analysis • Understand that research carries the potential for error

4. Evaluating and Interpreting Not all information is equal. Not all interpretations are equal either:

5. Evaluate the Sources Start the evaluating and interpreting process by evaluating your sources: • Determine the currency of the source. • Assess the reputation of a printed source. • Assess the perspective of a digital source. • Consider the possible motives of those who have funded astudy. • Cross-check the source against other, similar sources.

6. Evaluate the Evidence Then evaluate the evidence. Evidenceis any finding used to support or refute a particular claim: • Determine the sufficiency of the evidence. • Differentiate hard from soft evidence. Hard evidence is facts, expert opinion, and statistics; soft evidence is uninformed opinion and speculation. • Decide whether the presentation of evidence is balanced and reasonable. • Consider how the facts are being framed.

7. Interpret Your Findings Then try to uncover the truth of your findings: • Identify your level of certainty. Is it a conclusive answer (irrefutable), the probable answer (likely), or an inconclusive answer (uncertain)? • Examine the underlying assumptions. Assumptions are ideas we often accept without proof. Does the information you located make assumptions? • Be alert to personal bias. Does the information you’ve found seem biased? • Consider other possible interpretations. Is there another way you can look at what you’ve found?

8. Avoid Distorted or Unethical Reasoning Next apply reason to the information you’ve found by seeing if it contains errors in reasoning, such as the following logical errors: • Faulty generalization. Generalizing on the basis of limited evidence. Ask yourself: “How much can we generalize from these findings?” • Faulty causal reasoning. Confusing or distorting why one thing caused or will cause another thing to happen. Ask yourself: “Did X possibly, probably, or definitely cause Y?”

9. Avoid Distorted or Unethical Reasoning (continued) • Faulty statistical analysis. Interpreting the numbers in a misleading way. Ask yourself: “Are these numbers really accurate?” Avoid: * sanitized statistics (numbers that have manipulated to obscure the facts) * meaningless statistics (quantifying things that really can’t be quantified) * undefined averages (averages that are mathematically skewed) * distorted percentage figures (percentages that aren’t mathematically valid or ignore the margin or error)

10. Avoid Distorted or Unethical Reasoning (continued) * bogus rankings (items compared on the basis of ill-defined criteria) * confusion of correlation and causation (misinterpreting coincidence as truth) * biased meta-analysis (using only findings from other studies that support your own bias) * fallible computer model (trusting a computer with biased assumptions programmed in) * misleading terminology (using statistical terms that hide meaning)

11. Acknowledge the Limits of Research Lastly, understand that there are limits to even the most careful research you use to support your own conclusions. • Not all research is valid (correct) and reliable (repeatable). Even a valid survey can lead to invalid results when respondents misunderstand survey questions, answer questions dishonestly, or respond in a way that the think they ought to. Even a reliable survey can become unreliable when the same respondents answer the same survey questions differently the second time.

12. Acknowledge the Limits of Research (continued) • Research studies can be flawed. Epidemiological studies (the study of populations), laboratory studies (studies conducted under laboratory conditions), and human exposure studies (clinical trials) can go wrong in a variety of ways. • Research can sometimes be deceptively reported. Sometimes researchers avoid reporting results that are awkward, embarrassing, or unpopular.

13. Review Questions 1. What is the difference between evaluation and interpretation of information? 2. What are the five ways to evaluate a source? 3. What are four ways to evaluate evidence? 4. What are hard evidence and soft evidence? 5. What are the four ways of interpreting your findings? 6. What are the three levels of certainty? 7. What is a faulty generalization?

14. Review Questions (continued) 8. What is faulty causal reasoning? 9. What are five types of faulty statistical reasoning? 10. In what three ways can research findings be limited?