1 / 45

Research Design & Analysis 2: Class 24

Research Design & Analysis 2: Class 24. Reminders: Extra class: April 10th 10-12 BAC 237 www.courseeval.com Signal Detection Theory tutorial run: y:percept at menu pick “E. Theory and Methodology” at menu pick “B. Signal Detection Theory” The introduction works, part B usually doesn’t

sevita
Download Presentation

Research Design & Analysis 2: Class 24

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Design & Analysis 2: Class 24 Reminders: • Extra class: April 10th 10-12 BAC 237 • www.courseeval.com • Signal Detection Theory tutorial run: y:percept at menu pick “E. Theory and Methodology” at menu pick “B. Signal Detection Theory” The introduction works, part B usually doesn’t • Role of theory in science • Material covered since last midterm

  2. Number Numbness … With the US court antitrust ruling against Microsoft, Bill Gates lost $13,000,000,000 yesterday • Approximately the GDP of Lebanon • Enough to send 6 space stations into orbit • Build 20 confederation bridges • Sympathy? • He is still worth $72,000,000,000 psyc 2023 class #24 (c) Peter McLeod

  3. Calculating d' From a Single Outcome matrix Data required for each point on an isosensitivity (ROC) curve requires hundreds of trials (to get accurate probabilities for Hits and False Alarms). With a few assumptions, d' can be calculated from a single outcome matrix using Signal Detection Theory. psyc 2023 class #24 (c) Peter McLeod

  4. Signal Detection Theory Assumptions 1) Noise is normally distributed. Presenting a signal on top of that noise, will therefore shift the amount of sensory activity to the right (higher), by an amount equal to that sensory systems sensitivity to that signal. The difference between the mean amount of sensory activity generated by the noise alone trials and the signal+noise trials will equal sensitivity (d') measured in z-score (standard deviation) units. psyc 2023 class #24 (c) Peter McLeod

  5. d’ Mean of noise alone distribution Mean of signal plus noise distribution Signal absent trials Signal present trials psyc 2023 class #24 (c) Peter McLeod

  6. d’ Stronger Signal (or More Sensitive Receiver) Signal present trials Signal absent trials psyc 2023 class #24 (c) Peter McLeod

  7. Signal Detection Theory Assumptions 2) Participants adopt a criterion () for dealing with those values of sensory activity that could result from either noise alone or signal plus noise (the area where the noise and signal+noise distributions overlap). If the amount of sensory activity exceeds that amount, the participant will say the detected the signal, any amount less than that and they will say they did not detect the signal. psyc 2023 class #24 (c) Peter McLeod

  8. Criterion  Say “NO” Say “YES” Range of sensory activity that could arise from either noise or the signal Signal present trials Signal absent trials psyc 2023 class #24 (c) Peter McLeod

  9. Manipulation of Bias We can now interpret the manipulation of a receiver’s motivation to say “yes” when in doubt (due to either changing expectations of payoffs) as effecting the placement of the criteria psyc 2023 class #24 (c) Peter McLeod

  10. Lax or Liberal Criterion Say “NO” Say “YES” Signal absent trials Signal present trials psyc 2023 class #24 (c) Peter McLeod

  11. Strict or Conservative Criterion Say “NO” Say “YES” Signal absent trials Signal present trials psyc 2023 class #24 (c) Peter McLeod

  12. Sensitivity Criterion location has no effect on sensitivity Sensitivity refers to the average amount of sensory activity generated by a signal compared with the average amount of noise generated sensory activity psyc 2023 class #24 (c) Peter McLeod

  13. Signal Detection Theory With two assumptions: 1) Noise is normally distributed, 2) Participants adopt a criterion () for dealing with values of sensory activity that could result from either noise or signal plus noise, The four cells of an outcome matrix (Hits, Misses, False Alarms & Correct Negatives) can be represented as areas under the two normal distributions. psyc 2023 class #24 (c) Peter McLeod

  14. Criterion  Say “NO” Say “YES” Signal absent trials Signal present trials Hits psyc 2023 class #24 (c) Peter McLeod

  15. Criterion  Say “NO” Say “YES” Signal absent trials Signal present trials Misses psyc 2023 class #24 (c) Peter McLeod

  16. Criterion  Say “NO” Say “YES” Signal absent trials Signal present trials False Alarms psyc 2023 class #24 (c) Peter McLeod

  17. Criterion  Say “NO” Say “YES” Signal absent trials Signal present trials Correct Negatives psyc 2023 class #24 (c) Peter McLeod

  18. Signal Detection Theory d’ can then be measured in z-sore units by: d' = ZFA - ZHit Tables for the z-score distribution or percent area under the normal curve typically present the z-score distances between the mean and the Criterion value (). If you are using such a table, ZFA can be found by looking up the z-score associated with (50 - False Alarm %). psyc 2023 class #24 (c) Peter McLeod

  19. Signal Detection Theory d' = ZFA - ZHit If this number (50-FA%) is positive, then the z-score to be put into the above formula will also be positive, if it is negative, the z-score value for the formula will also be negative. It is essential that the proper signs be used. A good way of checking would be to draw the distributions and the criterion and see the relationship between d' and the two z-scores. Similarly, to find ZHit, look up (50 - Hit %), again, the resulting sign will be the same as is used for the z-score in the formula. psyc 2023 class #24 (c) Peter McLeod

  20. Example d' = ZFA - ZHit = Z (50-20) - Z (50-60) z-score associated with 50-20= 30% of the normal curve = .842; for 50-60= -10% it is -.253 d' =.842-(-.253) = .842+.253= 1.095 z-score units psyc 2023 class #24 (c) Peter McLeod

  21. d’  Say “YES” Hits 60% 20% Signal present trials Signal absent trials False Alarms psyc 2023 class #24 (c) Peter McLeod

  22. d’  10% 30% Signal present trials Signal absent trials psyc 2023 class #24 (c) Peter McLeod

  23. d’  10% 30% d’=.842-(-.253) =1.095 Signal present trials Signal absent trials Z =. 842 Z =-.253 psyc 2023 class #24 (c) Peter McLeod

  24. Example 2 d' = ZFA - ZHit = Z (50-75) - Z (50-95) z-score associated with 50-75= -25% of the normal curve = -.675; for 50-95= -45% it is -1.645 d' =-.675-(-1.645) = .970 z-score units Did this subject have a Lax or strict criterion? psyc 2023 class #24 (c) Peter McLeod

  25. d’  Say “YES” 25% 45% Signal absent trials Signal present trials psyc 2023 class #24 (c) Peter McLeod

  26. d’  Say “YES” d’=-.675-(-1.645) = .970 Signal absent trials Signal present trials zFA =-. 675 zHit =-1.645 psyc 2023 class #24 (c) Peter McLeod

  27. Sensitivity to Pain: An Experimental Study of Acupuncture (Clark & Yang, 1974) psyc 2023 class #24 (c) Peter McLeod

  28. Using Theory: Chapter 15 Distinctions among: • Hypotheses • Laws • Models • & Scientific Theories psyc 2023 class #24 (c) Peter McLeod

  29. Ways to Distinguished Among Theories Quantitative vs. Qualitative Levels of Description • Descriptive • Analogical • Fundamental Domain of a Theory psyc 2023 class #24 (c) Peter McLeod

  30. Some Roles of Theory in Science • Understanding • Prediction • Organizing & Interpreting data • Generating Research psyc 2023 class #24 (c) Peter McLeod

  31. Characteristics of Good Theories • Account for data • Have explanatory relevance • Are testable • Predict novel events • Are parsimonious psyc 2023 class #24 (c) Peter McLeod

  32. Steps in Developing Theories 1. Defining scope of your theory 2. Knowing the literature 3. Formulating the theory • preparedness • using analogy • using introspection 4. Establishing predictive value 5. Testing your theory empirically psyc 2023 class #24 (c) Peter McLeod

  33. 2 4 8 16 32 ? Formulate your hypothesis then test it. (I will tell you if it is acceptable as the next number in the sequence.) Numbers get larger. What Rule Generated this Sequence? psyc 2023 class #24 (c) Peter McLeod

  34. Hexp: All “X” also have a O X 1 2 3 4 psyc 2023 class #24 (c) Peter McLeod

  35. Hexp: All “X” also have a Confirmational Strategy: Will have a star if Hexp is correct Even if there is a star, however, your hypothesis might not be correct. X psyc 2023 class #24 (c) Peter McLeod

  36. Hexp: All “X” also have a Disconfirmational Strategy: If there is an “X” then our theory is definitely wrong. O psyc 2023 class #24 (c) Peter McLeod

  37. Hexp: All “X” also have a O Irrelevant. The hypothesis says nothing about “Os.” psyc 2023 class #24 (c) Peter McLeod

  38. Hexp: All “X” also have a Irrelevant. The hypothesis doesn’t say all stars have Xs. psyc 2023 class #24 (c) Peter McLeod

  39. Testing Theories Both confirmation and disconfirmational strategies can be used to test theories. It is best to use both. psyc 2023 class #24 (c) Peter McLeod

  40. Review: Material Since Last Midterm... • Quasi-analytic experiments: Bivalent correlation designs • Calculate the extent to which the two variables are systematically related • Graph data (scatterplot): predictor (assumed causal or IV) variable on abscissa (X-axis) and criterion or DV on ordinate (Y-axis) • Pearson's product moment correlation coefficient (for interval or ratio data) measures the direction and degree of association. • r is the mean of z-score cross-products psyc 2023 class #24 (c) Peter McLeod

  41. Review: Material Since Last Midterm... • cautions - assumes linear relations, truncated ranges, outliers, heteroscedasticity, combining group data • So: examine scatterplots!! • Problems interpreting the results of this type of research: • third variable problem • directionality (not always an issue), • regression artifact (e.g., Rushton), • floor and ceiling effects, • look for converging evidence psyc 2023 class #24 (c) Peter McLeod

  42. Review: Material Since Last Midterm... • Correlation versus ex-post facto design • Interpretation problems are not related to the statistical choice, rather due to the design • Causation not a simple concept • Simpson's Paradox. • Partial correlation • Remember: can be other confounding variable not measured • Semipartial correlations (sometimes called Part correlations) psyc 2023 class #24 (c) Peter McLeod

  43. Review: Material Since Last Midterm... • Multiple Correlation and Regression • formulae, types, importance of order of entry • considerations about R2 • Developmental Designs • longitudinal, cross-sectional, & cohort-sequential • Cohort: a group with common experiences psyc 2023 class #24 (c) Peter McLeod

  44. Review: Material Since Last Midterm... • Discrete trials designs - Psychophysics • Signal Detection Theory • teasing apart sensory ability and decision to say “yes” • Isosensitivity curves also called ROC (receiver operating characteristic) curves: • calculate d' from hit and false alarm probabilities (using tables of areas under the normal curve) psyc 2023 class #24 (c) Peter McLeod

  45. Review: Material Since Last Midterm... • Scientific theories: types of theories, functions of theories • Evaluation on the basis of : parsimony, testability, precision • Confirming vs. disconfirming strategies (confirmational bias) psyc 2023 class #24 (c) Peter McLeod

More Related