Educational Research: Instruments (“ caveat emptor ”) - PowerPoint PPT Presentation

educational research instruments caveat emptor n.
Skip this Video
Loading SlideShow in 5 Seconds..
Educational Research: Instruments (“ caveat emptor ”) PowerPoint Presentation
Download Presentation
Educational Research: Instruments (“ caveat emptor ”)

play fullscreen
1 / 100
Educational Research: Instruments (“ caveat emptor ”)
Download Presentation
Download Presentation

Educational Research: Instruments (“ caveat emptor ”)

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Educational Research: Instruments (“caveat emptor”) EDU 8603 Educational Research Richard M. Jacobs, OSA, Ph.D.

  2. Instruments… • tools researchers use to collect data for research studies (alternatively called “tests”)

  3. The types of instruments… 1. Cognitive Instruments 2.Affective Instruments 3.Projective Instruments

  4. 1. Cognitive instruments... • Measure an individual’s attainment in academic areas typically used to diagnose strengths and weaknesses

  5. Types of cognitive instruments... • achievement tests …provide information about how well the test takers have learned what they have been taught in school …achievement is determined by comparing it to the norm, the performance of a national group of similar students who have taken the same test

  6. aptitude tests …measure the intellect and abilities not normally taught and often are used to predict future performance …typically provide an overall score, a verbal score, and a quantitative score

  7. 2. Affective instruments... • Measure characteristics of individuals along a number of dimensions and to assess feelings, values, and attitudes toward self, others, and a variety of other activities, institutions, and situations

  8. Types of affective instruments... • attitude scales …self-reports of an individual’s beliefs, perceptions, or feelings about self, others, and a variety of activities, institutions, and situations …frequently use Likert, semantic differential, Thurstone , or Guttman scales

  9. values tests …measure the relative strength of an individual’s valuing of theoretical, economic, aesthetic, social, political, and religious values

  10. personality inventories …an individual’s self-report measuring how behaviors characteristic of defined personality traits describe that individual

  11. 3. Projective instruments... • Measure a respondent’s feelings or thoughts to an ambiguous stimulus

  12. Primary type of projective test... • associational tests …participants react to a stimulus such as a picture, inkblot or word onto which they project a description

  13. Selecting an instrument... 1. determine precisely the type of instrument needed 2. identify and locate appropriate instruments 3. compare and analyze instruments 4. select best instrument

  14. Instrument sources… Burros’ Mental Measurements Yearbook Tests in Print PRO-ED Publications Test Critiques Compendium ETS Test Collection Database ERIC/AE Test Review Locator ERIC/Burros Test Publisher Directory

  15. Rules governing the selection instruments... 1. the highest validity 2. the highest reliability 3. the greatest ease of administration, scoring, and interpretation 4. test takers’ lack of familiarity with instrument 5. avoids potentially controversial matters

  16. Administering the instrument... 1. make arrangements in advance 2. ensure ideal testing environment 3. be prepared for all probable contingencies

  17. Two issues in using instruments... 1.Validity: the degree to which the instrument measures what it purports to measure 2.Reliability: the degree to which the instrument consistently measures what it purports to measure

  18. Types of validity... 1. Content validity 2.Criterion-related validity 3.Construct validity

  19. 1.Content validity: the degree to which an instrument measures an intended content area

  20. forms of content validity… …sampling validity: does the instrument reflect the total content area? …item validity: are the items included on the instrument relevant to the measurement of the intended content area?

  21. 2.Criterion-related validity: an individual takes two forms of an instrument which are then correlated to discriminate between those individuals who possess a certain characteristic from those who do not

  22. forms of criterion-related validity… …concurrent validity: the degree to which scores on one test correlate to scores on another test when both tests are administered in the same time frame …predictive validity: the degree to which a test can predict how well individual will do in a future situation

  23. 3.Construct validity: a series of studies validate that the instrument really measures what it purports to measure

  24. Types of reliability... 1. Stability 2.Equivalence 3.Internal consistency

  25. 1.Stability (“test-retest”): the degree to which two scores on the same instrument are consistent over time

  26. 2.Equivalence (“equivalent forms”): the degree to which identical instruments (except for the actual items included) yield identical scores

  27. 3.Internal consistency (“split-half” reliability with Spearman-Brown correction formula , Kuder-Richardson and Cronback’s Alpha reliabilities, scorer/rater reliability): the degree to which one instrument yields consistent results

  28. Terms associated with instruments...

  29. Data… …the pieces of information researchers collect through instruments to examine a topic or hypothesis

  30. Constructs… …abstractions of behavioral factors that cannot be observed directly and which researchers invent to explain behavior

  31. Variable… …a construct that can take on two or more values or scores

  32. Raw scores… …the number of items an individual scored on an instrument

  33. Measurement scales… …the representation of variables so that they can be quantified

  34. Measurement scales... Qualitative(categorical) 1. nominal variables Quantitative (continuous) 2. ordinal variables 3. interval variables 4. ratio variables

  35. 1.nominal (“categorical”): classifies persons or objects into two or more categories

  36. 2.ordinal (“order”): classifies persons or objects and ranks them in terms of the degree to which those persons or objects possess a characteristic of interest

  37. 3.interval: ranks, orders, and classifies persons or objects according to equal differences with no true zero point

  38. 4.ratio: ranks, orders, classifies persons or objects according to equal differences with a true zero point

  39. Norm reference… …provides an indication about how one individual performed on an instrument compared to the other students performing on the same instrument

  40. Criterion reference… …involves a comparison against predetermined levels of performance

  41. Self reference… …involves measuring how an individual’s performance changes over time

  42. Operationalize… …the process of defining behavioral processes that can be observed

  43. Standard error of measurement… …an estimate of how often a researcher can expect errors of a given size on an instrument

  44. Mini-Quiz… • True or false… …a large standard error of measurement indicates a high degree of reliability false

  45. True or false… …a large standard error of measurement indicates low reliability true

  46. True or false… …most affective tests are projective false

  47. True or false… …the primary source of test information for educational researchers is the Burros Mental Measurements Yearbook true

  48. True or false… …research hypotheses are usually stated in terms of variables true

  49. True or false… …similar to a Thurstone scale, a Guttman scale attempts to determine whether an attitude is unidimensional true

  50. True or false… …validity requires the collection of evidence to support the desired interpretation true