Educational Research: Descriptive Research

1. Educational Research: Descriptive Research EDU 8603 Educational Research Richard M. Jacobs, OSA, Ph.D.

2. Research... • The systematic application of a family of methods employed to provide trustworthy information about problems …an ongoing process based on many accumulated understandings and explanations that, when taken together, lead to generalizations about problems and the development of theories

3. Quantitative research methods... …collect and analyze numerical data obtained from formal instruments

4. Quantitative methods... • descriptive research (“survey research”) • correlational research • causal-comparative research (“ex post facto research”) • experimental research

5. descriptive research (“survey research”) …collects data in order to answer questions about the current status of the subject or topic of study …uses formal instruments to study preferences, attitudes, practices, concerns, or interests of a sample

6. The basic steps of descriptive research... An orderly scientific and disciplined process, involving: recognizing and identifying a topic to be studied selecting an appropriate sample of participants collecting valid and reliable data reporting conclusions

7. Classifications of descriptive research studies... cross-sectional survey… …involves collecting data from selected individuals in a single time period however long it takes to collect data from participants

8. longitudinal survey… …involves collecting data at two or more instances in order to measure changes over time

9. self-report survey… …requires individuals to respond to a series of statements or questions about themselves

10. observation study… …the researcher obtains data by watching participants in a situation

11. Types of longitudinal surveys... • trend survey • cohort survey • panel survey • follow-up survey

12. trend survey …a study where a sample is taken from the general population in order to collect data over time …involves different groups and different samples over time

13. cohort survey …a study where a specific population is examined by sampling different groups within the population over time …involves the same group but different samples from that group over time

14. panel survey …a study where the same group and the same sample are examined over time

15. follow-up survey …a study undertaken after a panel survey and seeks to examine subsequent development or change

16. Conducting a questionnaire study… 1. state the problem 2. select participants 3. construct the questionnaire 4. prepare cover letter 5. pretest questionnaire 6. follow up activities 7. analyze/report results

17. 1. state the problem… …topic must be of sufficient significance to motivate potential respondents and justify the research effort in the first place …define topic in terms of specific objectives indicating the kind of information needed

18. 2. select participants… …use an appropriate sampling technique …participants must be able to provide the desired information sought and willing to provide it to the researcher

19. 3. construct the instrument… …should be attractive, brief, and easy to fill out …there are a variety of data collection methods that can be used, including: questionnaires, interviews, observations, and telephone calls

20. types of self-report items …scaled …ranked …checklist …free response

21. Designing an instrument… …include a brief statement describing the study and its purpose at the top of the instrument …provide standardized directions …organize items leaving sufficient white space on instrument

22. …place similar items together …ask general, non-threatening items first, moving into more specific, self-revealing items

23. some do’s and don’t’s… …construct items according to a set of predetermined guidelines …include only items relating directly to the purpose of the study …avoid jargon or any term or concept that might mean different things to different people

24. …each item should deal with a single concept and be worded as clearly as possible …items should indicate a point of reference beyond the self in order to judge the self …avoid ambiguous terms like several or usually

25. …avoid touchy matters in items which respondents might not respond to honestly or at all …avoid leading questions which suggest that one response may be more appropriate than another …each item must be able to stand on its own

26. …don’t jam items together …don’t put the most important questions at the end of the instrument

27. 4. prepare the cover letter… …a brief, neat, explanation of the significance of the study and what is being asked of the respondent and why …addressed to the specific, potential respondent

28. …an endorsement adds credibility …guarantee anonymity or confidentiality …include a specific deadline date by which the completed instrument (“questionnaire”) is to be returned

29. …individually sign each letter …include a self-addressed, stamped, return envelope

30. 5. pretest the instrument… …conduct a pilot study to gather information about deficiencies and suggestions for improving the instrument …provides greater content validity

31. 6. follow-up activities… …because first mailing results tend to be low (30% - 50% response rate), researchers need strategies to increase the response rate

32. initial follow-up strategies to increase response rate up to 20%… …send out reminder postcard …mail a second packet with a new, positively worded cover letter and another self-addressed, stamped, return envelope

33. secondary follow-up strategies to increase response rate up to 10%… …telephone nonrespondents to encourage them to respond

34. dealing with nonrespondents …study whether nonrespondents differ from respondents in some systematic way by selecting a sample from the nonrespondents …telephone sample, aggregate data, include in report

35. dealing with nonresponses …study the items to determine the problem with the item …include description of the nonresponses and the determination of the reason in the report

36. 7. analyze/report results… …discriminant item analysis includes giving the response rate for each item as well as the total sample size and the overall percentage of returns, since not all respondents will answer questions

37. …group items into clusters that address the same issue and develop total scores across an item cluster in order to avoid “information overload”

38. Conducting an interview study… 1. state the problem 2. select participants 3. construct the interview guide 4. communicate professionally and record responses accurately 5. pretest interview procedure 6. analyze/report results

39. 1. state the problem… …topic must be of sufficient significance to motivate individuals to participate and justify the research effort in the first place …define topic in terms of specific objectives indicating the kind of information needed

40. 2. select participants… …use an appropriate sampling technique …participants must be able to provide the desired information sought and willing to provide it to the researcher

41. 3. construct the interview instrument (“instrument guide”)… …indicates the question to be asked, the order, and how much additional prompting or probing is permitted …the goal is to obtain standardized, comparable data from each interviewee

42. 4. communicate professionally and record responses accurately… …effective interviewing requires training and experience to avoid interviewer bias and interviewer error

43. …record responses manually on the interview guide …(requires interviewee permission) use a tape recorder or VCR to verify accuracy of responses

44. 5. pretest interview procedure… …use a small group from the same population or a similar population to the one being studied to validate the interview instrument and the interviewer’s communication and recording skills

45. 6. analyze/report results… …item analysis includes giving the response rate for each item, as well as the total sample size and the overall percentage of returns, since not every respondent will answer all questions

46. …also include in-depth data to open-ended interview items to portray responses in more accurate and honest terms

47. Conducting an observational study… 1. state the problem 2. select participants 3. define the observational variables 4. record observations 5. assess observer reliability 6. reduce observer bias 7. analyze/report results

48. 1. state the problem… …topic must be of sufficient significance to motivate potential respondents and justify the research effort in the first place …define topic in terms of specific objectives indicating the kind of information needed

49. 2. select participants… …use an appropriate sampling technique …participants must be able to provide the desired information sought and willing to provide it to the researcher

50. 3. define the observational variables… …operationalize the variables to be observed in terms of specific behaviors that can be quantified …ensure that the observations can be quantified in a way so that all observers will count the behavioral activities in exactly the same way