educational research
Download
Skip this Video
Download Presentation
Educational Research

Loading in 2 Seconds...

play fullscreen
1 / 33

Educational Research - PowerPoint PPT Presentation


  • 130 Views
  • Uploaded on

Educational Research. Chapter 22 Evaluating a Research Report Gay, Mills, and Airasian 10 th Edition. Topics Discussed in this Chapter. Gathering information General evaluation criteria Design specific evaluation criteria Qualitative research in general Observational research

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Educational Research' - donna-albert


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
educational research

Educational Research

Chapter 22

Evaluating a Research Report

Gay, Mills, and Airasian

10th Edition

topics discussed in this chapter
Topics Discussed in this Chapter
  • Gathering information
  • General evaluation criteria
  • Design specific evaluation criteria
    • Qualitative research in general
    • Observational research
    • Historical research
    • Survey – Questionnaire and Interview
    • Correlational – Relationship and Prediction
    • Causal-Comparative
    • Experimental
gathering information
Gathering Information
  • Necessity of knowing what was done
    • Examples
      • What was the problem?
      • Who were the subjects?
      • What research design was used?
      • What were the results and conclusions?
      • What are the implications of the research?
    • Basic formats to collect information for quantitative and qualitative research
gathering information quantitative
Gathering Information - Quantitative
  • Introduction
    • Problem
      • Provide a general statement of the problem that includes the variables and the relationships between them
      • State the importance of the study
    • Review of the Literature
      • List the major issues identified in the review
    • Hypothesis
      • State the specific hypothesis or hypotheses being investigated
gathering information quantitative1
Gathering Information - Quantitative
  • Method
    • Participants
      • Identify the population and sample
      • Describe the sampling and/or assignment procedures
      • Identify the size of the total sample and of each group if applicable
      • Describe the general characteristics of the subjects
gathering information quantitative2
Gathering Information - Quantitative
  • Method (continued)
    • Instruments
      • List the specific instruments used in the study
      • Describe the evidence of validity provided for each instrument
      • Describe the reliability evidence cited for each instrument
      • Describe the information needed to interpret the scores for each instrument
gathering information quantitative3
Gathering Information - Quantitative
  • Method (continued)
    • Design and Procedures
      • Identify the specific type of research design
      • Identify any threats to internal validity
      • Identify any threats to external validity
    • Results
      • Identify the specific analyses being used
        • A comparison between the mean scores for a control and experimental group
        • A correlation between students’ math attitudes and achievement
        • A survey of parental attitudes toward an extended school year
gathering information quantitative4
Gathering Information - Quantitative
  • Method (continued)
    • Results (continued)
      • Identify any descriptive statistics used and summarize the results
      • Identify the specific statistical test of significance, report the test statistic itself, and report its level of significance
        • The experimental group means were significantly higher (t = 5.68, p = .023) than those for the control group
        • There was a significant (r2 = 0.91, p = .001) positive relationship between students’ attitudes and achievement
gathering information quantitative5
Gathering Information - Quantitative
  • Discussion
    • Identify the specific conclusions of the researchers
    • Discuss the implications described by the researchers
gathering information qualitative
Gathering Information – Qualitative
  • Introduction
    • Research topic
      • Provide a statement of the general issue, topic, or question being investigated
      • Describe any reformulation of the topic on the basis of the ongoing interactive nature of the collection, analysis, and synthesis of data
      • Discuss the importance of the topic
gathering information qualitative1
Gathering Information – Qualitative
  • Introduction (continued)
    • Review of the literature
      • Describe the nature of the review of the literature
      • List the major issues identified in the review of the literature
gathering information qualitative2
Gathering Information – Qualitative
  • Method
    • Site and participant selection
      • Describe the strategies used to gain entry to the site
      • Describe the site
      • Identify the participant(s) and list the sampling strategies used to select them
      • Describe the characteristics of the participant(s)
gathering information qualitative3
Gathering Information – Qualitative
  • Method (continued)
    • Data collection and analysis
      • Describe the researcher’s role in the study
      • Report the data collection strategies used
      • Identify any instruments or protocols used by the researchers
      • Identify any threats to the quality of the data (i.e., observer bias and observer effect)
      • Describe the strategies used to enhance validity and reduce bias in data collection
      • Describe the strategies used to classify and interpret data
gathering information qualitative4
Gathering Information – Qualitative
  • Method (continued)
    • Research approach and procedures
      • Identify the research approach
      • Briefly describe the procedures used
      • Identify any ethical issues related to the study
  • Results
    • Report the findings
    • Describe the researcher’s interpretation of the findings
gathering information qualitative5
Gathering Information – Qualitative
  • Discussion
    • Report the researcher’s conclusions
    • State the relationship between the conclusions and the initial problem
focus of general evaluation criteria
Focus of General Evaluation Criteria
  • See the evaluation criteria in the text and on the web site
  • Introduction
    • Problem
    • Review of the related literature
    • Hypotheses
  • Methods
    • Participants
    • Instruments
    • Research design and procedures
focus of general evaluation criteria1
Focus of General Evaluation Criteria
  • Results
  • Discussion
  • Abstract or summary
type specific evaluation criteria
Type-Specific Evaluation Criteria
  • Descriptive research
    • Questionnaire studies
      • Are pilot study procedures and results described?
      • Are directions to questionnaire respondents clear?
      • Does each item in the questionnaire relate to one of the objectives of the study?
      • Does each questionnaire item deal with a single concept?
      • When necessary, is a point of reference given for questionnaire scales?
type specific evaluation criteria1
Type-Specific Evaluation Criteria
  • Descriptive research (continued)
    • Questionnaire studies (continued)
      • Are leading questions avoided in the questionnaire?
      • Are there sufficient alternatives for each questionnaire item?
      • Does the cover letter explain the purpose and importance of the study and give the potential respondent a good reason to co-operate?
      • If appropriate, is confidentiality or anonymity assured in the cover letter?
type specific evaluation criteria2
Type-Specific Evaluation Criteria
  • Descriptive research (continued)
    • Questionnaire studies (continued)
      • What is the percentage of returns and how does this affect the study results?
      • Are follow-up activities to increase returns described?
      • If the response rate was low, was any attempt made to determine any major differences between respondents and non-respondents?
      • Are data analyzed in groups or clusters rather than a series of many single variable analyses?
type specific evaluation criteria3
Type-Specific Evaluation Criteria
  • Correlational research
    • Relationships
      • Were variables carefully selected?
      • Is the rationale for variable selection described?
      • Do the conclusions avoid suggesting causal relationships between variables?
type specific evaluation criteria4
Type-Specific Evaluation Criteria
  • Causal-comparative research
    • Are the characteristics or experiences that differentiate the groups clearly defined or described?
    • Are critical extraneous variables identified?
    • Were any control procedures applied to equate the groups on extraneous variables?
    • Are plausible alternative hypotheses discussed?
type specific evaluation criteria5
Type-Specific Evaluation Criteria
  • Experimental research
    • Was an appropriate experimental design selected?
    • Is a rationale given for the design selected?
    • Is the method of group formation described?
    • Was the experimental group formed in the same way as the control group?
    • Were groups randomly formed and the use of existing groups avoided?
type specific evaluation criteria6
Type-Specific Evaluation Criteria
  • Experimental research (continued)
    • Were treatments randomly assigned to groups?
    • Were critical extraneous variables identified?
    • Were any control procedures applied to equate groups on extraneous variables?
    • Were the results generalized to the appropriate group?
type specific evaluation criteria7
Type-Specific Evaluation Criteria
  • Interview studies
    • Were the interview procedures pretested?
    • Are pilot study procedures and results described?
    • Does each item in the interview guide relate to a specific objective of the study?
    • When necessary, is a point of reference given in the guide for interview items?
    • Are leading questions avoided in the interview guide?
    • Is the language and complexity of the questions appropriate for the participants?
type specific evaluation criteria8
Type-Specific Evaluation Criteria
  • Interview studies (continued)
    • Does the interview guide indicate the type and amount of prompting and probing that was permitted?
    • Are the qualifications and special training of the interviewers described?
    • Is the method used to record responses described?
    • Did the researcher use the most reliable, unbiased method of recording responses?
type specific evaluation criteria9
Type-Specific Evaluation Criteria
  • Mixed methods research
    • Does the study use at least one quantitative and one qualitative research method?
    • Does the study include a rationale for using a mixed methods research design?
    • Does the study include a classification of the type of mixed methods research design?
type specific evaluation criteria10
Type-Specific Evaluation Criteria
  • Mixed methods research (continued)
    • Was the study feasible given the amount of data to be collected and concomitant issues of resources, time, and expertise?
    • Does the study include both quantitative and qualitative research questions?
    • Does the study clearly identify qualitative and quantitative data collection techniques?
    • Does the study use appropriate data analysis techniques for the type of mixed methods design?
validity and reliability
Validity and Reliability
  • Threats to internal validity in qualitative studies
    • Did the researcher effectively deal with problems of history and maturation by documenting historical changes over time?
    • Did the researcher effectively deal with problems of mortality by using a large enough sample?
    • Was the researcher in the field long enough to effectively minimize observer effects?
    • Did the researcher take the time to become familiar and comfortable with participants?
validity and reliability1
Validity and Reliability
  • Threats to internal validity in qualitative studies (continued)
    • Were the interview questions pretested?
    • Did the researcher interview key informants to verify field observations?
    • Were participants demographically screened to ensure that they were representative of the larger population?
validity and reliability2
Validity and Reliability
  • Threats to internal validity in qualitative studies (continued)
    • Was the data collected using different media to facilitate cross-validation?
    • Were participants allowed to evaluate the researcher results before publication?
    • Is sufficient data presented to support findings and conclusions?
validity and reliability3
Validity and Reliability
  • Threats to reliability in qualitative studies (continued)
    • Is the researcher’s relationship with the group and setting fully described?
    • Is all field documentation comprehensive, fully cross-referenced and annotated, and rigorously detailed?
validity and reliability4
Validity and Reliability
  • Threats to reliability in qualitative studies
    • Is construction, planning, and testing of all instruments documented?
    • Are key informants fully described, including information on groups they represent and their community status?
    • Are sampling techniques fully documented as being sufficient for the study?
ad