Methodology

1. Methodology Tips for Constructing Instruments

2. Matching methods to research paradigm

3. Quantitative/Descriptive Studies involve specific Research Designs • Designs structure how the interviews are conducted. • Allow different groups and time periods to be compared. • Minimize threats to internal and external validity.

4. Components of Designs • Sampling (Random; Other probability sampling or nonprobability – such as convenience, purposive, or snowball) • Groups – one group; two, or multiple group comparisons. • Time intervals – pre and post tests or multiple observations.

5. Common design types • Experimental Design (pre-post; random sample, two or more groups compared. • Quasi Experimental with nonequivalent comparisons groups- pre and post. • Quasi Experiments with post-test only. • One group - pre and post test only. • One group – multiple time periods measured (time series or longitudinal studies) • One group – one measure – usually called cross-sectional survey.

6. Random designs help ensure internal/external validity • Internal validity – attribute of people included or way the research was conducted may limit our ability to establish a real connection between the independent and dependent variables. • External validity – can we generalize results to other people, settings, and times. (Note for some types of studies – qualitative research and needs assessments and some types of evaluations – we do not need generalizability. We are only interested in finding things out about a particular situation).

7. Exceptions to Experimental/Quasi-experimental Designs: One group studies • Case studies (combination of methods looking at one individual, group, family, organization, or community) • Survey research • Consumer satisfaction surveys • Needs Assessments

8. Use these alternative methods • For descriptive studies • When you do not have an intervention • When you want to look at a specific situation or group and you do not intend to generalize. • When it would be difficult or unethical to create an experimental or control group. • When you do not have time or money to conduct an experiment.

9. Threats to Internal Validity Include: • History • Maturation • Testing • Instrumentation • Regression to the mean • Selection • Interaction between Selection and the other threats to Internal Validity.

10. Qualitative Research • Usually one group • Comparisons may be made after data analysis – decision to compare made based on obvious patterns in the data. • Sampling – nonprobability – purposive, convenience, snowball • Uses observation, interviews, and content analysis.

11. Focus groups may also be used to conduct qualitative research. Focus groups: • Are used to assess people’s perceptions about programs or services. • Are used to generate recommendations for changing programs and services. • Are often used in conjunction with needs assessments or to generate ideas or explore themes that can be used in descriptive studies. • Are used to answer questions about how or why programs work. • May be used in conjunction with other research methods. • Include six to eight participants who are selected using purposive sampling methods. • Utilize a semi-structured interview guide with 6-8 questions. • Require that a facilitator ask each of the questions and solicit responses from members. • Are used to generate a common response or a consensus from group members.

12. In any study: • A mixture of methods may be used. • Researchers may seek to increase reliability and validity by using both quantitative and qualitative approaches. This is called triangulation! • Methods used should match the research question: • What happens/what is the outcome? quantitative) • Why does something happen? How does it happen? (qualitative) • A description of what exists (Descriptive)

13. Descriptive studies Generally involve surveys, standardized interviews, or the use of pre-existing (secondary data). • Include needs assessments. • Can include both qualitative and quantitative data collection. In some cases, one instrument can include a mix of qualitative and quantitative data collection methods.

14. Survey and Interview Construction Tips • Only measure one concept per question. • Don’t write the question to solicit a specific response • Response categories (if used) should be mutually exclusive and exhaustive • Questions should be tactful and should not invoke a social stigma. • May be filter questions – respondents can be directed to successive questions based on one response. • Demographic questions are generally put at the beginning or the end of the instrument. • Be aware that respondents do not like to answer some types of questions (age, income, organization budgets, etc.)

15. Example of closed-ended (response category) questions 1. What is your gender 1. Male 2. Female 2. What is your ethnic identification? 1. African American 2. American Indian 3. Asian/Pacific Islander 4. European American 5. Mexican American/Latino 6. Multicultural or multi-racial 7. Other ____________________

16. More examples 3. What is your age? 1. 18 or under 2. 19-29 3. 30-49 4. 50 – 64 5. Over 65 4. How satisfied are you with Social Work 292? 1. Very satisfied 2. Satisfied 3. Neutral 4. Unsatisfied 5. Very Unsatisfied

17. Sample question: How would you assess your own practice ability in the following areas:

18. SAMPLE QUALITATIVE INTERVIEW QUESTIONS Interview Protocol: Perspectives on Deafness

19. Each type of question corresponds to a level of measurement. These levels of measurement will determine what type of data analysis we use: • All qualitative research is nominal. • All categorical data in which there is no difference in value among the categories is nominal. • All categorical data in which there is an implied ranking (for example, high-medium-low) is ordinal. • Any question that asks respondents to rank order responses is also ordinal. • Any question in which the response is a number or can be interpreted as a number with equal values among the data points is ratio.

20. For example, ratio data can include: • What is your age? _______ • Five point likert scales (1=Very likely, 2 = likely, etc. Usually likert scales are illustrated on the survey instrument as 1 2 3 4 5. Respondents are asked to circle the best response. • Scores on standardized instruments are also interpreted as ratio data.