Integrating Teaching and Research: Using Student Derived Data

1. Integrating Teaching and Research: Using Student Derived Data Dr. Emily Summers Dr. Jon Lasser Dr. Gail Ryser Mr. William Barry

2. What is student derived data? • Student Data from Your Class(es) • Student Data from Others’ Class(es) • Student Data from University-Associated Sources

3. Why use student derived data in your research? • To Benefit Students or Instruction • Improve Instruction • Understand Students’ Learning • Increase Responsiveness to Student or Instructional Needs • Allow Your Teaching to Inform Your Research and Your Research to Inform Your Teaching in a Hermeneutically Refining Exchange • (Not to Increase Publications; Should Have an Authentic Purpose Situated in Teaching, Learning, and/or Instructional Practices)

4. Setting up a study using student derived data • IRB • Quantitative design and analysis including statistical power • Other Power (all the time)

5. Institutional review board (IRB) 101An Introduction to the IRBTexas State University-San Marcos

6. Mission “…charged with protecting the rights and welfare of human research subjects. The IRB reviews proposed research to ensure that the proposed project follows federal guidelines and accepted ethical principles to meet that goal.”

7. Secondary Mission To help faculty and students proceed with their research plans in a timely manner that ensures compliance with human subject protection regulations.

8. Determining the Need for Review Must answer two questions: • Is it research? • Are human subjects involved?

9. Is it research? For purposes of IRB review, "research" is defined as "a systematic investigation designed to develop or contribute to generalizable knowledge." (45.CFR 46.102). Projects at the undergraduate or graduate level such as thesis, honors, or seminar projects may also be considered "research."

10. Are human subjects involved? For purposes of IRB review, a "human subject" is defined as "a living individual about whom an investigator conducting research obtains (1) data through interaction or intervention with the individual or (2) identifiable private information." (45.CFR 46.102)

11. Types of Review • Exempt • Expedited • Full • Continuation

12. Is it research? No Yes What kind of review? No need for any review Exemption review (very brief) Expedited Full seated review

13. Balancing Risk and Benefit All human subject research presents some risk Benefits must outweigh risks Consider benefit to subjects vs. society Researchers tend to overestimate benefits and underestimate risks

14. Exempt: Six Federal Criteria See IRB website for the six exemption categories. Examples: Studies of normal educational practices Studies of existing data sets without identifying information And low risk

15. How to apply at Texas State Complete CITI training: https://www.citiprogram.org/default.asp?language=english Collaborative Institutional Training Initiative

16. How to apply at Texas State TSU IRB home page: http://www.txstate.edu/research/irb/index.php TSU IRB application page: http://www.osp.txstate.edu/irb/ Registration required, using TSU e-mail address

17. The Review Process • Most reviews at Texas State are expedited. • How do reviewers evaluate applications? • Common errors • No synopsis submitted • No consent form submitted • Consent form does not meet requirements • Applicant does not address risk • Student fails to gain faculty approval

18. Becoming a Reviewer • Interested in becoming a reviewer? Contact Jon Lasser (lasser@txstate.edu) or 245-3413

19. Quantitative Design • Posttest only • Pre and posttest • Pre and posttest with control group • Add additional control variables

20. Quantitative Design • Covariates variables: occur in a model in which the independent variables of interest are categorical, but you also need to control for an observed, continuous variable–the covariate. • Moderator variables: relationship between two variables depends on the third variable (also known as an interaction effect) • Mediator variables: intervening and helps explain why two or more other variables are related, may be causally related to the dependent variable

21. Quantitative Design • Statistical Power • Four interrelated components that influence the conclusions you might reach when you run a statistical test: • Alpha level (risk of Type I error) • Power(opposite of Type II error or the probability of detecting a hypothesized effect if one actually exists) • Sample size (n) • Effect size (practical significance or the degree to which the phenomenon exists) • Power Calculators • http://homepage.cs.uiowa.edu/~rlenth/Power/ • http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/download-and-register

22. Quantitative Design • Collect data over time • Cross sectional versus longitudinal

23. Other non-Statistical Power • Inherent Instructor/Student Power Relationships • Ways to Mediate this Balance • In-Class Forms • After-Class Forms • Open Conversations • Make the Default Participation is Non-Participation

24. Tips… • Utilize a Neutral Third Party to Distribute/Collect Consent Forms & Data • Employ Temporal Delays (in Receiving, Collecting, and/or Analyzing Data) • Notify Students Early in Semester about Intent to Ask Permission to Use Class Data as Part of a Research Study, Especially for Journals or Other Subjective Reflection Work • Waive Written Consent for No Record of Who Participated • Try Anonymous Online Surveys • Use Known Identities to Secure Consent After Grades Post

25. Cautions… • Be Aware of Additional Peer Coercion to Participate • Practice Keen Awareness of Unequal Risks, as Some Student Subgroups May be More Identifiable than Others; • Watch Extra Credit Options • If Offering Extra Credit for Participation Provide Equivalent Alternatives for Extra Credit • Give Credit for Partial Participation to Ensure Students’ Right to Withdrawal • Recognize that Some Professor/Students Roles Inherently Have Increased Risk of Coercion • Thesis Students • Research Advisees • On the Cusp Grades • Mentees

26. Resources • Online and Hybrid data • Testing, Research-Support, and Evaluation Center (TREC) • Snap Surveys • NVivo 10 • Initiative for Interdisciplinary Research Design and Analysis (IIRDA) • Institutional Research

27. Online & Hybrid Data • BalckBoard Archives • Tracs Data • Discussion Boards • Be Specific (Not a Catch All)

28. TREC: Snap Surveys • PC based software that can be used to create and manage web-based surveys • Allows the developer to incorporate the security, access, and permissions needed to meet the IRB guidelines • Can only access Snap Surveys if connected to the Texas State domain; user must have administrative rights to the PC • User must take free recorded training • http://www.txstate.edu/trec/iirda/researchSupport/snap.html

29. TREC: NVivo 10 • PC based qualitative software • Removes many of the manual tasks associated with qualitative analysis (e.g., classifying, sorting, arranging of information) • Designed to support a wide range of research methods including grounded theory, ethnography, discourse analysis • Allows you to code your data, search your data using queries, and examine your data visually using models and charts • http://www.txstate.edu/trec/iirda/researchSupport/NVivo.html

30. TREC: IIRDA • IIRDA's mission is to serve as a resource that meets the need for state-of-the-art research support in a comprehensive, rigorous and interdisciplinary way. • Services • Externally funded proposal assistance • Faculty consultation services • Professional data analysis services • Contains a resources page with annotated references, calculators, online trainings, and quantitative methods websites. • http://www.txstate.edu/trec/iirda/resource.html

31. Institutional Research (IR) • Will provide additional student data for IRB approved research projects • http://www.ir.txstate.edu/

32. Sample Studies • Microsoft Word study • Core course study

33. Audience Questions • “I was curious to hear a little about the extent to which we could collaborate with students in our classes that are teachers, or conducting field work, to implement a research study and use the data they would be using for class projects or papers, for research as well.” • Others?