Cultural Competency in an RCR Training Program: Focus on Misconduct

1. Cultural Competency in an RCR Training Program: Focus on Misconduct Ellen R. Fisher, Department of Chemistry Kathy Partin, Office of the VP for Research & Department of Biomedical Sciences Paul Stretesky, Department of Sociology

2. How this project got started… • Faculty member in Chemistry (my day job) • Providing RCR training to undergrads, grad students • Developed tools specifically for different groups.1 • Involved in institutional project to expand RCR • What is our constituency? • Land Grant Institution/Research I • Over $300 M in annual research expenditures • ~1450 faculty (~950 tenure track) • Student Population • Undergraduate (~21,000) and graduate (~5000) • Vet school (~500) 1E. R. Fisher and N. E. Levinger, J. Chem. Ed. 2008, 85, 796.

3. Is what’s good for Chemistry good for everyone? • There is cultural bias in misconduct • Non-U.S. Citizens make up ~6% of all scientists in U.S. • Non-U.S. Citizens make up ~31% of all misconduct cases • Literature: differences in plagiarism attitudes1 • We want to be sensitive to cultural differences • Ethnicity, race, gender, disability, citizenship • Discipline • CSU has diverse research programs • Social “soft” sciences • “Hard” sciences and Engineering • Agricultural sciences, natural resources, cross-disciplinary • Infectious disease 1Marshall & Garry Int. J. Ed. Integrity2, 26 (2006).

4. Can we create an institutional-wide RCR program that is “culturally” sensitive?

5. What is culturally-sensitive about RCR? • Anecdotal: my experience • Differences between populations (e.g. undergrads vs. grads) • Level of intellectual maturity • Level of experience • Differences in sub-disciplines of chemistry • Organic chemistry vs. physical chemistry • Very clear sub-cultures with respect to research practices • Differences between disciplines • Are the differences real and if so, where do they exist? • Tailor RCR programs for highest effectiveness

6. Institutional Project: Research Integrity Office • IRB protocol: pre- and post-assessments of current RCR training efforts • Tuned to areas of difference • Chemistry course and introductory seminars-initial populations • Some questions • Are there core areas we can identify that are discipline-independent? • Are there areas that are discipline-specific? • Are there more or less effective methods of teaching RCR and do they differ between populations? • What are effective methods for assessment of RCR training?

7. Pilot Program: Preliminary Data • Pre-survey • Developed from F. Macrina, Research Integrity • Mostly “yes” or “no” questions • Likert-scale responses • Behavioral – “Not justifiable” – “Always justifiable” • Statements: strongly disagree – strongly agree • Where does training occur? • Two different disciplines (populations) • Chemistry (CHEM) graduate students (n=27/28) • Health and Exercise Science (HES) graduate students (n = 14)

8. Population 1: The “Culture” of Chemistry • Fairly traditional, laboratory science • Hierarchical structure • PI (Faculty member) • Postdocs, graduate students, undergraduates • Mentor-protégé relationships • Sub-disciplines • At least 4 different major subdivisions • Each has their own culture, norms and practices • More subtle subdivisions • Cross-disciplinary studies • Many students/research projects now crossing boundaries

9. Demographics of CSU Chemistry Department1 • Student Community • Total enrollment : ~150 students • >90% Ph.D. • Most (~90%) are “traditional” • Gender • ~2:1 ratio of men to women (grad students) • 7/27 (26%) female regular faculty (5 tenured) • Nation of Origin/Ethnicity • ~16% non U.S. citizen • ~10% underrepresented minority 1Based on institutional data, http://www.ir.colostate.edu/

10. Population 2: The “Culture” of HES • Mixed Research Mission • Physiology, neurobiology, integrative biology, biochemistry • “Hard” science • Animal models and human subjects • Health attitudes and behaviors, social influences • “Softer”-sciences • Psychology and mental health • Human subjects • Shared laboratories/resources • Research teams • Long-term studies

11. Demographics of CSU HES Department1 • Student Community • Total enrollment: ~35 students • ~25% Ph.D. (new program) • Most are “traditional” • Gender • 60% female (grad students) • 3/18 (17%) female regular faculty (2 tenured) • Ethnicity/Race • 100% non-minority, non-Hispanic 1Based on institutional data, http://www.ir.colostate.edu/

12. Demographics – Survey Respondents • Major population differences • Year in program: MS vs. Ph.D. • Gender • Very little ethnic/racial diversity

13. Survey Results: Research Misconduct • Fabrication, falsification and plagiarism (FFP) • Data manipulation • Data analysis • Reporting practices • Record keeping • Survey questions • Yes/No questions • Situational or behavioral questions • Agree/disagree • Justifiable/not justifiable • Where RCR topics are discussed

14. Falsifying Data - Behavioral Altering experimental data to make an experiment look better than it actually was.

15. Fabricated data – Behavioral • Reporting experimental data that have been created without actually having conducted the experiment.

16. Falsification & Data Handling – Yes/No • Is it ever OK to disregard or modify data points that are part of a data set in order to prove a hypothesis in a research project (e.g. if one point represents an outlier)? • Is it ever OK to disregard or modify data points that are part of a data set in a laboratory class? • Do you know how many times any particular experiment must be repeated in order to feel satisfied that the results are trustworthy?

17. Where Are Students Exposed to RCR Topics? Which of the following topics have you discussed in a class, with peers, in a research group meeting or with your research advisor or other faculty member? For each item in the table, check all the boxes that apply.

18. Where do students learn about data handling? • Importance of honestly reporting what you find • Methods for proper record keeping • Some differences exist between populations • Overall: >80% report discussions in at least 1 venue

19. Data Analysis: Differences in Populations Trying a variety of different methods of analysis until one is found that yields a result that is statistically significant. p = 0.007

20. Knowledge of Plagiarism Chemistry REU Students • Do you have first-hand knowledge of scientists plagiarizing the work of someone else? • Have you ever plagiarized the work of someone else?

21. Reporting Misconduct • Would you report a coworker who you believe has violated scientific integrity standards? • Would you report a supervisor/advisor who you believe has violated scientific integrity standards?

22. Reporting Misconduct - Peers • If you witnessed a coworker committing research misconduct, you would know who would be the appropriate person at CSU to report it to. p = 0.24

23. Reporting Misconduct - Supervisors • If you witnessed a supervisor or principal investigator committing research misconduct, you would know who would be the appropriate person at CSU to report it to. p = 0.08

24. Witnessing Misconduct • Responsibilities and strategies for action after having witnessed research misconduct

25. Authorship and Citation Practices 90% 41% 39% • Have you ever been an author of a published paper? • Do you believe you have a good understanding of when and how to credit another person’s words and/or ideas when you use them in your own paper? • Do rules governing references and citations differ depending on the type of paper you might be writing?

26. Citations – Do they really understand? Republishing data without citation of the earlier work. p = 0.03

27. Where Are Students Exposed to RCR Topics? Which of the following topics have you discussed in a class, with peers, in a research group meeting or with your research advisor or other faculty member? For each item in the table, check all the boxes that apply.

28. Authors and Authorship Criteria • Nearly 20% of student authors have never had a conversation about authorship criteria • Less than 2/3 of student authors have discussed authorship criteria with a faculty member

29. Where Are Students Exposed to RCR Topics? Which of the following topics have you discussed in a class, with peers, in a research group meeting or with your research advisor or other faculty member? For each item in the table, check all the boxes that apply.

30. One Major Area of Difference • More than 2/3 Chemistry students report never having had a discussion about animal/human subjects • More than 85% of HES students have discussed human subjects in at least 1 venue (predominantly in classes)

31. Preliminary Conclusions • Some clear differences between the two populations • Have we answered any of our questions? • Are there core areas we can identify that are discipline-independent? • Data handling, reporting and record keeping • Others? • Are there areas that are discipline-specific? • Animal/human subjects • Analysis methods? • Are there more or less effective methods of teaching RCR and do they differ between populations? • What are effective methods for assessment of RCR training?

32. Next Steps • Pre-survey: • More analysis of data • Revise and modify questions • Additional data from other disciplines • Computer science • Psychology (matches with HES) • Engineering programs • Veterinarian Medicine • Develop post-RCR training survey • Administer at least 6 months after training (annually?) • Focus on awareness

33. Developing a Culturally-Competent Institutional RCR Program The Action Plan

34. CSU’s RCR Efforts Represent a Mosaic

35. Goal 1Construct a Uniform and Elevated “Floor” of RCR Training Implementation of Section 7008 and 7009 of the America COMPETES Act is going to “stimulate” the laying of this floor …

36. Goal 2: Facilitate the Building of Discipline-Specific RCR Training Exercise Sciences Chemistry

37. Campus-Wide Initiative • Small working group on implementation of RCR training program (March) • Campus-wide workshop (June) • Undergraduates and graduate students, postdocs • Faculty and staff • Administration • RCR instructors • Compliance Committees (IRB, IACUC, IBC) • The Teaching & Learning Institute • Business & Finance folks • Ethicists • Nay-sayers

38. Goals of CSU Workshop • Training (Dr. Ken Pimple’s keynote address) • Talking (about how RCR training enhances trainees’ experiences and their future success) • Listening (to concerns and solutions) • Identifying “local values” • What is our repertoire of “core” competencies? • Who do we want to teach it to? • How do we want to teach it? • How are we going to verify teaching to meet mandates? • How are we going to assess its efficacy? • Building consensus on a path forward

39. Guiding Principles • The goal of RCR training is not to prevent MiS and not to make better citizens (although both are admirable goals). The goal is to make better science (and better scientists). • An RCR training program, no matter how effective, fails if it also impedes scientific productivity. • Cultural competency is much more complicated and important than we first imagined: the critical details of planning an RCR training program are predicated on a careful analysis of what our local “culture” is.

40. Acknowledgments • People • Prof. Lisa Dysleski (Chemistry) • Prof. Dawn Rickey (Chemistry) • Molly Gutilla (RICRO) • Financial Support • Office of the Vice President for Research • College of Natural Sciences • NSF-REU/Ethics supplement (NSF-0649623)

42. Data Reporting: pubs vs. grants • It is more important that data reporting be completely truthful in a publication than in a grant application.

43. Fabricated data: Authors share blame • If fabricated data are discovered in a published paper, all coauthors must equally share in the blame.

44. Fabricated data: Authors share punishment • If fabricated data are discovered in a published paper, all coauthors must receive the same punishment.