EXPLORING THE EFFECTIVENESS OF RCR EDUCATION IN THE SOCIAL AND BEHAVIORAL SCIENCES Jim Vander Putten Department of Educational Leadership Amanda L. Nolen Department of Teacher Education University of Arkansas-Little Rock Paper presented at the 2009 Research Conference on Research Integrity, May 15 - 17, 2009 Niagara Falls, NY.
Purpose and Background • Critical issue analysis of problems and issues in conducting research on responsible conduct of research education, instruction and training (RCR EIT) in the Social and Behavioral Sciences • From descriptive institutional data collected over two years, we recommend strategies for future research in this area • Importance of studying less prestigious segments of professions • can yield perspectives very different from elite cultures that receive the majority of media attention • University of Arkansas-Little Rock (UALR) is a metropolitan commuter institution in the Doctoral/Research-Intensive Carnegie institutional classification
Context • August 2003: UALR implemented the CITI online social and behavioral sciences RCR EIT program • Between 2004 and 2006, approximately 875 UALR faculty, staff, and students participated in the CITI training program: • 88.5% (n=770) completion rate: • Students (n=551) • Principal Investigators (n=197), or • IRB Members (n=22) • Non-completers (11.5%, n=97) identified themselves in one of several research roles:, Student (74), Principal Investigator (18), or IRB Member (5).
Institutional Data • Between 2004 and 2006, approximately 625 IRB proposals were submitted for review • Roughly 25% were unsuccessful (defined as being disapproved on first submission) for inappropriate research practices. Most common reasons for IRB protocol rejection: • incomplete IRB protocols, • inappropriate consent processes, • missing elements in informed consent documents, • insufficient data security measures • coercion in the data collection process. • Inspection of CITI training completion dates on unsuccessful IRB proposals indicated that the vast majority had been completed within 2 weeks prior to IRB proposal submission and review.
Analysis of Issues • Evaluation surveys of CITI basic and refresher courses “have been very positive” • In the Social and Behavioral Sciences, the least useful testing is indirect measurements polling students and asking them how they feel about their education. • At UALR, the short timelines between CITI training completion and IRB proposal rejections don’t yield any insights into what researchers learn. • Braunschweiger (2007) noted “the very fact that only 2% of survey responders reported that they preferred traditional classroom instruction to the Web-based approach that CITI provides clearly indicates that old paradigms must be reviewed and new ways to effectively deliver ethics education tested.” • Question: “Although students are the best judges of what they want, are they the best judges of what they need?”
Critical Issues in ‘Learning’ LEARNING – A relatively permanent change in the capacity of an organism to make a response (adaptation) that cannot be explained by maturation or the passage of time. ISSUES: • CITI Training Validity • Problem Solving – Algorithm vs Heuristic • Knowledge Transfer
CITI Training Validity • Format • Presents content, case studies, scenarios • Researcher answers a series of multiple choice and true/false items based on content • Multiple choice and true false items assess the researcher’s understanding at the knowledge level.
TAXONOMY OF LEARNING • KNOW – Recall or Recognition • UNDERSTAND – Internalize • APPLY – Apply to new similar problem • ANALYZE– Reduce problem into parts • EVALUATE– Identify criteria and judge quality • CREATE– Novel problem Anderson, et al. 2001; Bloom, et al. 1958
TAXONOMY OF LEARNING • KNOW – Recall or Recognition • UNDERSTAND – Internalize • APPLY – Apply to new similar problem • ANALYZE– Reduce problem into parts • EVALUATE– Identify criteria and judge quality • CREATE– Novel problem CITI Training Assessment RCR Behavior
Problem Solving • Algorithm – Well defined problems • If X, then apply Y, the result will be Z • A recipe, a math problem, a science experiment • Heuristic – Poorly defined problems • Social sciences research involving human subjects with free will! Ormrod, 2008
Knowledge Transfer • Applying knowledge, skills, and abilities from the classroom/hypothetical setting to real situations with real consequences • Multiple models/examples • Relevant models • New knowledge built on prior knowledge Bandura, 1986
Conclusions and Recommendations • Create a more appropriate assessment of researchers’ knowledge, skill, and ability before completing RCR education modules; • Teach researchers appropriate heuristic problem solving techniques; • Provide examples and scenarios from across the disciplines that more closely resemble the research being conducted at that university; • Provide a breadth of examples of ethical issues involving human participants (Breadth & Depth)