EXPLORING THE EFFECTIVENESS OF RCR EDUCATION IN THE SOCIAL AND BEHAVIORAL SCIENCES

1. EXPLORING THE EFFECTIVENESS OF RCR EDUCATION IN THE SOCIAL AND BEHAVIORAL SCIENCES Jim Vander Putten Department of Educational Leadership Amanda L. Nolen Department of Teacher Education University of Arkansas-Little Rock Paper presented at the 2009 Research Conference on Research Integrity, May 15 - 17, 2009 Niagara Falls, NY.

2. Purpose and Background • Critical issue analysis of problems and issues in conducting research on responsible conduct of research education, instruction and training (RCR EIT) in the Social and Behavioral Sciences • From descriptive institutional data collected over two years, we recommend strategies for future research in this area • Importance of studying less prestigious segments of professions • can yield perspectives very different from elite cultures that receive the majority of media attention • University of Arkansas-Little Rock (UALR) is a metropolitan commuter institution in the Doctoral/Research-Intensive Carnegie institutional classification

3. Context • August 2003: UALR implemented the CITI online social and behavioral sciences RCR EIT program • Between 2004 and 2006, approximately 875 UALR faculty, staff, and students participated in the CITI training program: • 88.5% (n=770) completion rate: • Students (n=551) • Principal Investigators (n=197), or • IRB Members (n=22) • Non-completers (11.5%, n=97) identified themselves in one of several research roles:, Student (74), Principal Investigator (18), or IRB Member (5).

4. Institutional Data • Between 2004 and 2006, approximately 625 IRB proposals were submitted for review • Roughly 25% were unsuccessful (defined as being disapproved on first submission) for inappropriate research practices. Most common reasons for IRB protocol rejection: • incomplete IRB protocols, • inappropriate consent processes, • missing elements in informed consent documents, • insufficient data security measures • coercion in the data collection process. • Inspection of CITI training completion dates on unsuccessful IRB proposals indicated that the vast majority had been completed within 2 weeks prior to IRB proposal submission and review.

5. Analysis of Issues • Evaluation surveys of CITI basic and refresher courses “have been very positive” • In the Social and Behavioral Sciences, the least useful testing is indirect measurements polling students and asking them how they feel about their education. • At UALR, the short timelines between CITI training completion and IRB proposal rejections don’t yield any insights into what researchers learn. • Braunschweiger (2007) noted “the very fact that only 2% of survey responders reported that they preferred traditional classroom instruction to the Web-based approach that CITI provides clearly indicates that old paradigms must be reviewed and new ways to effectively deliver ethics education tested.” • Question: “Although students are the best judges of what they want, are they the best judges of what they need?”

6. Critical Issues in ‘Learning’ LEARNING – A relatively permanent change in the capacity of an organism to make a response (adaptation) that cannot be explained by maturation or the passage of time. ISSUES: • CITI Training Validity • Problem Solving – Algorithm vs Heuristic • Knowledge Transfer

7. CITI Training Validity • Format • Presents content, case studies, scenarios • Researcher answers a series of multiple choice and true/false items based on content • Multiple choice and true false items assess the researcher’s understanding at the knowledge level.

8. TAXONOMY OF LEARNING • KNOW – Recall or Recognition • UNDERSTAND – Internalize • APPLY – Apply to new similar problem • ANALYZE– Reduce problem into parts • EVALUATE– Identify criteria and judge quality • CREATE– Novel problem Anderson, et al. 2001; Bloom, et al. 1958

9. TAXONOMY OF LEARNING • KNOW – Recall or Recognition • UNDERSTAND – Internalize • APPLY – Apply to new similar problem • ANALYZE– Reduce problem into parts • EVALUATE– Identify criteria and judge quality • CREATE– Novel problem CITI Training Assessment RCR Behavior

10. Problem Solving • Algorithm – Well defined problems • If X, then apply Y, the result will be Z • A recipe, a math problem, a science experiment • Heuristic – Poorly defined problems • Social sciences research involving human subjects with free will! Ormrod, 2008

11. Knowledge Transfer • Applying knowledge, skills, and abilities from the classroom/hypothetical setting to real situations with real consequences • Multiple models/examples • Relevant models • New knowledge built on prior knowledge Bandura, 1986

12. Conclusions and Recommendations • Create a more appropriate assessment of researchers’ knowledge, skill, and ability before completing RCR education modules; • Teach researchers appropriate heuristic problem solving techniques; • Provide examples and scenarios from across the disciplines that more closely resemble the research being conducted at that university; • Provide a breadth of examples of ethical issues involving human participants (Breadth & Depth)