Abstract

1. The Department of Psychology has initiated an Academic Achievement Learning Group (AALG) to investigate how and to what extent students learn psychology subject content. AALG is exploring ways to improve students’ acquisition of psychological concepts and critical thinking. To support its inquiries, the AALG is creating learning outcome assessments that are being administered at the beginning, middle and end of semesters. The information obtained will be used to develop interventions that may change and improve the Department of Psychology’s teaching and learning outcomes. The research is designed to help us create innovative teaching and learning techniques and methodologies that use digital technologies (including computer-based learning and assessments). These techniques will help students understand research methods, obtain subject content knowledge of psychology, develop and use communication and critical thinking skills, and transfer applicable skills to other areas of learning and life (e.g., careers, family, social relationships). Abstract Objectives 1. Critical Thinking 2. Communication Quality RESULTS The AALG developed and distributed a measure of learning outcomes to psychology students (N=954) in various core courses (100, 200, 221, 300). Course Descriptions: PSYC100 Introduction to Psychology- A basic introductory course, intended to bring the student into contact with the major problems confronting psychology and the more important attempts at their solution PSYC200 Statistical Methods in Psychology-A basic introduction to quantitative methods used in psychological research PSYC221 Social Psychology- The influence of social factors on the individual and on interpersonal behavior. Includes topics such as conformity, attitude change, person perception, interpersonal attraction, and group behavior PSYC300 Research Methods in Psychology Laboratory- A general introduction and overview to the fundamental theoretical, conceptual, and practical issues in psychological research in both the laboratory and the field Measure administrations: beginning of semester (pre-test), middle of semester, end of semester Students’ demographic information was collected simultaneously with the pre-test. The mid-semester measure also included a questionnaire that aimed to gather data regarding predictor variables of student performance. The measure was based on the 4 stated learning objectives (critical thinking, communication, content, and research) for undergraduates of University of Maryland. University of Maryland’s stated goals for undergraduates: 1. Critical Thinking Skills in PsychologyStudents should be able to use critical and creative thinking, skeptical inquiry, and, when possible, the scientific approach to solve problems related to behavior and mental processes. Sample question from measure: Recent data has shown that in states that have banned texting while driving, the rate of road accidents has not significantly decreased.  Some people have therefore concluded that texting while driving is not dangerous. Which of the following, if true, would undermine this contention? The bans were not effective and drivers continued to text anyway C) Very few drivers are texting while driving B) The drivers who continued to text are also better drivers D) The states that banned texting while driving are all on the East coast METHODS/MATERIALS METHODS/MATERIALS Cont’d. 2. Communication SkillsStudents should be able to communicate effectively in a variety of formats. Sample writing prompt from measure: In one paragraph, explain whether you agree or disagree with the following statement and what psychological principles help you come to this conclusion:People are generally evil – they are likely to harm others if they can get way with it. 3. Content in PsychologyStudents should demonstrate familiarity with the questions that gave rise to content knowledge, a sampling of the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology." Sample question from measure: Which of the following is a group of brain structures that controls emotions and memory? Limbic System C) Parietal Lobe B) Occipital Lobe D) Temporal Lobe 3. Research Methods in PsychologyStudents should understand and apply basic research methods in psychology including research design, data analysis, and interpretation. Sample question from measure: Which of the following sequences best reflects the order of events in a typical experimental session? informed consent -> experiment -> debriefing B) informed consent -> debriefing -> experiment C) experiment -> debriefing -> informed consent D) None of the above are correct 3. Argument Strength (Figures 1, 2, &3: Critical thinking and communication skills were assessed as described in the Methods section. Communication was evaluated based on a writing prompt that was graded by multiple research assistants in order to ensure inter-rater reliability. The prompt was assessed based on communication quality (grammar, format, etc.) and argument quality. Our results show that there was a significant increase in critical thinking, and in students’ ability to formulate a strong argument across courses. However, there was no significant change in the quality of students’ communication skills (grammar, format, etc.). CONCLUSIONS/FUTURE DIRECTIONS The data collected thus far shows that in the current sample (N=954), across 4 different core psychology courses (100, 200, 221, 300), on all 4 learning objectives, students progressed in the areas of critical thinking as well as in communication. As this is the beginning of the development of a departmental methodology, we will continue this research across the 4-year period that makes up an undergraduate career. This research can be applied to and used within departments and across colleges. On the basis of our identification of predictor variables of learning, we plan to use the data to Assess the correlation between predictor variables and students’ grades Develop interventions that target and address factors that may contribute to underachievement Develop and carry out experimental studies in order to determine effective interventions This is an ongoing project—updates will be provided on a recurring basis and at the next Innovations in Teaching and Learning Conference in 2013. Recommendations resulting from this AALG project will be used to generate departmental wide discussions resulting in decisions applicable to enhanced instruction and learning. Objective 1: To develop a broad and complex departmental methodology to measure learning outcomes across all four years, and across multiple core courses. Why is this innovative? Until now, we have measured learning outcomes exclusively on a course-by-course basis. This is the first measurement that: assesses learning outcomes across an entire department, using a range of core courses over a 4-year time span. provides educators with the capability to easily identify factors that contribute to underachievement enables educators to track individual students and therefore target students who may be at risk for underachievement. This information can than be utilized to inform decisions regarding the implementation of interventions evidenced to help to change this trajectory. By developing such a systemic interface, as well as creating more standardized measures of expected learning outcomes, we are facilitating an effective advancement of the curriculum in our department. Objective 2: To identify predictor variables of learning in order to help us implement evidence-based interventions that target contributing factors of underachievement. Examples of predictor variables: Demographics study habits perceptions of learning experiences Co-Presenters: Ryan Curtis, Cassandra Jones-Abdelmeguid, Emily Gordon, Alexandra Jacobsen, Charles Stangor, Scott Roberts, Psychology Twenty-First Century Learning: Assessing and Improving Learning on the Department Level