370 likes | 461 Views
Fall Faculty Workshop August 21, 2008 Assessment [ CLA ]. Why Assessment Four Approaches to Assessment The Collegiate Learning Assessment [ CLA ] Using the CLA at Morningside. Why Measure Educational Outcomes?. We measure what we value, we value what we measure
E N D
Fall Faculty Workshop August 21, 2008 Assessment [CLA]
Why Assessment • Four Approaches to Assessment • The Collegiate Learning Assessment [CLA] • Using the CLA at Morningside
Why Measure Educational Outcomes? • We measure what we value, we value what we measure • Improve quality of student learning by creating a culture of assessment • Informing pedagogy and curricular design • Making assessment integral part of teaching and learning • Create formative evaluation for the institution, programs, major • Enable conversations about academic expectations and standards CAE –[CLA] slide
Why Measure Educational Outcomes? • Given accountability pressures, if we don’t initiate, it will be done for us CAE –[CLA] slide
Four Approaches • Actuarial indicators (graduation rates, access) • Quality rankings (US News & World Report) • Student surveys (NSSE, CIRP) • Direct measures of student learning [CLA] CAE –[CLA] slide
Four Approaches: Actuarial Indicators • What are the admissions test scores of entering students? • What percent of students graduated? • How diverse is the student body? • For example, see U-Can at: http://www.ucan-network.org/searchInstitution.asp CAE –[CLA] slide content
Four Approaches: Surveys To what extent has your experience at this institution contributed to your ability to think critically and analytically? Very Much Quite a Bit Some Very Little For example: The National Survey of Student Engagement (NSSE) http://www.usatoday.com/news/education/2007-11-04-nsse-how-to_N.htm?loc=interstitialskip CAE –[CLA] slide content
Four Approaches: Rankings • Question for presidents, provosts and deans of admission: rate the academic program of other universities, where 1 = marginal and 5 = distinguished. • Measuring value-added: what percentage of your students did you expect to graduate, and what percentage actually did? • For example: US News http://colleges.usnews.rankingsandreviews.com/usnews/edu/college/rankings/brief/t1ccbach_mw_brief.php CAE –[CLA] slide content
Current Approaches: Direct Measures • Measures that assess what students are actually able to do For example, The Collegiate Learning Assessment [CLA] CAE –[CLA] slide
The CLA measures four sets of higher order skills The CLA measures in a holistic manner: Critical thinking Analytic reasoning Problem solving Written communication CAE –[CLA] slide
CLA Scoring CriteriaCritical Thinking, Analytic Reasoning, and Problem Solving Skills • Evaluation of Evidence • Analysis and Synthesis of Evidence • Drawing Conclusion • Acknowledging Alternative Explanations/Viewpoints CAE –[CLA] slide
CLA Scoring CriteriaWriting Skills • Presentation • Development • Persuasiveness • Mechanics • Interest CAE –[CLA] slide
The CLA gets at some of our educational outcomes: • Demonstrate analytic, synthetic, creative, evaluative, and quantitative thinking. + • Communicate effectively + • Behave ethically and responsibly - • Use knowledge of cultures to enhance understanding of themselves and others - • Apply knowledge and skills from multiple, diverse disciplines and practical experiences to understand complex issues and solve problems ? • Exhibit a passion for life-long learning - • Effect positive change through leadership or active participation in communities - • Articulate their own spirituality and values, while understanding those of others -
There are three types of CLA measures Analytic Writing Task: Make-an-Argument Analytic Writing Task: Critique-an-Argument Performance Task CAE –[CLA] slide
Analytic Writing Task: “Make-an-Argument” Writing a persuasive, analytic essay to support a position on an issue CAE –[CLA] slide
Make-an-Argument “Government funding would be better spent on preventing crime than in dealing with criminals after the fact.” Directions: 45 minutes, present your perspective on the issue, using relevant reasons and/or examples to support your views CAE –[CLA] slide
Government Funding Establish a thesis Maintain the thesis Support the thesis with examples Anticipate and counter opposing arguments CAE –[CLA] slide
Analytic Writing Task: “Critique-an-Argument” Critiquing Written Arguments CAE –[CLA] slide
Analytic Writing Task: “Critique-an Argument” “Butter has now been replaced by margarine in Happy Pancake House restaurants throughout the southwestern United States. Only about 2 percent of customers have complained, indicating that 98 people out of 100 are happy with the change. Furthermore, many servers have reported that a number of customers who still ask for butter do not complain when they are given margarine instead. Clearly, either these customers cannot distinguish margarine from butter, or they use the term "butter" to refer to either butter or margarine. Thus, to avoid the expense of purchasing butter, the Happy Pancake House should extend this cost-saving change to its restaurants in the southeast and northeast as well.” Directions: 30 minutes, discuss what is wrong with the argument CAE –[CLA] slide
The Happy Pancake House Identify logical flaws or fallacies How do these logical flaws influence the argument’s conclusion CAE –[CLA] slide
Performance Task Analyzing complex, realistic scenarios CAE –[CLA] slide
Performance Task • Features: • Open-Ended • Document-Based • “Real Life” Scenarios • Components: • Document Library CAE –[CLA] slide
Crime Reduction “Pat Stone is running for reelection as the mayor of Jefferson, a city in the state of Columbia. Mayor Stone’s opponent in this contest is Dr. Jamie Eager. Dr. Eager is a member of the Jefferson City Council. You are a consultant to Mayor Stone. Mayor Stone wants to make sure the best policy for reducing crime in Jefferson is identified, so has asked you to analyze the strengths and/or limitations of Dr. Eager’s proposal.” Directions: 90 minutes, use the evidence from the Document Library to answer the following questions. CAE –[CLA] slide
Performance Task Synthesize information from multiple sources Recognize conflicting evidence Interpret data, tables, figures correctly Identify logical fallacies Develop conclusions based on available evidence CAE –[CLA] slide
Crime Reduction • Student is advising the mayor, who is running for re-election. • There is an upcoming mayoral debate, for which the student must help the incumbent mayor prepare. CAE –[CLA] slide
Crime Reduction • There are two policy approaches for reducing crime: • Drug education program • Increasing number of police officers on the streets CAE –[CLA] slide
Crime Reduction • Documents available to the student include: • Newspaper article about crime in the community • Research abstracts about drug education program • Report about success of a drug education program in another community • Police report (with table of data) about crime and drug use in the community • Plots of the relationship between police offers and crime • Private investigator report about possible connection between opponent and drug education program CAE –[CLA] slide
Crime Reduction • Using these data, what does the student advise that the mayor should do? CAE –[CLA] slide
Performance Tasks • Not all quantitative • Artwork and Technology CAE –[CLA] slide
At Morningside College • Invited to join the CIC-CLA consortium • approx 40 colleges • four year commitment • Will administer the CLA to at least 100 first year students and 100 seniors each year • This year all 300 first year students • Share data and work with the consortium CAE –[CLA] slide
CIC-CLA Consortium Alaska Pacific University Allegheny College Aurora University Averett University Barton College Bethel University (MN) Cabrini College Carlow University Charleston Southern Univ. College of Notre Dame (MD) College of St. Benedict/Saint John’s University Dominican University Drake University Franklin Pierce University Hilbert College Illinois College Indiana Wesleyan Univ. Jamestown College John Carrol Univ. Juniata College LaGrange College Lynchburg College Marion University (WI) Morningside College Nebraska Wesleyan Univ. Pace University Pacific University Seton Hill University Southwestern University (TX) Springfield College Stephens College Stonehill College Texas Lutheran University The College of St. Scholastica University of Findlay Trinity Christian College University of Charleston University of Evansville University of Great Falls Upper Iowa University Ursinus College Ursiline College Wagner College Wartburg College Westminster College (MO) Westminster College (UT) Williamette University William Woods University
Questions? Other Options
CLA in the Classroom helps address the “What Now?” question The key to accountability is not just transparency, but also improvement CLA in the Classroom connects the institution-wide CLA results to the classroom-level and faculty work on student learning A means to link institution-wide summative assessment with local, formative work
One component of the program supports diagnostic work Use of a retired (“[cla]ssic”) performance task in a classroom, with administration, scoring and advising resources In addition, classroom activities are suggested Designed to provide diagnostic feedback to students to understand why they achieved the scores they did, and what to do next to improve their skills
Another key component enables curricular efforts In addition, faculty walk through a process to develop their own performance tasks Faculty can embed course content in the tasks they create These faculty-developed tasks could be used as class assignments or group projects
CLA in the Classroom Academy Gain a deeper understanding of the CLA Learn how to administer and score a disclosed CLA Performance Task Develop performance tasks where course content can be embedded Learn more about rubric-based assessment as it applies to the CLA Brainstorm strategies for using these approaches in courses or across the institutions