“ Just Do It” Assessment: A Five Step Plan for Busy Faculty

1. “Just Do It” Assessment:A Five Step Plan for Busy Faculty Assessment Bites #4 March 1, 2007 Presenter: Mimi Harris Steadman Ed.D.Director of Institutional Assessment (716) 839-8567 msteadma@daemen.edu

2. Agenda • Assessment at Daemen • Focused Five Step Assessment Plan • Developing Learning Objectives • Collecting Information • Using Results • Good Assessment

3. Workshop Learning Objectives Participants will be able to: • Identify different types of assessment at Daemen College • List the steps of a basic assessment plan • Articulate learning objectives for a course or program • Describe a variety of direct and indirect assessment methods

4. Assessment Round Up What kinds of assessment activities are happening at Daemen? What information about student learning and institutional effectiveness do we already collect?

5. Assessment at Daemen • Institutional Assessment • The overarching plan for collecting and reporting information about institutional effectiveness and student learning across the institution. • Information collected at an institution-wide level, such as enrollment and retention data, and surveys like the NSSE, CIRP and YFCY. • Core Curriculum Assessment • Assessing student achievement of the core competencies, through direct methods such as capstone courses, senior projects, and other student work collected via course instructors or e-portfolios, and through indirect methods such as student surveys or focus groups. • Departmental Assessment • Assessing student learning in majors or programs through direct methods such as written student work or performances, capstone courses, and success rates on licensure exams or national exams, and through indirect measures such as program reviews, job placement rates, and employer, alumni or student surveys. • Classroom or Course Assessment • Assessment that determines whether students are learning what we think we are teaching, and assessment that provides feedback to learners. Direct methods for classroom assessment include assignments, tests, papers, CATs and projects or presentations.

6. 5 Step Assessment: ACRRU BRAINSTORM the basic steps of assessment: • A is for… • C is for… • R is for… • R is for… • U is for…

7. Focused Five Step Assessment Plan (ACRRU) 1. Articulate learning objectives -As a result of this program/course, students should be able to… (*Offer instruction or other learning experiences*) 2. Collect information/evidence -Using direct and indirect methods 3. Review/analyze data and reflect on findings 4. Report results 5. Use information for decision making and improvement

8. 1. Articulate Learning Objectives • Before we can plan for assessment, we must start by defining learning objectives. • Learning objectives describe what a student knows, believes, or is able to do. • Consider the alignment of objectives with your program or institutional mission. • Learning objectives (or goals) are broader at the institution or general education level and get narrower at the program, department or course level. • Learning objectives guide course design and instruction. • A few will do. Stagger your assessment and focus on different objectives each year.

9. Why learning goals and not teaching goals? The true test of teaching effectiveness lies not in what was taught, but in what was learned. Teaching without learning is just talking.-Tom Angelo and Pat Cross • In other words, just because I “covered” the material, doesn’t mean my students learned it. Articulate Learning Objectives

10. Useful learning objectives: • Describe what we expect our students to know and be able to do • Use verbs that describe observable action • Describe learner rather than teacher actions • Do not describe course content coverage • Describe an acceptable performance level • Can be assessed by one or more methods • Make sense to colleagues and are clear to outsiders Articulate Learning Objectives

11. Sample learning objectives • Program: General Education • Program Goal: Recognize and appreciate artistic and literary contributions of diverse cultures • Course in Program: Caribbean Literature • Course Goal: Demonstrate familiarity with themes and genres of classic and contemporary Caribbean literature (taken from Student Learning Assessment: Options and Resources, MSCHE 1993, p. 14) Articulate Learning Objectives

12. More sample objectives • Program: History • Program Level Goal: Communicate orally and in writing about historical topics • Course in program: Medieval History • Course Goal: Present a cogently-written, critical analysis of gender and class roles in Medieval England (taken from Student Learning Assessment: Options and Resources, MSCHE, 1993, p. 14) Articulate Learning Objectives

13. Developing learning objectives Start with: • Existing departmental goal statements (including those that just describe teaching goals rather than learning goals) • General education goal statements (e.g., critical thinking, lifelong learning) • Disciplinary organizations • Professional accrediting organizations • Assessment plans and reports from other colleges (see http://www2.acs.ncsu.edu/UPA/assmt/resource.htm) Articulate Learning Objectives

14. Specifying learning objectives Instructions: With one or two others, role play student and teacher and generate a response to the following: “If I’m your student, what do I have to do to convince you that I’m where you want me to be at the end of this lesson, unit or course?” • _________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Reference: R. M. Diamond (1998). Designing and Assessing Courses and Curriculum. San Francisco: Jossey-Bass, pages 134-135. Articulate Learning Objectives

15. 2. Collect Information/Evidence Use at least two measures, one direct and one indirect. Direct Measures: direct review of student work on an exam, a project, or performance (with set criteria for evaluation), licensure or certification exams, etc. Indirect Measures: student reflections on learning, student, alumni, or employer surveys, focus groups, placement rates, etc. Data can be new or existing, qualitative or quantitative, collected at the course, program, or institutional level. You don’t need lots of data, just the right data. If we knew what it was we were doing, it would not be called research, would it? –Albert Einstein

16. Data Collection Resources See handouts: • Examples of Direct and Indirect Measures of Student Learning • Quick Tip 6.0 • Data Gathering: Methods • Quick Tip 6.02 • Choosing a Data Gathering Method Collect Information

17. Do not let the perfect be the enemy of the good. “To avoid paralysis, a campus is well advised to remember that assessment is an ongoing, flexible, naturally evolving process offering many opportunities to fine-tune or change direction. Meanwhile, even an imperfect process can yield useful results” (Leskes and Wright, 2005, p. 23). Collect Information

18. 3. Review and Reflect on Findings • Display and discuss findings in a group (ideally in an annual retreat meeting format) • Start by looking for general trends, then analyze key issues more closely • Identify areas for action or improvement • Identify additional questions raised and future focus areas for assessment The outcome of any serious research can only be to make two questions grown where only one grew before. -Thorstein Veblen

19. 4. Report Results • In a concise format, report on findings, plans for action, plans for future assessment, and resource issues. • Who are the important audiences for your report, and how will information reach them? • Are there other venues for sharing assessment findings? • Don’t forget to celebrate successes. How important it is for us to recognize and celebrate our heroes and she-roes! -Maya Angelou

20. 5. Use Assessment Information for Decision Making & Improvement “The aim of assessment is primarily to educate and improve student performance, not merely to audit it.” -Grant Wiggins (1998). Educative Assessment, San Francisco: Jossey-Bass, p. 7.

21. Suskie’s Five Dimensions of Good Assessment • Used • Cost-Effective • Reasonably accurate and truthful results • Valued • Focus on and flow from important goals Linda Suskie, Middle States Commission on Higher Education, October 10, 2006 (presented at the IUPUI 2006 Assessment Institute in Indianapolis).

22. Aim for an assessment process that is scholarly and professionally engaging, rather than tedious and time consuming. Keep your plans lean, focused, and useful so that the assessment process is sustainable and supports, rather than detracts from, the work that matters most.

23. Resources Marilee J. Bresciani, Carrie L. Zelna and James A. Anderson. (2004). Assessing Student Learning and Development. A Handbook for Practitioners. The National Association of Student Personnel Administrators. Andrea Leskes and Barbara D. Wright (2005). The Art & Science of Assessing General Education Outcomes. Washington, D.C.: The Association of American Colleges and Universities. Middle States Commission on Higher Education (2003). Student Learning Assessment: Options and Resources. Philadelphia: MSCHE. Barbara E. Walvoord (2004). Assessment Clear and Simple. A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass.

24. Instructional Assessment Resources from The University of Texas at Austin, Division of Instructional Innovation and Assessment, (including “Quick Tip” sheets on topics such as rubrics, surveys, qualitative analysis and more). http://www.utexas.edu/academic/diia/assessment/iar/index.php • Internet Resources for Higher Education Outcomes Assessmentfrom the North Carolina State University Office of University Planning and Analysis. Links to everything related to assessment in higher education, including articles and associations, organized by technique, by discipline, by institution, and more. NCSU hosts the 2007 NC State Undergraduate Assessment Symposiumin the Raleigh area April 13 - 15, 2007. http://www2.acs.ncsu.edu/UPA/assmt/resource.htm • National Assessment Institute sponsored by IUPUI (Indiana University-Purdue University Indianapolis) is a comprehensive assessment conference each fall. http://www.planning.iupui.edu/51.html • ***COMING SOON: The Daemen College Assessment Web Page***