Assessment 101: From Definitions to Practice

1. Assessment 101:From Definitions to Practice Marcel S. Kerr Summer 2007

2. Presentation Impetus & Purpose Impetus: Committee work (GEC) Faculty professional development survey Current piece mill training Purpose: Provide foundation of assessment terms Differentiate levels of assessment Review Wesleyan�s recurrent assessments

3. Workshop Agenda General Definitions Assessment at Wesleyan Reference Worksheet Responsibility Assessment Tools Individual Feedback Quiz Completion Certificate

4. General Definitions Assessment � a systematic process of gathering information to meet a broad range of evaluation needs. Purpose: Formative � information used as feedback to modify teaching and learning; improvement; when = any time; repeatedly Summative - information gathered at the conclusion of a course, program, or undergraduate career to improve learning; evaluation or accountability; when = end of assessment period Type: Direct � embedded course assessment; Ex. Class assignment Indirect � external; Exs. National standardized tests; GPA; student satisfaction surveys 1) In discussing the definitions of basic assessment terminologies, I will step us through the assessment process.1) In discussing the definitions of basic assessment terminologies, I will step us through the assessment process.

5. General Definitions Level of Assessment � the unit that is assessed Five Levels � Miller & Leskes, 2005: Individual Learning within a Course Individual Learning across Courses Courses Programs The Institution After you have selected type of assessment and purpose, you want to clarify the assessment level. In higher education, we have five levels. 1. Classroom assessment Assessment of individual students performance at the course level by an instructors � within & across 3. Course assessment Assessment of how well a course is meeting student learning outcomes 4. Program assessment Assessment of how well an academic program is meeting student learning outcomes Assessment of how well an educational support program is meeting its objectives 5. Institutional assessment Assessment of campus-wide issues or programs Where are the faculty the units of assessment? After you have selected type of assessment and purpose, you want to clarify the assessment level. In higher education, we have five levels. 1. Classroom assessment Assessment of individual students performance at the course level by an instructors � within & across 3. Course assessment Assessment of how well a course is meeting student learning outcomes 4. Program assessment Assessment of how well an academic program is meeting student learning outcomes Assessment of how well an educational support program is meeting its objectives 5. Institutional assessment Assessment of campus-wide issues or programs Where are the faculty the units of assessment?

6. General Definitions Objectives � specific statements of achievement; future tense Ex. Students will be able to solve problems using the scientific method. Outcomes � specific descriptions of acquired achievement; past tense; are derived from the objectives; any end result. Intentional � mirror the objectives; Ex. 85% of the students earned a 4 or higher on scientific problem solving project Unintentional � additional ones; Ex. alumni surveys revealed that students desired more real problem-based assignments in the curriculum. In Step 3, we must decide what it is about the unit we are assessing � OUTCOMES ASSESSMENT means quality, effectiveness, achievement; in HE, that basically is student learning. 1)Often used interchangeably = objectives = expected outcomes 2) Really only differ in when they are used or written. 3) Unintentional outcomes � may be positive or negativeIn Step 3, we must decide what it is about the unit we are assessing � OUTCOMES ASSESSMENT means quality, effectiveness, achievement; in HE, that basically is student learning. 1)Often used interchangeably = objectives = expected outcomes 2) Really only differ in when they are used or written. 3) Unintentional outcomes � may be positive or negative

7. General Definitions Objectives and outcomes are expressed with standards. Standards � achievement of the specified unit competencies at the level of performance established for successful completion; criteria is synonym Content - comprise the knowledge and skills specific to a given discipline. Performance - established level of achievement, quality, or proficiency. Performance standards set expectations about how well the unit should perform. As part of step 3� objectives and outcomes are often expressed with criteria. In psychology, Majors must have a GPA of 2.5 or higher to graduate with a Bachelors degree in Psychology. What is the unit of assessment in this outcome? 2) Content � Students will differentiate graduate degrees in psychology by their purpose. Objective or outcome? 3) Performance standard � Advisors will articulate each advisee�s transcripts prior to the completion of 60 credit hours. Unit? Objective or outcome? 4) At Wesleyan = we use the word criteria when we refer to performance standards.As part of step 3� objectives and outcomes are often expressed with criteria. In psychology, Majors must have a GPA of 2.5 or higher to graduate with a Bachelors degree in Psychology. What is the unit of assessment in this outcome? 2) Content � Students will differentiate graduate degrees in psychology by their purpose. Objective or outcome? 3) Performance standard � Advisors will articulate each advisee�s transcripts prior to the completion of 60 credit hours. Unit? Objective or outcome? 4) At Wesleyan = we use the word criteria when we refer to performance standards.

8. General Definitions Measurement Indicators � the actual measures or data collected and used to assess the unit; evidence of achievement Student grades, course enrollment history, #of graduate majors over time Qualitative � non-numeric; answer how and why; acquired via open-ended questioning Quantitative � numeric data; answer what, where, and when; acquired via forced choice (yes/no, Likert, MC) Objective � forced-choice Subjective � open ended Using multiple measurements is key! Once objectives or outcomes are written, we are ready to identify and collect some data in step 4. Qualitative � why did you choose to major in English? Quantitative � course completion rates Which are easier to analyze and interpret? Q vs Q and Obj vs Sub? 5) Several indicators are generally needed to adequately describe each outcome. (McTighe & Ferrara) Once objectives or outcomes are written, we are ready to identify and collect some data in step 4. Qualitative � why did you choose to major in English? Quantitative � course completion rates Which are easier to analyze and interpret? Q vs Q and Obj vs Sub? 5) Several indicators are generally needed to adequately describe each outcome. (McTighe & Ferrara)

9. Effective Program Assessments Answer these Questions What are you trying to accomplish? How well are you doing it? How, using the answers to the first two questions, can you improve what you are doing? What and how does a program contribute to the development and growth of its students and/or the support of its customers? How can student learning be improved? After completing the program, what knowledge and skills should they have acquired? Pet-Armacost & Armacost, 2006 In choosing measurement indicators, keep the following questions in mind. Answering these questions completes the assessment process for a program.In choosing measurement indicators, keep the following questions in mind. Answering these questions completes the assessment process for a program.

10. Assessment at Wesleyan 1) Programs primary purpose is continual improvement. Formative or Summative? 2) University primary purpose is accountability and evaluation. Formative or Summative? 3) All together determine the university�s effectiveness. Each assessment affects the others. 4) All feed and affect budgetary decision making. Pull out Reference Worksheet1) Programs primary purpose is continual improvement. Formative or Summative? 2) University primary purpose is accountability and evaluation. Formative or Summative? 3) All together determine the university�s effectiveness. Each assessment affects the others. 4) All feed and affect budgetary decision making. Pull out Reference Worksheet

11. Annual Program Assessment

12. Program Self-Study Audit why: improvement or evaluation? Formative or summative?why: improvement or evaluation? Formative or summative?

13. Program Review 1) Why � formative or summative?1) Why � formative or summative?

14. Strategic Planning How: In academic division, the provost writes the plan with the Deans (Provost�s Council).How: In academic division, the provost writes the plan with the Deans (Provost�s Council).

15. Assessment atWesleyan

16. Responsibility Administration � institutional effectiveness Deans, Directors, & VPs � Programs & Divisions Faculty � Programs, Courses, Students Students �learning & advising Administration � participants and oversight Who has the Lion�s share of assessment work load?Administration � participants and oversight Who has the Lion�s share of assessment work load?

17. Assessment Tools Online � plans, rubrics, portfolios, shared learning objects Inventory of Higher Education Assessment Instruments (NCPI) My Surveys Use Wesleyan login userid and password Create surveys, quizzes, end of course evaluations Scores quizzes Generates summary reports URL: http://web1.txwes.edu/survey/login.jsp?r=1 Merlot National Center for Postsecondary ImprovementMerlot National Center for Postsecondary Improvement

18. Self Assessment What have we learned? Please complete the 10-item online quiz. It is a quick formative assessment of what we�ve covered today. Upon completion, you will receive a feedback page with your final score. Please print a copy of this page and forward it to your Dean or VP as evidence that you have completed the assessment training.

19. References Pet-Armacost, J., & Armacost, R. (2006, February). Think �system� to achieve department and program assessment success. Paper presented at the 6th Annual Texas A&M Assessment Conference. McTighe, J., & Ferrara, S. (1995). Assessing Learning in the Classroom. Journal of Quality Learning, 95-112.). Miller, R., & Leskes, A. (2005). Levels of assessment: From the student to the institution. DC: Association of American Colleges and Schools. National Center for Postsecondary Improvement. (2007). Inventory of Higher Education Assessment Instruments. Retrieved June 10, 2007 from http://www.stanford.edu/group/ncpi/unspecified/assessment_states/instruments.html