Assessment Workshop Prairie View A&M University November 21, 2008

1. Assessment WorkshopPrairie View A&M UniversityNovember 21, 2008 Celebrating 25 Years of Excellence in Teaching and Learning - 1983-2008 http://assessment.tamu.edu

2. Outcomes for Today’s Session: By the end of this session, participants will be able to: • Describe the role of program assessment in SACS accreditation • Identify the steps of a program assessment plan and complete a plan given a sample program learning objective • Describe the benefits of course embedded assessment used for program assessment • Create part of a rubric that could be used in program assessment • Self-assess your own knowledge with regards to program assessment

3. Agenda • Introductions (5 min.) • Accreditation Overview (15 min.) • Writing Program Learning Objectives and the Program Assessment Cycle (15 min.) • Break (10 min.) • Course-embedded Assessment (30 min.) • Rubric Use and Development (40 min.) • Resources (10 min.) • Questions (15 min.)

4. Assessment Steps-- Where are You? 1. Written Mission 2. Developed program-level learning outcomes 3. Determined measures 4. Gathered results 5. Used the results for program improvement

6. Accreditation “Within the framework of the mission….”

7. What does accreditation mean for faculty? • Assessment of student learning

8. SACS Standard: Institutional Effectiveness “The institution identifies expected outcomes, assesses whether it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:

9. SACS Standard:Institutional Effectiveness • Educational programs, including student learning outcomes • Administrative support services • Educational support services • Research within its educational mission, if appropriate • Community/public service within its educational mission, if appropriate Revisions to the Principles of Accreditation: Foundations for Quality Enhancement, December 12, 2006

10. Level of Commitment/Interest High Med Faculty Administration Low Institutional Program Course Assessment

11. Develop Program Mission & Outcomes Design an Assessment Plan Modify and Improve Assessment Process Interpret/Evaluate Information Implement the Plan & Gather Information Adapted from: Trudy Banta, IUPUI

13. Prior Knowledge? Program Assessment Cycle Program Mission PROGRAM LEARNING OBJECTIVES INTERPRETATION: Reflection/ Documentation Course Portfolio ALL ASSESSMENT: Capstone Course project or assignment Nat’l Normed Exam Eval of Portfolio Internship Perf. Review Senior Survey Alumni Survey SELECTED ASSESSMENT: Capstone Nat’l Exam Portfolio Sr Survey

14. Form of Learning Outcomes • Learning Outcomes must: • Contain a verb describing an observable or identifiable action • Focus on the student as the performer • What is the student expected to be able to know? • What is a student expected to be able to do? • How is a student expected to be able to think?

15. Activity • Choose one program learning objective for your area • What will the graduate know, be able to do and how will they think when they walk across the stage? • How will you know? • Share with a partner, improve with their feedback

16. Program Assessment Plan Plan Development Steps • Specify the Program Learning Objectives • Identify related course(s) • Identify course embedded assessment techniques • Establish assessment criteria

17. From Program Learning Objectives to Course Embedded Assessment • Specifying Program Learning Objectives • What will the graduate know, be able to do and how will they think when they walk across the stage? • How will you know? • Identifying Related Courses and Learning Outcomes • What courses reveal student mastery and summative assessment to address this program learning objective? • What must students do to prove that they have mastery?

18. Where will the Program Learning Outcomes be Assessed? Assess to determine achievement of program learning outcomes Student Teaching Course 14 Capstone Course 16 Senior Course 9 Course 10 Internship ?????? Junior Assess to determine where and how interventions to improve program learning outcomes might be accomplished Course 5 Course 6 Course 7 Course 8 Sophomore Course 1 Course 2 Course 3 Course 4 Freshman

19. Advantages of Course Embedded Assessment You’re doing this already… • Relationship to program learning objectives is obvious • Flexibility in assessment design and administration—Faculty ownership • No additional cost for instrumentation or assessment techniques • Efficient

20. Course Embedded Assessment Techniques • Questions embedded in final exams/tests • Common exams/tests questions across sections of a course • Use of rubrics, checklists, rating scales, etc., to evaluate student performance in applicable knowledge/skills areas on essays, research papers, case study reports, oral presentations • Pre- & post-tests

21. Program Assessment Plan Sample 1 The Department of Chemical Engineering, Texas A&M University Program Learning Objective: An ability to use the techniques, skills, and modern engineering tools necessary for chemical engineering practice Course Learning Outcome: Use advanced chemical process simulation software to design integrated unit operations at the chemical plant scale. Related Course: CHEN 426: Plant Design Assessment Measure: As a course requirement, students write a final design report and produce an oral presentation. An industry panel evaluate students’ presentations using an evaluation rubric. One of the questions address students’ skill to use advanced chemical process simulation software to design integrated unit operations at the chemical plant scale. Assessment Criteria: At least 85% of students will be rated at 2 (“meets expectations”) and above on ability to use advanced chemical process simulation software to design integrated unit operations at the chemical plant scale.

22. Program Assessment Plan Sample 2 Program Learning Objective: Students will recognize and apply Texas law. Course Learning Outcome: Students will define elements of crimes and penalties using Texas statutes as illustrations. Related Course(s): CMJU 401 Assessment Measures: Students will complete a final examination in which questions related to their understanding of the elements of crimes and penalties using Texas statutes have been embedded. Members of the program assessment committee will evaluate students’ answers to the embedded questions. Assessment Criteria: At least 70% of students completing the final examination will respond correctly to 80% or more of embedded questions.

23. Examples from the Participants • Brainstorm several course embedded techniques that address your program learning objective.

24. What is a rubric? • AKA: Rating scale or grading template • A chart or list that states the criteria and/or characteristics upon which student work will be evaluated; and • Defines and describes two or more levels of quality for each criteria. • A tool to measure attainment of selected program learning outcomes

26. Why use a rubric? • As an evaluation tool for instructors: • At the program level: • Assists in making course outcomes and program learning objectives more explicit and transparent • Ensures consistency and fairness in analyzing program learning objectives through shared analysis • Saves time • Helps analyze program strengths and weaknesses. • At the course level: • Helps define “A” work, “B” work, etc. • Increases level of trust between students and instructor • Saves time • Helps analyze student strengths and weaknesses. • Assists in making course outcomes and program learning objectives more explicit and transparent

27. Why use a rubric? – cont. • As an instructional tool with students • Reveals the standards of the discipline/course/assignment • Can be used to involve students in setting standards and criteria for performance • To provide students with benchmarks for developing, judging, and revising their own work, thus developing their ability to self-monitor, self-direct and self-assess their own learning

28. Ex: Rubric – Article Review

29. How to create a rubric Identify characteristics you use in evaluating student work For each characteristic, describe what exemplary student work would look like, do the same for the lowest level Decide how many levels are appropriate or necessary for each characteristic Describe the remaining levels Assign a weight for each characteristic Check for reliability, e.g., practice with sample student work, ask a colleague to review/use the rubric with the samples, revise, revise, etc.

30. Activity: Program Assessment Plan Program Learning Objective: Your program area example… Course Learning Outcome: Related Course(s): Assessment Measures: Assessment Criteria:

31. Take-Home Messages • You do not have to assess everything every year • Modify something already being done that is meaningful to the program • Be flexible—this is an iterative process • This is a faculty-driven process (The outcomes you choose should be important to you.)

32. References Palomba, C.A. and Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in Higher Education. San Francisco: Jossey-Bass. Huba, M.A. & Freed, J.E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Boston: Allyn and Bacon. pp 151-200. Andrade, H. G. (February, 2000). What do we mean by results?: using rubrics to promote thinking and learning, Educational Leadership, 57(5), pp13-18.

33. What do we want every undergraduate to be able to know and do when he or she graduates from Texas A&M University? A TAMU graduate will: • Master the depth of knowledge required of a discipline. • Demonstrate critical analytical skills. • Communicate effectively in writing and speaking. • Provide ethical leadership in a global and diverse society.

34. For more information on the conference and registration, visit http://assessment.tamu.edu/conference February 22-24, 2009 College Station, TX