Retention: YOU are the Key! Instructor Strategies for Retaining Students

1. Retention: YOU are the Key!Instructor Strategies for Retaining Students Troy University eTROY Colloquium April 17-18, 2012

2. Assessment is Not a Four-Letter Word!  Assessment roles of faculty and adjuncts in eTROY Dr. Wendy Bailey – Sorrell College of Business Dr. Christina Martin – College of Health & Human Services Dr. Isabelle L. Warren – College of Education

3. Objectives of this presentation • Review the purpose and goals of assessment of student learning • Clear up concerns about assessment • Provide an introduction to the assessment cycle • University and Program, Linkages, Outcomes, Data Collection/Reporting, Action Plans • Define the assessment role of faculty and adjunct professors • Explain how Blackboard Outcomes can help and the timeline for implementation

4. So what exactly is assessment? Assessment is the ongoing process of: Establishing clear, measurable expected outcomes of student learning tied to program and university missions Ensuring that students have sufficient opportunities to achieve those outcomes in the curriculum Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations Using the resulting information to make changes that improve student learning Continuing to measure, evaluate and make further changes as part of a process of continuous improvement - adapted from Linda Suskie, Assessing Student Learning, 2009

5. Once upon a time…. • Programs designed curriculum they thought would give students the background they need in a particular field • Instructors taught courses in the curriculum and gave grades for their course based on student performance overall • If students did well in the course, we assumed they knew what we wanted them to • …. But did they?

6. Contemporary view Learning is a function of the curriculum, not the course. • Curriculums need to be more than a collection of courses. They need to ensure that every student has ample opportunity to achieve key institutional and program learning goals. • Institutional and program goals are broader than the course goals themselves, and should be reinforced throughout the curriculum. • Students need to see the connections among their courses and other learning experiences as this makes learning deeper and longer lasting.

7. Why are we doing assessment? The goal of assessment is to improve our academic programs. Good assessment programs: • Help us to stay focused on and to continually evaluate our stated program goals • Bring faculty and staff together to discuss important issues related to teaching and standards used • Help faculty, staff, and students see how courses link together to achieve program goals • Identify issues that may impede student learning • Allow us to make better decisions that are based on data, rather than hunches, anecdotes, or intuition

8. Class vs. Program-Level Assessment When a professor assesses the work done in a course, assessment has a different purpose than program-level assessment. • In a class, every student is assessed, the professor sets their own criteria and standards (explicit or not), the professor evaluates the work and the student gets the result. • In program-level assessment, sampling is acceptable, the faculty (not one professor) sets explicit criteria and standards, the faculty or outsiders do the evaluation, and the feedback goes to the program faculty.

9. What is the goal? Program-level assessment checks if students are on track to achieve or have achieved important program goals. • Since these goals are reinforced throughout the curriculum, an assessment in one course doesn’t measure what that professor did or did not do, but what the program to that date has achieved. • Evaluation of individual faculty should never be the goal of program-level assessment and the results should never be used that way. • Assessment is a team sport! We celebrate our victories together, and we figure out how to change things together if results are disappointing.

10. Common Objections to Assessment “I have assessed students! I assigned them a course grade.” • Course grades are not sufficient for program-level assessment. • An overall course grade of a B doesn’t tell us which skills and concepts they have mastered. • Sally may have gotten a C average on exams and an A in her project, while John may have gotten an A average on his exams and a C in his project. • Both earned a B, but we won’t know looking at the grade alone what each has mastered and what skills need more work.

11. Common Objections to Assessment “Mandating program level assessment violates the principle of academic freedom.” • “Academic freedom does not absolve instructors of their responsibility to ensure that all students in their program… have sufficient opportunity to achieve those goals that the faculty collectively agree are essential… • “Academic freedom also does not relieve faculty of the obligation to assess student learning of their subject… -- Linda Suskie, Assessing Student Learning: A Common Sense Approach, 2009

12. Common Objections to Assessment “I do not have time to conduct assessments – as it is not within my job description.” • Assessment is everyone’sresponsibility. • Accrediting agencies want to see that faculty know the program’s goals and that they are involved in assessment activities. • To save time, be smart about assessment. Course-embedded assessments can serve double-duty. Example: capstone research project • Instructor evaluates for course grade (content, etc.) • Samples of the same projects can be evaluated by faculty using two different rubrics for writing and critical thinking skills to evaluate the achievement of program-level goals.

13. Assessment Cycle • Mission

14. Step 1: Set goals, objectives, & SLO’s Program goals are broad, conceptual statements that show the long-term aim or purpose of the entire course of study. Goals should be related to the college’s mission, as well as the institution’s. Program objectives are more specific than goals and are more short-term. They indicating the intended consequences of instruction within a timeframe. Program student learning outcomes (SLO’s) describe significant & essential learning that students should achieve or reliably demonstrate at the end of a program. SLO’s must be specific, observable, & measurable.

15. Step 2: Align curriculum with outcomes Next, examine your curriculum. • Where are concepts related to each outcome introduced, reinforced, or mastered? • Students should be given multiple opportunities to master important program goals. • Are there gaps in your curriculum?

16. Example of Curriculum Map I = introduced, R = reinforced, M = mastered

17. Step 3: Establishing Measures and Criteria Assessments measure whether students have achieved the program learning outcomes we’ve set. • Direct assessment measures provide the strongest evidence of student learning because they are based on actual student performance on a task. • Indirect assessment measures supplement direct measures. They provide information about student perceptions about learning experiences & attitudes towards the learning process, as well as program quality. • No assessment of learning outcomes should be based on indirect measures of learning alone!

18. Specify Your Assessment Criteria Criteria are standards of performance required to meet the objective/outcome, that is, the quality to be judged in the assessment task: • Quality words often used in criteria: clarity, accuracy, depth, legibility, impact, relevance, etc. • Example: “Clarity of explanation” is a criterion for “Students will be able to explain how concepts in the subject interrelate.” • May be expressed as a percentage, a target number of accomplishment, a rate, an increase over a previous criterion, completion of a task or event, etc. • Example: 85% of the students will be able to analyze … using the correct statistical procedures.

19. Step 4: Gather the data Assessment data needs to be collected in order to analyze it. Accrediting agencies often will allow sampling, rather than a census, provided the sample is representative and data is provided by program and location. This is one place where products such as Blackboard Outcomes can be very valuable.

20. Step 5: Evaluate, report, and share the data Assessment data needs to be evaluated and the results communicated to others in order for it to inform decisions about programs. • Assessment doesn’t bring improvements in student learning; analysis and use of the results do. • If your assessment data is lying in a corner gathering dust, ask yourself whether the information gathered is useful. If not, figure out why. • Assessment results also need to be communicated to others (faculty, students, stakeholders) who can use them to make decisions.

21. Step 6: Make meaningful changes Assessment is only valuable if the results of our analyses are used to make meaningful changes. • “Closing the loop” simply means using the data to make changes. These changes need not be huge, but they should be meaningful. • Examples of such changes could be new or modified courses, better coordination among courses or sections, modifications in concentrations, curriculum development grants, new course sequencing or prerequisites, opportunities for remedial work, new common assignments to address weaknesses, etc.

22. Assessment Role of Faculty and Adjunct Professors Be knowledgeable about your program’s student learning outcomes. • Know the courses that are selected for assessment activity • If you are the instructor on record for assessment courses, know what measures are being utilized for assessment activity….and USE them. • Ensure completion of the assessment activity • This can be a graded or non-graded assignment • Ensure that the assessment activity was evaluated and documented! • Maintain this documentation; report findings to your assessment point person (program/department chairs).

23. The Assessment Role of Faculty and Adjunct Professors In maintaining your assessment documents, specify the raw number of students who are “satisfactorily” and/or “unsatisfactorily” completing the identified assessments. • Example: 7/10 students scored 80% or higher (satisfactorily) on the diversity project. This information will help to make informed decisions and to enhance program quality. • Troy University HOMER Report Sample.

24. How can Blackboard outcomes help? • Curriculum maps • Tagging of questions or assignments to particular learning objectives • Gathering and sampling of assessment data • Analysis of assessment results • And more! • Timeline for implementation at Troy

25. Questions? Wendy Bailey wcbailey@troy.edu Christina Martin cllmartin@troy.edu Isabelle Warren iwarren@troy.edu Kang Bai bkang@troy.edu Wendy Broyles whuckabee@troy.edu