Identifying At-risk Students

1. Identifying At-risk Students Presented by Ryan Kuhne Educational Technology Consultant Carleton University in Ottawa, Canada

2. Agenda What is an “At Risk” student? • What does “At Risk” mean for your course? • How do we identify these Students? • Grade related data • Engagement related data • What can we can do to help? • Now • For Future Courses carleton.ca/edc/atrisk

3. Why do this? • It will help students in need! • It will help keep students engaged! • It will help you think about your course design and assessments! • It will give you insight into how your students are interacting with the online portion of your course! • It will potentially improve your teaching review scores! • It will potentially reduce you DFW rates! Retention • It's really easy and does not take much time!

4. What is an "At Risk" student? "At Risk" simply refers to a student in danger of under-performing academically. At Risk

7. Academic Analytics InteractionsStudent – Content Student – Instructor Student – Student Student – Interface

8. SIS data (historic) CMS and other technologies (real time) Other data Chart from Using Analytics to Intervene with Underperforming College StudentsEDUCAUSE Learning Initiative, Annual ConferenceKunnen, Eric J. and John Fritz 2010 Prediction Specific and customizable interventions

9. Retention – Institutional • Signals by Perdue and Sungardsungardhe.com/signalson NBC Nightly News story • Starfish Retention Solutionsstarfishsolutions.comAlabama State, Seton Hall, U of Chicago, Cincinnati, Pittsburgh…Bb World 08 - http://www.viddler.com/explore/starfish/videos/1/ • Less than sixty percent of undergraduates now graduate within six years. • The result is lost revenue, lost potential, and lost dreams. • Increasingly, campuses are being held accountable for their graduation rates…and their drop-out rates. What Faculty Members Need to Know About Retention: A Magna White Paper

10. Starfish • Identify, Track, Retain - Engage, Motivate, Graduate • Connects students, instructors, counselors and data Chart from Starfish Corporate OvervieW.pdf

11. Categorizing "At Risk" students? • High-risk (Red)studentsMedium-risk (Yellow) students • Low-risk (Green)students Predictive modeling and historical data suggests that students who are more engaged and have higher marks early on in the semester will continue to be high achieving (Green) and unengaged students with low or missing grades early in the semester are much more likely to end up "At Risk" (Red).

12. High-risk (Red) Students Traits • Frequently absent from class • Rare-low WebCT use • Participation (in-class/online) is non existent or extremely low • Consistent low or failing grades • Work not handed in or of poor quality • At risk of failing course (cumulative grade of D+ or lower) Photo by otubo

13. Medium-risk (Yellow) Students Traits • Irregular attendance patterns • Low-average WebCT use • Participation (in-class/online) is low-average  • Consistently low or erratic grades • Work is below average quality • Minor risk of failing (cumulative grade between C+ and C-) Photo by Latente

14. Low-risk (Green) Students Traits • Regularly attends classes • Average-high WebCT use • Participation (in-class/online) is average-high  • Consistent above average or high grades • Work is above average quality • No visible risk of failure (cumulative grade of B- or higher) Photo by włodi

15. What is an "At Risk" student?

16. So what can instructors do? It’s as easy as 1, 2, 3 • What does “At Risk” mean for your course? • How do we identify these Students? • What can you do to help?

17. 1. What does "At Risk” mean for your course? • What do you consider a failing grade for this course? Institutional/Program • For the course, does at risk mean that the student is presently failing, or close to failing. • Will the student have enough opportunity to improve their grade after our initial contact? • What support are you willing to provide?

18. 2. How do we identify these Students? • Grade Related Data • Engagement Data Grades Assessments Assignments Participation Engagement Data Time online First Access # Online Sessions Discussions read/posted Mail Files Viewed MyGrades views

19. 2. How do we identify these Students?Grade Related Data • What assessments are done to date? • Are there any lab or tutorial grades? • Do you have any participation or attendance grades? Sort the gradebook columns to see who is doing poorly. Set up a couple course check points (calculated column). • Figure out the formula for final course grade if the class were to end today. We will look at an example in a few slides.

20. 2. How do we identify these Students?Engagement Data

21. Engagement DataTracking / Metrics / Analytics You are able to export the report data and import it back to the gradebook to use in identifying At Risk students.

23. Engagement Data • The New Diagnostics - Oct. 30, 2009 Inside Higher EdRio Salado Collage –Have devised a predictive modeling system that they believe to be 70% accurate afterh the first week of class • Early identification of at-risk students using a personal response systemThe order in which students registered their devices was an accurate predictor of final student grades. (Griff & Matter 2008) • "Hits" (not "Discussion Posts") predict student success in online coursesGeneral LMS activity can be used to identify at risk students, as early as two weeks from the beginning of the course. Research with a broader range of course subjects and a larger number of courses is needed to substantiate a general claim. (Ramos & Yudko 2008) • Mining LMS data to develop an “early warning system” for educatorsThe number of forum postings, mail sent, and assessments completed function as significantly predictive variables of student final grades. Findings suggest that for the purposes of monitoring student activity and achievement, predictive models must be developed at the course level.(Macfadyen & Dawson 2009)

24. SNAPPSocial Networks Adapting Pedagogical Practice • research.uow.edu.au/learningnetworks/seeing/snapp

25. 3. What can we do to help? Olive branch

26. 3. Helping Your Current Courses Hands Off • Use selective release based on grades to show at risk students extra resources in the course. • Over communicate dates, deadlines and resources.5 tips to over communicating without seeming obvious in targeting at-risk students • Encourage students to check the course more frequently. • Put links to tutoring and other support services in the course. Hands On • Email the students to let them know how they are doing in the course compared to their classmates (offer help).  • Make yourself  available for questions and extra help. • Meet with the students to talk about why they are not engaged with the course.5 goals you should strive for in your meeting.

28. 3. Helping Your Future Courses • Design your course to have assessment spread out through the term and early on in the term so it is easier for students to track their progress and have a chance to correct. • Build in course check points at several points throughout the term and make students aware of them. • Over communicate – Place info/links in syllabus and online • Your office hours • Your availability to meet • Discipline specific resources • Where to get help on campus • Setup peer study sessions

29. EX 1 A 2nd Year Psychology course • What does At Risk mean for your course? Below 60% / D+ or lower • Identify the At Risk studentsMake a Check Point calculated column in the Gradebook The final grade is calculated as such: 40 % - 12 Weekly Tests (best 10 out of 12) 20 % - 4 Assignments 40 % - Final Exam 5 Quizzes (take best 4 of 5) and 1 Assignment are completedSeparate students into groupsNothing handed in, Currently failing (below 50%), D students (between 50%-60%)See sample image on next slide • Contact the students offering helpSee sample emails

30. EX 1 A 2nd Year Psychology course

31. EX 1 A 2nd Year Psychology course

32. Nothing Handed in Dear Student, I am very concerned about your lack of participation in Course Name. To date, you have not handed in any of the course assignments or taken any of the quizzes. As a result you are currently failing the course and have lost ??% of the term marks available to you. It's important that you take an active role in your own success and if you need help, please do not be afraid to ask.The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions(PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.You can of course visit me during my office hours, (Date and Time of Office Hours or make an appointment if you have a conflict) and we can discuss how to improve your performance.If you attempted to drop this course earlier in the term, it's important that you follow-up on Carleton Central to ensure this course no longer appears in your schedule because you are still registered in this course.Please consider taking an active role in improving your academic performance. Let me know if there is anything I can do to help you. Best,

33. Have below an F or below 50% Dear Student,I am concerned that you may be experiencing difficulty in Course Name, as you are currently failing the course with an average of less than 50% on your course assessments. It's important that you take an active role in your own success and if you need extra help, do not be afraid to ask. There is still time for you to improve your grades, and by doing so, I know you will succeed.The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions (PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.You can of course visit me during my office hours, (Date and Time of Office Hours or make an appointment if you have a conflict) and we can discuss how to improve your performance.Please consider taking an active role in improving your academic performance. Let me know if there is anything I can do to help you.Best,

34. Have D- to D+ or 50%-60% Dear Student,I am concerned that you may be experiencing difficulty in Course Name, as you are currently in the D grade range with an average between 50% and 60% on your course assessments. It's important that you take an active role in your own success and if you need extra help, do not be afraid to ask. There is still time for you to improve your grades, and by doing so, I know you will succeed. The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions (PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.You can of course visit me during my office hours, (Date and Time of Office Hours or make an appointment if you have a conflict) and we can discuss how to improve your performance.Please consider taking an active role in improving your academic performance. Let me know if there is anything I can do to help you.Best,

35. Sample Email Dear Student, I are concerned that you may be experiencing difficulty in this course. I want you to succeed and there is still lots of time in the semester to improve your grade by doing the following things: • Attend or watch all tutorials. Attempt the problems yourself. • Read the relevant parts of the text as they are assigned. • Do all the problems as they are assigned.  • When you realize that there is something you do not understand, get help ASAP – from a friend, a TA, or from me. • Attend PASS sessions if you can schedule them. These are known to improve students’ grades dramatically. • Come and see me during Office hours ###### I know that university can be very challenging and in my experience, students seeking support at this early point in the term have a much better chance of successfully completing their courses. There is help available and the following resources can get you back on track; • At the Student Academic Success Centre you can receive help in the following areas: • a) Academic Improvement Planning, a step-by-step guide to put you on the pathway of improving your academic performance.b) Learning Support Services, which include workshops on study skills, writing tutorials, group study space, tutor referrals and more.c) Peer Assisted Study Sessions, a study group program that's focused on this course. Students meet together to go over concepts, course material, etc.d) Writing Tutorial Services provides assistance in the area of academic writing.e) Advising Centre is available for you to go over the requirements for their major, make changes and get support. • The Ecumenical Chaplaincy is here to provide religious support, counselling, one-on-one support, study groups, meditation and more. • The Health and Counselling Service is a multidisciplinary service providing medical services, counselling services, and health promotion. Do not give up! You can go on to do well in Course Name if you put in the effort. Best,

36. EX 2 A 2nd Year Environmental Science course • What does At Risk mean for your course? Below 60% / D+ or lowerThe final grade is calculated as such:25% - Midterm 50% - Final Exam 15% - Assignment 10% - Online Quizzes (Best 4 out of 5) • Identify the At Risk students Make a Check Point calculated column in the GradebookAfter first 3 weeks 2 Quizzes have been completed Use tracking data to see who has not logged in to the course site yetSeparate students into groupsNothing handed in, Mix of Poor Grades and Missing Assessments (between 20%-60%)See sample image on next slide • Contact the students offering helpSee sample emails

38. Take aways…. You have the power to change the life of a student The New Diagnostics - Oct. 30, 2009 Inside Higher Ed • “at-risk” students generally took that information as either a motivational kick in the rear or were prompted to quickly drop the class -- and were grateful in any case.” • “A double-blind study conducted during the first two years of the Purdue's program, called Signals, revealed that 67 percent of students who learned they were in the middle- or high-risk categories were able to improve their grades."

39. Please provide feedback for this session by emailingBbWorldFeedback@blackboard.com. The title of this session is: Identifying At-risk Students carleton.ca/edc/atrisk Presented by Ryan Kuhne ryan_kuhne@carleton.ca Educational Technology Consultant Carleton University in Ottawa, Canada

40. 5 tips to over communicate, without seeming obvious in targeting at-risk students • Re-establish your/TA office hours. Send out an email to all students as a reminder. • In preparation for midterm or final exams, invite the students to submit topics for you to emphasize. Let them help you create the review day that focuses on their needs. • Consider incorporating technology into your teaching style. Start a blog, or a wiki as a way to interact with students outside of the classroom in a medium that they know best. Contact the Educational Development Centre for tips and one-on-one training on technologies. • Offer optional homework or practice exam questions that will help students get an understanding of what to expect. • Consider allowing students a re-write option for one assignment in an effort to improve a grade, using your feedback as a guideline. Most students, regardless of standing, would appreciate a second crack at it.

41. 5 goals you should strive for in this meeting • Establish yourself as someone who not only cares about their success, but is available to help them along the way. • Try to identify the source of the problem. Is the student struggling to comprehend material, or is it a problem outside of the classroom? • Have the student work out a plan of action. Don't force something down their throat, let them create a plan that is feasible and realistic to them. Help by   providing perspective on what assignments are left, their weight, expectations, etc.This is a great tool to help students set some goals based on their grades. Grading Tool. • Be informative. Provide them with some referrals to other areas around campus that may be able to help. Counselling? Writing services? Mentoring? Find out how Carleton can help them. • Remind them that you're here. Encourage them to come back or contact you whenever they need help with course material. Show them a hand that they can reach out for if they need (metaphorically speaking...)