Becoming a GALIS Data Detective

1. Technical College System of Georgia Office of Adult Education November 13, 2013 Becoming a GALIS Data Detective

2. Technical Housekeeping • On the day of the teleconference, call 1-866-590-5055 and enter access code 8019870# • Please mute your phone line to minimize background noise. • You can access presentation materials on the GALIS “Help” page under Technically Speaking • Technical Difficulties? Email kbryant@tcsg.edu. • When asking questions, please state your name, program, and location. • Stay tuned at the end for a link to an online evaluation form and information about the next ESL Technically Speaking.

3. Teleconference Overview • Introduction – Charita Boles, GPS Coordinator • Presenters • Karen Kirchler – Vice President of Adult Education, West Georgia Technical College • Stephanie Rooks – Dean of Adult Education, Gwinnett Technical College • Brent Stubbs – Dean of Adult Education, Savannah Technical College • Questions of Presenters • Sharing from Others • Closing Remarks – Charita Boles

4. GALIS Data Detective • The data is used to monitor compliance with NRS and state requirements • Keep an open mind when reviewing data • Use the data to make program decision • Engaging staff in data is a powerful motivator

6. Identifying and Providing Support to Low Performing Sites/Instructors • Identifying issues requires that: • you have a definition of what is “low performing” • you understand that “low performing” can be relative to the type of site or students served. • you have a baseline for comparison.

7. Identifying and Providing Support to Low Performing Sites/Instructors • We look at a handful of measures that we find helpful: • Attrition 1 – The number of students lost between the point of intake (orientation/assessment) and the start of classes. • Attrition 2 – The number of students lost between entering class and achieving enrollment status.

8. Identifying and Providing Support to Low Performing Sites/Instructors • We look at a handful of measures that we find helpful: • Retention: The number of students retained to a point of post-test eligibility (30, 40, 60 hours depending on EFL). • Level Completions • Post-test completion rate

9. Identifying and Providing Support to Low Performing Sites/Instructors • Attrition 1- Number of students lost between intake and entering class • Important to look at as a whole, in terms of the program. • Relevant to types of sites or environments • Relevant to groups (ESL, ABE) • Not necessarily helpful at the level of evaluating individual instructors.

10. Identifying and Providing Support to Low Performing Sites/Instructors • Attrition 2 – Students lost between entering class and enrollment. • Important to look at as a whole, in terms of the program. • Relevant to types of sites or environments • Relevant to groups (ESL, ABE) • Extremely important when evaluating instructors!

11. Identifying and Providing Support to Low Performing Sites/Instructors • Retention – Students retained to a point of post-test eligibility. • Important to look at as a whole, in terms of the program. • Relevant to types of sites or environments • Relevant to groups (ESL, ABE) • Extremely important when evaluating instructors!

12. Identifying and Providing Support to Low Performing Sites/Instructors On all other measures we look at how the individual instructor compares to the overall program numbers. We look for outliers. We don’t measure this at the individual instructor level. • Report Cards: This analysis of all instructors is provided to the Lead Teachers and is used to inform their efforts in program improvement.

13. Identifying and Providing Support to Low Performing Sites/Instructors • Report Cards: Ima is an outlier. She’s an exceptional instructor. What to do with Imabesides try and clone her? Have her do some staff development sessions…let new instructors observe her in action…try and identify what she does that works and replicate it!

14. Identifying and Providing Support to Low Performing Sites/Instructors • Report Cards: Nada is also an outlier. We are not enchanted with her performance. However, she is teaching at a community site and there might be some issues with the environment that could affect attrition and retention. But, her post-test level completion rate is way out of range as well. Nada’s continued employment with our agency is in question.

15. Identifying and Providing Support to Low Performing Sites/Instructors • Report Cards: So what’s up with Malcolm? He’s an outlier in many respects. His performance is really good…with the students he doesn’t lose in the first week! Why is his Attrition 2 measure so high? What can we do to mediate this issue and keep more students in Malcolm’s class?

16. Identifying and Providing Support to Low Performing Sites/Instructors • Things to consider: • Rates that seem out of range from the baseline. • What other factors are affecting the outcomes you are seeing? • If the post-test completion rate is out of line, it’s wise to break it out by EFL to try and pinpoint the instructor’s area of weakness. • Report cards with program comparisons are a good way to keep instructors aware that there’s more to the game than just LC rates.

17. Identifying and Providing Support to Low Performing Sites/Instructors • Nuts and Bolts: • While we may give instructors individual report cards, we look at the data in a more global format via spreadsheets and charts. • It requires a little time, and compiling all of these measures is best done only about twice per year. • It’s accomplished most easily if you are familiar with Excel and Access, though most of this can be done with just Excel. • It allows us to look at the program in different ways in different breakdowns.

18. Identifying and Providing Support to Low Performing Sites/Instructors • How we do it: • Export AL137 – All Students and Status by Instructor to Excel (This report is used because we need the last attended date and the CRN for each student’s attendance. Students will be duplicated if they attended more than one class—but it gives a snapshot of teachers’ individual attrition.) • Clean up the data – Paste the instructor’s name into the far right column. This allows for sorting. If you have instructors teaching multiple classes (particularly if they are different types of classes or sites) you’ll need to add an identifier so you can distinguish between the classes. • If you have a large program and you have Lead Teachers supporting part-timers, you may find some benefit in creating an Access table with all of the identifying info you wish to attach to the class or CRN (Lead, location, etc.) You can export the Excel sheet into Access and it creates a little more flexibility in manipulating the data.

19. Identifying and Providing Support to Low Performing Sites/Instructors • How we do it: • Sort and filter the spreadsheet: • For “Attrition 1” filter out any who have more than 3.5 hours. The group that remains gets divided by the total # of students. • For “Attrition 2” eliminate those with less than 3.5 hours. Filter the group that remains to determine the number with less than 12 hours. Divide this by the total # of students. • For “Retention” eliminate those with less than 12 hours. Sort by EFL and then filter to appropriate hour level = to post-test eligibility. Divide these numbers by the total # of students. • Export and marry the Table 4 and 4b rates to your spreadsheet by instructor. (For instructors teaching at multiple sites or different types of classes, this requires more manipulation.)

20. Identifying and Providing Support to Low Performing Sites/Instructors • Example Instructor A: Total number of students in AL 137 = 35 Total number of students in Table 4 = 25 Attrition 1 - Students with less than 3.5 hrs = 4(Eliminate – they never came to class) Attrition 2 - Students with less than 12 hours = 6 (6/31 =19%)(We eliminated 4 from the original 35 to get our base of 31 students.) Retention - Students who make it from enrollment to post-test eligibility = 14 (14/25 = 56%) (Use your enrollment number as your base of students.)

22. Retention: The number or % of students who remain long enough to be post-tested and make a gain. Money Gains Retention Hours

23. Breaking It Down… • Many small programs-Not one large one • 12 sites • 15 ABE & ESL programs • 203 classes • Retention by Lead Instructor • Retention by site • Retention by class/instructor The Best Tool?

24. ….Classroom Sign-in Sheet • Submitted and entered into GALIS weekly • Wednesdays-each sheet reviewed and % attended/absent recorded • Broken down by: Each Lead Instructors’ sites by class • Spreadsheets sent to Leads – any class <70% attendance is flagged

26. Instructor • Leads – Should know about every class • Confer with instructors • Review attendance • Observe class • SEP followed? • Students called? • Follow-Up Form Completed?

27. Retention OutcomeNumber and Percent Retained

29. Analyzing Your Programwith Data • Start with the end in the mind – create processes that match your goals • Set data benchmarks for attrition and enrollment • If you look at it daily, it must be entered daily • Contact Hours Tell the Story

30. Backward Engineering • Start with the end in mind • Our goals: 2,700 enrolled, 60% completions, 90% GED completion, 10% attrition • Do my processes match these goals? Am enrolling enough new students per month to reach my enrollment goal? • At 80% effectiveness, am I giving enough post-tests per month? • Result: Targeted Recruitment

31. Attrition & Enrollment • We set monthly enrollment goals in: • NRS Enrollment (A) • State Enrollment (B) • “In GALIS” (C) • AL077 report can be manipulated to tell the story of attrition • Attrition = C-B/C and B-A/A

32. Attrition is Important • C-B/C could mean • We are entering students into GALIS we should not be (wasting time) • We are not capturing attendance for some students (process or responsibility is undefined) • B/A-A could mean • Our success at retaining students • A breakdown in the linkage between Orientation and students starting class

33. Is Your Data Good Daily? • Attendance is entered daily • If you need to make real-time decisions, you must have real-time information • 24 hour completion entry goal • How can you make decisions about enrollment, teacher effectiveness, post-testing, etc., if you do not have real-time information?

34. Being in front of students • Contact hours tells the entire story • I budget for contact hours • We are on track to increase our contact hours by 30% this year. • Why? Students in class, post tests happen, more opportunities for student success

35. Being in front of students • Over the last 3 years, what was your average contact hour per student? • Ours: 66.8 • Our goal: 90 • Which teacher had the most contact with students? • Did it correlate to better success? • Students vote with their feet.

36. Questions for the Presenters ? Please say your name, program and location before asking your question

37. Sharing of Ideas from other Adult Education Professionals

38. Contact Information

39. Thank you for your participation! Please complete an evaluation of this session at http://surveymonkey.com/s/TechnicallySpeakingDetective Next Program Administration Technically Speaking: Getting to Know your Staff through Visitation and Monitoring January 22, 2014 2:00-3:30 pm