Using Data Effectively

1. Using Data Effectively ABE Directors’ Meeting October 9th and 10th, 2002

2. AGENDA • Current Use of Data • What Types of Data Can Be Used • Improving Your Program: Current Practices • A Model for Program Improvement • Case Study • Next Steps: Develop a Program Improvement Plan • Evaluation

3. Current Use of Data • What two statistics (e.g., attendance hours, enrollment rates) do you use frequently or would you like to use?

4. What Types of Data Can Be Used • Learner Data - Demographic / background characteristics - Labor force status - Educational gains - Education and employment outcomes - Class participation - Community participation - Family outcomes - Learners’ perceptions of instructional services

5. Staff Data - Background of instructional staff - Staffs’ employment and training experience - Characteristics of instructional approaches

6. Program Operations - Recruitment - Learner intake - Learner placement - Structure and hours of instruction - Assessment instruments and procedures - Support services - Learner transition - Fiscal management - Program management - Interagency collaboration - Data management

7. Improving Your Program: Current Practices • What internal and external priorities have prompted changes or improvements? • What types of data do you collect? • How do you use these data? • What actions have you taken, or what changes have you made based on the analysis of data?

8. A Model for Program Improvement • Analyze current data and determine questions to ask • Identify areas for improvement • Design a program improvement plan • Document and monitor use of the new practice • Evaluate implementation and outcomes

9. Analyze Current Data and Determine Questions to Ask • Identify trends in data that raise questions • Analyze data related to an issue or question that has prompted interest • Identify and review additional information that can be examined to confirm or clarify results of data analysis • Define preliminary issues or problems that need to be addressed

10. Identify Areas for Improvement • Use benchmarks to assess extent of problem and target population • Determine area for change based on importance of problem and other factors • Create team to identify new practice / process for target population; promote effort • Access information about practices from online systems, websites, reports, other programs/states

11. Design a Program Improvement Plan • Determine new practice • Specify learners, classes, or sites for pilot • Develop communication networks / new policies • Identify training and technical assistance • Determine resources / staff • Set timeline for training and pilot testing the practice • Specify data collection • Review the change process

12. Document and Monitor Use of New Practice • Collect information about implementation of practice • Specify materials, staff, and resources • Chart time for implementing • Identify barriers to use • Identify aspects of practice or use that appear to be successful • Identify factors affecting use • Chart unexpected outcomes

13. Evaluate Implementation and Outcomes • Analyze data to identify elements of the practice implemented and not implemented • Assess what worked, what didn’t, and why • Identify missing elements in practice • Describe outcomes from using practice • Refine practice and its use

14. Case Study Funder: Local Education Agency (LEA) Services Funded: ABE, ESOL, Family Literacy, Workplace Sites: Five sites - three sites provide ABE and ESOL services, one site provides Even Start, and the other site provides workplace literacy Total Enrollments: 524 * approximately 50% of the students are ESOL - of those 30% are Brazilian * approximately 10% of the ABE students entered at the beginning literacy level, 20% were at the ABE/Pre ASE level, and 20% were at the ASE level Assessment to Measure Educational Gain: * For ESOL students, last year the program used a program-developed tool with a rubric developed by one ESOL teacher. This year the program will use BEST and REEP. * For Beginning Literacy ABE students, last year the program used a home grown tool translated into Spanish. They will continue to use that tool this year. * For ABE, Pre ASE, and ASE students, last year the program used TABE 5 Level M for all students. This year the program will use TABE 7 and 8. Assessments for Class Placement: * For ESOL students, the program uses a program-developed instrument which includes an oral and writing section. * For all ABE students, the program uses the TABE Locator.

15. Instructional Methods: The delivery of instruction has not been an emphasis in professional development in recent years. Each teacher uses methods that he/she feels comfortable with. There is no formal curriculum in place and the program does not use the Curriculum Frameworks. The ASE classes are totally individualized and student attendance is sporadic. The ESOL attendance is high yet the % completing a level is low. On the Federal Report Table 4, the overall percent for completing the educational levels is 20.56% for one site and Cognos shows that the overall percent for class attendance for one of the sites is 54%. Program and Staff Development: The program development is determined by the Program Coordinator who assigns hours equally to all staff members. Staff development is decided by each staff person with no plan in place. Counseling: Counseling hours are allotted to each ESOL teacher, and the Program Coordinator provides counseling to ABE/ASE students. Staffing: All staff are white. One teacher is bilingual in Spanish. All staff are part time and are paid $20/hour with no benefits. They are not compensated for prep time. They are allotted additional paid hours for assessment and follow-up.

16. Next Steps: Develop a Program Improvement Plan • Analyze current data and determine questions to ask • Identify areas for improvement • Design a program improvement plan • Document and monitor use of the new practice • Evaluate implementation and outcomes