Integrated Performance Information Systems: Florida’s Experience Phoenix, Arizona June 14-16, 2005

1. Integrated Performance Information Systems: Florida’s Experience Phoenix, Arizona June 14-16, 2005 Jay Pfeiffer, Assistant Deputy Commissioner Division of Accountability, Research, and Measurement Florida Department of Education 850-245-0437 Jay.Pfeiffer@FLDOE.org

2. Florida: The Home of Integrated Performance Bonnie Ivan Jeanne Charley Francis Source: Florida K20 Education Data Warehouse

3. K-20 Governance Organization: Alignment of Data Systems Office of K-20 Education Information & Accountability Source Information Systems Integrating Systems Additional role: Department-wide conduit for LMS, LED, Census, BEBR publications Community College & Technical Center K-20 Education Data Warehouse Public Schools University FETPIP K-12 Employment Earnings Postsecondary Ed. Welfare Incarceration Licensure Adult Education Career/Technical Associate Degree Baccalaureate Masters Ph.D./Professional Source Systems Employment Assessment Financial Aid Teacher Certification Facilities Annual Administration & Accountability Continuous/Longitudinal

4. FETPIP – The Florida Education and Training Placement Information Program

5. Florida Education and Training Placement Information Program (FETPIP) Collection Cycle June, 2005 FL UC Covered Employment High School Performance and Accountability Reports Fed Government & Postal Service Vocational Center Uniformed Military Service Community College Workforce Funding Formula Out-of-State Employment Public University Out-of-State Postsecondary Ed. Independent University Ad Hoc Reports Merge Files Unduplicate Edit Names & SSN System Exits Public University Enrollment Other Programs Public Adult Ed. & Vocational Enrollment “CHOICES” Career Counseling Software Federal Programs Private College & University Enrollment Create Master Follow-up File Welfare Transition Services Public Community College Enrollment FL Corrections Releases Internet Education & WIA Reports Create Tapes & Files for Record Links FL Corrections Incarcerations Special Projects, Longitudes Public Assistance Receipt Inputs Linked Files Outputs

6. Input Data Examples • Student/Participant - demographic, socio-economic, enrollment/services, educational programs, promotion, attendance, GPA, test scores… • Educational Institution - types, location, performance statistics, private/public… • Financial Aid - federal/state/local, loans/grants, amounts disbursed… • Courses - offerings, student & teacher participation, instruction type, grades… • Educational Staff - demographic, awards, certification, teaching & work activity • Educational Awards - type, program, institution, date

7. Starting an Integrated Performance Information System June, 2005 State Performance and Accountability Reports High School Workforce Reporting State UC - Covered Employment Adult Education Fed Government & Postal Service . Vocational Center Ad Hoc Reports Merge Files Unduplicate Edit Names & SSN System Exits Uniformed Military Service Community College Career Counseling Software Postsecondary Enrollments WIA Participants Create Master Follow-up File Public Assistance Receipt Welfare Transition Services WIA Reports Create Tapes & Files for Record Links Inputs Linked Files Outputs

8. FETPIP Data Major Uses • Public Education Funding • Performance-Based Budgeting • Federal Performance Measurement: WIA, Wagner-Peyser, Voc Rehab, Blind Services, Perkins, NRS (Adult Education) • Private School Licensure • School District Accountability Reports • Community College Performance Reports • University System Program Reviews • ICUF Accountability Reporting • Legislative Performance Reviews • Workforce Estimating Conference • Policy Research • Career Counseling, Consumer and Guidance Information

9. FETPIP-Type Operations Benefits • Use existing data resources • Cheaper, more accurate, more complete, and with more capabilities, than traditional surveys • Data are regarded as objective- non-judgmental • Provide a basis for common core sets of outcome measures, comparisons, discussions of issues across many programs & operations

10. Challenges (part 1) • Administrative data are designed for purposes other than performance measurement • Definitions, taxonomies may differ • Time periods may differ • What is measured may not be exactly what is desired • Administrative data change over time • Owners of Administrative Data have legal, ethical, proprietary obligations • They won’t naturally want to share • “Legal” beliefs may be conjured up • Program interests may be different from those of technical staff • The “owners” change over time

11. Challenges (part 2) • Confidentiality of Information • Requirements may be different for each data source • The issue is complicated by legal issues as well as beliefs • Beliefs/requirements may change over time • Weakest Links • The strength of a system of linked data sets is determined by the weakest of the data resources • Opponents will focus on the weaknesses • The data lag actual performance • Support • On going, reliable funding • On going, reliable political support • Cooperative attitude among stake holders • All involved will change

12. What has worked for FETPIP? Support • Statutory authority, provisions, including cross references • Core funding from State General Revenue appropriated by the Legislature • Persistently collaborate with legislative, executive staff • Persistently collaborate with data owners, stakeholders in decisions about the mechanics of exchanges, reporting of results • Constantly anticipate roles that the system can play in accountability, performance measurement, cost return analysis, evaluation • Consistently work with data owners on changes, data anomalies, problems • Steadfastly make it easy for stake holders to participate, reduce burdens, provide data in exchange • Always facilitate discussions about common data elements, definitions, core performance measures, performance funding

13. What has worked for FETPIP? Confidentiality/Security • Location within the state education agency • Always surpass all requirements, expectations regarding data security • Faithfully limit uses to statistical purposes for evaluation, program improvement • Persistently keep exchange agreements current and up-to-date, continually brief employees, stake holders on security and confidentiality issues • Always refer inquiries about particular resources to the resource owners

14. What has worked for FETPIP? Changes • Strive to meet changes, challenges head-on • Persistently refresh understanding of resources, requirements, limitations • Persistently seek new data resources, expand coverage • Always revise business rules as understanding, conditions change • Establish short term and long term visions for the effort

15. What has worked for FETPIP? Other • Establish archival data resources • Accommodate ad hoc interests and requirements • Seek out opportunities to demonstrate your wares • Always collaborate

16. – K20 EDW – Florida’s K20 Education Data Warehouse

17. Employee Identity Educational Curriculum Employee Demographic Includes: Certified Staff Teachers Ÿ Faculty Members Ÿ Educational Support Staff Ÿ Institutions Instructional activity Course District Includes: Public Schools Ÿ Educational Facilities Colleges Ÿ Institution's Universities Ÿ Finances Private Ÿ Institutions Active Student Financial Assistance Student Student's includes: includes: Institution's Award Courses All enrollments Ÿ Federal Aid Ÿ Revenues Acceleration Ÿ State Aid Ÿ Attendance Ÿ Local Aid Ÿ Discipline Ÿ Bright Future Ÿ Exceptionality Program Ÿ Chart of Accounts Student Demographic Student (Ex-)student Includes Student's Student's Test Scores Employment SUS Application Ÿ Identity Identity Talented 20 Ÿ K-20 Budget Florida’s K20 Education Data Warehouse

18. K20 Data Warehouse: Student-Centered Demography Educational Awards Educational Institution Educational Staff Financial Aid Student Student Employment Courses Grades, Standardized Test Scores

19. Matching Individuals in Florida’s Education Data Warehouse: Sophisticating the Matches, Anonymizing Data