National Postschool Outcomes Center Florida’s Experience in Accessing Administrative Records

1. National Postschool Outcomes Center Florida’s Experience in Accessing Administrative Records Teleconference May 24, 2005 Jay Pfeiffer, Assistant Deputy Commissioner Division of Accountability, Research, and Measurement Florida Department of Education 850-245-0437 Jay.Pfeiffer@FLDOE.org

2. K-20 Governance Organization: Alignment of Data Systems Office of K-20 Education Information & Accountability Additional role: Department-wide conduit for LMS, LED, Census, BEBR publications Source Information Systems Integrating Systems Community College & Technical Center K-20 Education Data Warehouse Public Schools University FETPIP K-12 Employment Earnings Postsecondary Ed. Welfare Incarceration Licensure Adult Education Career/Technical Associate Degree Baccalaureate Masters Ph.D./Professional Source Systems Employment Assessment Financial Aid Teacher Certification Facilities Annual Administration & Accountability Continuous/Longitudinal

3. FETPIP – The Florida Education and Training Placement Information Program

4. “Instead of maintaining a separate program to collect industry occupational information, it would be more efficient in terms of resources expended to merge the data collection phase of the Occupational Employment Statistics (OES) program with the ES 202 program.” … Florida’s Occupational Information and Delivery System, Technical Assistance Project, U.S. Department of Commerce, Project Number 04-06-01109, May 1975: Tallahassee Florida • “The Unemployment Insurance Wage Report represents the best potential comprehensive source of occupational and wage information in the state, with the addition of occupational titles to the report.” This would be useful in: “labor market demand forecasting and analysis …economic analysis…evaluation and follow-up…” • Polivka, Larry; et al. Vocational Education Study: Labor Market Demand Information in Florida: Policy Issues and Alternatives Office of the Governor, August 1982: Tallahassee Florida Key Background Studies and Recommendations • “Priority should be given to examining the addition of occupational information to the UI Wage Report file.” • Herndon, John, et al. Project to Coordinate Economic Development, Employment and Training: Report 1 Issues and Recommendations, Report to the Governor 1 of the EDIT Project, March 1982: Tallahassee Florida • Regarding the wage record system: • “The addition of a single data element, an occupational title, would provide the possibility for the most useful single data base in the State of Florida.” • The Florida Occupational Information Coordinating Committee “A suggested approach to developing a comprehensive occupational information data base for Florida from existing data resource.” A concept paper, Florida Department of Labor and Employment Security, • June 1981: Tallahassee Florida

5. “Funds provided in Specific Appropriations 337A shall be used to conduct a study of the feasibility of implementing an automated system that matches social security numbers of persons completing vocational programs with wage, geographic, and occupational title data available through the Unemployment Insurance Wage Report and the Worker’s Compensation system.” Chapter 84-220 Laws of Florida, Specific Appropriation 337A: Tallahassee Florida Initial Legislative Direction • Specific Appropriation 337A established two important precedents: • Administrative records might have uses other than that which they were originally designed for; and, • Education money went to the DLES under the tutelage of the FCOVE and the SJTCC with the DOE reporting back results;

6. DEVELOPMENTS LEADING TO FETPIP'S CURRENT STATUS

7. Florida Education and Training Placement Information Program (FETPIP) Annual Collection Cycle February, 2003 FL UC Covered Employment High School Performance and Accountability Reports Fed Government & Postal Service Vocational Center Uniformed Military Service Community College Workforce Funding Formula Out-of-State Employment Public University Out-of-State Postsecondary Ed. Independent University Ad Hoc Reports Merge Files Unduplicate Edit Names & SSN System Exits Public University Enrollment Other Programs Public Adult Ed. & Vocational Enrollment “CHOICES” Career Counseling Software Federal Programs Private College & University Enrollment Create Master Follow-up File Welfare Transition Services Public Community College Enrollment FL Corrections Releases Internet Education & WIA Reports Create Tapes & Files for Record Links FL Corrections Incarcerations Special Projects, Longitudes Public Assistance Receipt Inputs Linked Files Outputs

8. Input Data Examples • Student/Participant - demographic, socio-economic, enrollment/services, educational programs, promotion, attendance, GPA, test scores… • Educational Institution - types, location, performance statistics, private/public… • Financial Aid - federal/state/local, loans/grants, amounts disbursed… • Courses - offerings, student & teacher participation, instruction type, grades… • Educational Staff - demographic, awards, certification, teaching & work activity • Educational Awards - type, program, institution, date

9. Regular Publication of Annual Performance Accountability Data Annual performance data include employment rates, earnings, full-time employment proxies, continuing postsecondary education statistics by educational sector, public assistance participation, and incarceration…

10. Most High School Grads Nation-wide Go On To Postsecondary Within 2 Years Source: NELS: 88, Second (1992) and Third (1994) Follow up; in, USDOE, NCES, “Access to Postsecondary Education for the 1992 High School Graduates”, 1998, Table 2.

12. Regular Publication of Longitudinal Information for Select Cohorts 1991 Public High School Graduates in 2002 – Standard Diplomas Information may be aggregated over long periods of time to show longer term results of continuing to pursue educational opportunities, employment, earnings, and interactions of various factors… Source: FETPIP Longitudinal *

13. Florida Public High School Class of 1991 Earnings by Educational Credential in 2002 Source: FETPIP Longitudinal

14. FETPIP Data Major Uses • Public Education Funding • Performance-Based Budgeting • Federal Performance Measurement: WIA, Wagner-Peyser, Voc Rehab, Blind Services, Perkins, NRS (Adult Education) • Private School Licensure • School District Accountability Reports • Community College Performance Reports • University System Program Reviews • ICUF Accountability Reporting • Legislative Performance Reviews • Workforce Estimating Conference • Policy Research • Career Counseling, Consumer and Guidance Information

15. FETPIP-Type Operations Benefits • Use existing data resources • Cheaper, more accurate, more complete, and with more capabilities, than traditional surveys • Data are regarded as objective- non-judgmental • Provide a basis for common core sets of outcome measures, comparisons, discussions of issues across many programs & operations

16. With such a system, what can you expect to hear? Part 1 Wow! You can do that? If I provide you with a set of student data can you…? How much will this cost me? Why should I participate in this?

17. With such a system, what can you expect to hear? Part 2 The System: • is too slow • is not done frequently enough • doesn’t cover students without correct SSNs • misses types of employment • misses types of postsecondary education • doesn’t cover the other, more important outcomes

18. Challenges (part 1) • Administrative data are designed for purposes other than performance measurement • Definitions, taxonomies may differ • Time periods may differ • What is measured may not be exactly what is desired • Administrative data change over time • Owners of Administrative Data have legal, ethical, proprietary obligations • They won’t naturally want to share • “Legal” beliefs may be conjured up • Program interests may be different from those of technical staff • The “owners” change over time

19. Challenges (part 2) • Confidentiality of Information • Requirements may be different for each data source • The issue is complicated by legal issues as well as beliefs • Beliefs/requirements may change over time • Weakest Links • The strength of a system of linked data sets is determined by the weakest of the data resources • Opponents will focus on the weaknesses • The data lag actual performance • Support • On going, reliable funding • On going, reliable political support • Cooperative attitude among stake holders • All involved will change

20. What has worked for FETPIP? Support • Statutory authority, provisions, including cross references • Core funding from State General Revenue appropriated by the Legislature • Persistently collaborate with legislative, executive staff • Persistently collaborate with data owners, stakeholders in decisions about the mechanics of exchanges, reporting of results • Constantly anticipate roles that the system can play in accountability, performance measurement, cost return analysis, evaluation • Consistently work with data owners on changes, data anomalies, problems • Steadfastly make it easy for stake holders to participate, reduce burdens, provide data in exchange • Always facilitate discussions about common data elements, definitions, core performance measures, performance funding

21. What has worked for FETPIP? Confidentiality/Security • Location within the state education agency • Always surpass all requirements, expectations regarding data security • Faithfully limit uses to statistical purposes for evaluation, program improvement • Persistently keep exchange agreements current and up-to-date, continually brief employees, stake holders on security and confidentiality issues • Always refer inquiries about particular resources to the resource owners

22. What has worked for FETPIP? Changes • Strive to meet changes, challenges head-on • Persistently refresh understanding of resources, requirements, limitations • Persistently seek new data resources, expand coverage • Always revise business rules as understanding, conditions change • Establish short term and long term visions for the effort

23. What has worked for FETPIP? Other • Establish archival data resources • Accommodate ad hoc interests and requirements • Seek out opportunities to demonstrate your wares • Always collaborate

24. K20 EDW – Florida’s K20 Education Data Warehouse

25. Employee Identity Educational Curriculum Employee Demographic Includes: Certified Staff Teachers Ÿ Faculty Members Ÿ Educational Support Staff Ÿ Institutions Instructional activity Course District Includes: Public Schools Ÿ Educational Facilities Colleges Ÿ Institution's Universities Ÿ Finances Private Ÿ Institutions Active Student Financial Assistance Student Student's includes: includes: Institution's Award Courses All enrollments Ÿ Federal Aid Ÿ Revenues Acceleration Ÿ State Aid Ÿ Attendance Ÿ Local Aid Ÿ Discipline Ÿ Bright Future Ÿ Exceptionality Program Ÿ Chart of Accounts Student Demographic Student (Ex-)student Includes Student's Student's Test Scores Employment SUS Application Ÿ Identity Identity Talented 20 Ÿ K-20 Budget Florida’s K20 Education Data Warehouse

26. K20 Data Warehouse: Student-Centered Demography Educational Awards Educational Institution Educational Staff Financial Aid Student Student Employment Courses Grades, Standardized Test Scores