Overview and Discussion of NRS Changes

1. Overview and Discussion of NRS Changes

2. NRS Changes for FY13 AGENDA Review changes on follow-up outcomes and new data requirements Review criteria for cohorts Review changes to reporting timeline and reporting tables Review questions and answers posted on the ACLS website Respond to participant questions Massachusetts Department of Elementary and Secondary Education

3. Time Period Definitions Program Year (PY)– Twelve month period from July 1 through June 30 defined by the calendar year at start of period. (federal terminology) For example, July 1, 2012 through June 30, 2013 is the time period corresponding to PY12. Fiscal Year (FY)– Twelve month period from July 1 through June 30 defined by the calendar year at end of period. (state terminology) For example, July 1, 2012 through June 30, 2013 is the time period corresponding to FY13. Massachusetts Department of Elementary and Secondary Education

4. Changes to Follow-up Outcomes: Automatic Cohort Designation Program Year (FY2012): Participants identified based on self-reported goals (program – level) Program Year (FY2013): Participants automatically identified based on “cohort” which are based on demographic variables in the MIS system (state – level) The definition for “cohort” varies by outcome. Massachusetts Department of Elementary and Secondary Education

5. Implications of “Automatic Cohort Designation” • Students belong to a group to be followed for NRS • post-program outcomes based on pre-defined • characteristics. • Following students for post program outcomes will • no longer be goal dependent. • As has always been the case, we are only concerned with students having at least 12 hours of attendance. Massachusetts Department of Elementary and Secondary Education

6. Do Student Goals Still Matter? • YES!! • Local staff should continue to set goals with • students • Goal setting is an essential component of a good • program and good instruction but…. • Students goals will no longer be used as a way to • identify students for the NRS follow-up measures Massachusetts Department of Elementary and Secondary Education

7. Massachusetts Department of Elementary and Secondary Education

8. Secondary Credential Cohort Obtain Secondary Credential Cohort • Learners who take all GED tests, or • Are enrolled in adult high school at the high ASE level, or • In the assessment phase of the EDP, and • Who exit during the program year Entry into cohort occurs only for those who take all 5 GED tests by the end of reporting period (December 31). Massachusetts Department of Elementary and Secondary Education

9. Q: Do students have to still be enrolled when they take the tests? No, students are included in the cohort if they take all five test before the end of the reporting period for the program year in which they exit (December 31) Q: Must a student pass all GED tests to be included as achieving the outcome? Yes, students must pass all GED tests to be included as achieving the outcome Question: Cohort for Exited Students and GED Tests Massachusetts Department of Elementary and Secondary Education

10. Example 1: A student who enrolls in fall of 2012, exits in March 2013, takes 3 tests then, takes 2 more in September 2013, would be included in the cohort for 2012-13 Example 2: A student who enrolls in October 2012, takes 2 GED tests in May 2013, exits in November 2013 and takes 3 GED tests would be included in the cohort for 2013-14 Example 3: A student who enrolls in December 2012, exits in March 2013, takes 3 tests then, takes 2 more in January 2014, would not be included in any cohort Passing GED Tests: Examples Massachusetts Department of Elementary and Secondary Education

11. Postsecondary Cohort Postsecondary Cohort All learners who • have earned a secondary credential while enrolled • hold a secondary credential • or are enrolled in a class specifically designed for • transitioning to community college • and who exit during the program year Massachusetts Department of Elementary and Secondary Education

12. Questions: Foreign Credential and Transition class Q: Are students with a secondary credential, earned in another country, in this cohort? Are ESOL students to be included? • Yes, these students are included in the cohort Q: How is a transition class defined ? • A class that has a specific purpose to prepare students • for entry into postsecondary education, training or an • apprenticeship program. • The rate based class focus would be Bridge to College/Math • or Bridge to College/Writing. The non rate based class focus • would be Bridge to College/Math, Bridge to College/Writing, • or College and Career Readiness. Massachusetts Department of Elementary and Secondary Education

13. Q: Please clarify multi-year reporting of entry into postsecondary education Students who exit during program year achieve this outcome if they enter postsecondary any time from exit through the end of the following program year (June 30) Follow students and report for two years, in two rows on Table 5 (unduplicated) First report (2012-13): report same as of now Future years: Update prior year—keep cohorts separate First two year reporting on 2013 report (due December 31, 2014) Reporting of Entry into Postsecondary--Timing Massachusetts Department of Elementary and Secondary Education

14. Entered Employment Cohort Learners who are not employed at time of entry and in the labor force who exit during the program year Employment Status in SMARTT: Employed Unemployed and Looking for Work Unemployed and Not Looking for Work Retired or Otherwise Not Looking for Work Homemaker Programs should exit students who are no longer in attendance by filling out the “Exit Student” screen in SMARTT. Employment Cohorts Massachusetts Department of Elementary and Secondary Education

15. Retained Employment Cohort Learners who were not employed at time of entry and in the labor force, who are employed by the first quarter after exit quarter Learners employed at entry who exit during the program year  Employment Status in SMARTT: Employed Unemployed and Looking for Work Unemployed and Not Looking for Work Retired or Otherwise Not Looking for Work Homemaker Programs should exit students who are no longer in attendance by filling out the “Exit Student” screen in SMARTT. Employment Cohorts Massachusetts Department of Elementary and Secondary Education

16. Students, in entered employment cohort, that are employed the first quarter after exit are also in the retained employment cohort The timeframe is the same as always–it has not been changed States must use students’ employment status at entry to determine cohort status for employment outcomes. Reminder on Retained Employment Massachusetts Department of Elementary and Secondary Education

17. Q: How is “unemployed, in the labor force defined?” NRS definition (Unchanged): Learners who are not employed but are seeking employment, make specific efforts to find a job and are available for work Employment Status in SMARTT: Unemployed and Looking for Work Question: Definition of Unemployed in Labor Force Massachusetts Department of Elementary and Secondary Education

18. Entered Employment Employed in the first quarter after exit. Retained Employment Employed in the third quarter after exit. Timing of Employment Measure Massachusetts Department of Elementary and Secondary Education

19. Collection: One quarter after exit for entered employment, three quarters after exit for retained employment Reporting lag (previous year) to accommodate lag in Unemployment Insurance (UI) wage records Review: Employment Data Collection and Reporting Massachusetts Department of Elementary and Secondary Education

20. Massachusetts Department of Elementary and Secondary Education

21. Enter Employment/Get a Job -- Those with SSN and signed release form. For all others, survey method must be used. Retain Employment -- Those with SSN and signed release form. For all others, survey method must be used. Survey form may be accessed at: http://www.doe.mass.edu/acls/smartt/nrssurvey_followup.pdf Data Matching Department of Revenue Massachusetts Department of Elementary and Secondary Education

22. Obtain GED -- All students (regardless if SSN provided and release form signed since match will be with the Oklahoma Scoring Service) Enter Postsecondary Education -- -- All students (regardless if SSN provided and release form signed since match will be with the National Student Clearinghouse) Data Matching Oklahoma Scoring Service, and National Student Clearinghouse Massachusetts Department of Elementary and Secondary Education

23. Follow-up Outcome Timing Secondary Credential • Obtained credential before the end of the reporting period (December 31) • Postsecondary Education • Entered any time from exit until the end of the following program year (June 30) • Report for two years, in two rows on Table 5 • (unduplicated) • Programs should exit students who are no longer in attendance by filling out the screen found by clicking on the “Exit Student” menu option within SMARTT . Massachusetts Department of Elementary and Secondary Education

24. Reporting Changes: Table 5 • Table 5 revised to include instructions for • automatic cohort designation • New row added for postsecondary • transition: Report two years (unduplicated) • Update prior year • First two year reporting on 2013 report • (Due December 31, 2014) Massachusetts Department of Elementary and Secondary Education

25. Reporting Postsecondary Example in 2013-14 • In 2013-14, 125 students in cohort exit, state • matches 100 and finds 50 have entered • postsecondary • State runs match on 2012-13 cohort again by • June 30, 2014 and finds another 20 students • have entered postsecondary in addition to the • 40 from previous year Massachusetts Department of Elementary and Secondary Education

26. Example of Table 5: 2013-14 Massachusetts Department of Elementary and Secondary Education

27. Example of Table 5: 2014-15 Massachusetts Department of Elementary and Secondary Education

28. Implications for Implementation • The number of learners to follow will • increase. • Obtaining SSN’s and signed release forms will lessen the follow-up burden. • Local staff will need to follow-up by survey method for cases where SSN’s and signed release forms were not obtained for those within any of the employment cohorts. Massachusetts Department of Elementary and Secondary Education

29. Q: How should we determine years of student education? Do we need to check? Self-reported by student, NRS does not require verification but the state may choose to do so Q: How do we count education in other countries that is different than our system? Use years of schooling Use in-country definition for secondary credential Questions: Reporting Student Education Massachusetts Department of Elementary and Secondary Education

30. Q: Can students belong to multiple cohorts? Yes, for example, an unemployed student in the labor force, with secondary credential who exits, would be in two cohorts Question: Multiple Cohorts Massachusetts Department of Elementary and Secondary Education

31. Q: Do we have to re-identify cohort membership for students continuing the next year? Yes, for example, employment status or secondary credential status may change But only collect measure at exit Question: Continuing Students Massachusetts Department of Elementary and Secondary Education

32. ADDITIONAL QUESTIONS?? Massachusetts Department of Elementary and Secondary Education