Office of Career and Technical Education

1. Office of Career and Technical Education ONSITE TRAC TRAINING May 16, 2014

2. WELCOME Joanne Mahony mahonyj@michigan.gov (517) 335-0405

3. TRAC OVERVIEW • TechnicalReview, Assistance, and Compliance • The purpose of TRAC is: • To provide technical assistance, and • Monitor for compliance with State and Federal laws

4. OVERVIEW OF TRAC PROCESS Each Region Every Five Years • Added Cost • CTE Perkins • CTE Instructional Programs

5. MAJOR STEPS • Pre-visit preparation and desk review • Onsite visit • Report to region • Compliance plan submitted by region • Compliance plan and evidence approved by OCTE

6. REVIEW CATEGORIES • Data • Finance • Grants • CTE Instructional Programs NOTE: The Office of Professional Preparation Services (OPPS) will join OCTE on TRAC reviews to complete OPPS review of Teacher Certification. More information will be given during OPPS Presentation this afternoon.

7. WARNING: Submission of documents that are subject to confidentiality and privacy laws should NOT be submitted electronically. These documents will be reviewed onsite.

8. QUESTIONS?

9. TIMELINE Elizabeth Collins collinse2@michigan.gov (517) 335-3149

10. TRAC TIMELINE (OCTE) • What’s happened: • Region selection • Onsite visit notification • TRAC training • What’s ahead: • Grant activity selections • Fiscal agency selections • CTE instructional program selections

11. TRAC TIMELINE (Region) Document Submission for Desk Review: • CIP Self-Review documentation for selected programs is submitted through Navigator • Data, Finance, and Grant documents submitted through GEMS (checklists) • All documents are due September 8, 2014 • Agenda (due three weeks prior to onsite)

12. TRAC TIMELINE • Day 1 - Program reviews begin • Day 2 - • Program reviews continue • Introduction meeting • Data/Finance/Grant reviews begin • Day 3 – • Program reviews continue • Interviews • Wrap Up meeting

13. TRAC TIMELINE • Desk Review and Onsite Visit • OCTE Report Of Findings in GEMS • Not In Compliance - Action Required • Recommendations (comments, technical assistance, observations) • Commendations (exemplary program, best practices)

14. TRAC TIMELINE • Appeals Process • Compliance Plan (created by Region) • Compliance Plan and Evidence (Approved by OCTE) • Final Letter

15. IMPORTANT CHANGES • Expanded State Desk Review • Documents Required • No Compliance Plan Workshop • No Priority (I/II) Findings • Implementation of GEMS • Navigator – CIP Programs

16. QUESTIONS?

17. GRANT REVIEW Joanne Mahony mahonyJ@michigan.gov (517) 335-0405

18. GRANTS • ALL desk review documents for Grant review must be uploaded to GEMS by September 8, 2014. • Personally identifiable information must be redacted from documents before upload or documents must be provided onsite.

19. GRANTS G01 – 20% of CPI Indicator Activities from 2013-14 application Factors considered in selection process: • CPI Deficiencies • Regional Improvement Plan Activities • Amount of grant funds budgeted • Relevancy of activity

20. GRANT G01 DOCUMENTATION • Evidence as identified and approved in the application must be provided • Evidence must reflect the expected outcomes as identified in the application • Evidence must be consistent with the scope of the activity and outcome

21. GO1 – Evidence Examples G01 evidence may include: • Meeting minutes and sign-in sheets • Pre- and post-test scores • Revised curriculum • Planning documents • Documentation of services provided to students (payroll records and work logs) • Dated vendor payments • Other

22. EXAMPLE EXPECTED OUTCOME CPI: 1S1 Academic Attainment/Reading Activity: Curriculum Integration Expected Outcome: 60% of the region's CTE teachers will participate in professional development related to the integration of reading concepts and will integrate contextual reading activities into curriculum to support student growth in reading.

23. EXAMPLE

24. G01 - Evidences • Provide Regional/CEPD/District documents (as appropriate) listed under DESK REVIEW for review items (participation may have been at the regional, CEPD or district level). • No personally identifiable information

25. Evidences Supporting: • Regional Long Range Plan • Eligibility for Perkins Funding • (G02 – G06)

26. G02 - Equity and Access Documents that demonstrate: • Students have equal access to CTE programs • All students are included in outreach activities • Compliance with federal law and regulations (includes eligible charter schools and home-schooled students)

27. G02 - Evidences Evidences may include: • Counseling, advising and admission policies and procedures demonstrating equal access • Outreach materials which may include brochures, videos, letters or flyers (including ELL languages as applicable)

28. G03 – Regional Planning/Stakeholder Documents that demonstrate: • Grant priorities • Joint planning and coordination of CTE programs • WDB/EAG was provided the opportunity to review and provide input

29. G04 – Special Population Services Documents that demonstrate: • Region/CEPD/District plan is in place to identify and provide services to SPOPS (LRP) • Region/CEPDs/Districts have assessed and met the needs of SPOPS • Region/CEPDs/Districts have conducted outreach targeting SPOPS

30. G05 – Professional Development Documents demonstrating Professional Development: • Secondary teachers and personnel • Administrators • Career Guidance and Academic Counselors

31. G06 – Nonpublic and Charter Schools Documents that demonstrate compliance with federal laws and regulations regarding participation by students, teachers and administrators in: • Nonprofit, nonpublic schools and home-schooled students • Charter schools

32. G07 – Previous Onsite Only provide documents if requested after the desk review is completed.

33. DATA REVIEW David MacQuarrie macquarried@michigan.gov (517) 241-6202 Krishnan Sudharsan sudharsank@michigan.gov (517) 241-7652 Valerie Felder felderv@michigan.gov (517) 335-1066

34. Focus of Data Review Federal Funding and Reporting • Federal Fund Reporting: • CTEIS Data Report Evidence • District Evidence • Accuracy between data evidences for Core Performance Indicators • Report Use for Program Improvement: • Share CPI Data Reports • Use CPI Data for Program Improvement • Policies and Activities that promote quality programs • State Fund Reporting: 61a1 • Course Enrollment Data Use for Program Improvement CTE Program State Funding & Reporting

35. DATA - Common Findings • Course section minutes or weeks reported in CTEIS and shown on X0107 report, do not match district documentation: • Master schedule • Bell schedule • District calendar • Failure to provide bell schedule, master schedule or district calendar for ALL PSNs specified by OCTE for review

36. DATA - Common Findings • Course section titles often don’t match those in CTEIS and shown on the X0107 report. • Could result in findings of noncompliance • If master schedule course names do not match course section names in CTEIS, please provide a crosswalk showing both course names • Enrollments reported to OCTE don’t match teacher class/course records • Please be sure to provide documents for the year under review (usually prior school year)

37. DATA - Common Findings • Documentation not provided to demonstrate that: • CPI reports were shared with instructors • CPI reports were used for program improvement • Follow Up Survey results, including related placement data shared with instructors (Examples of adequate evidence include meeting minutes or memos)

38. DATA - Common Findings • Data processes and procedures documentation such as data security procedures, not provided

39. DATA - Common Findings • Evidence not provided for all CEPDs or districts in Region. • PSN-specific documentation is required for data reported at the course-section level (and is only required for the specified PSNs): • Examples: • Class lists • Instructional Design • Minutes • Weeks

40. DATA - Common Findings • Evidence not provided for all CEPDs or districts in Region (continued): • Documentation demonstrating effort across the entire region is required for non-PSN-specific data activities: • Examples: Sharing CPI results, Sharing follow-up results, Data security procedures Documentation must be provided demonstrating that activity was carried out at the regional level or, if carried out at CEPD or district level, a representative sample of documentation must be provided for every CEPD.

41. TIPS FOR REVIEW ITEMS D01-Perkins quality data: • Provide teacher class lists for specified PSNs on-site (do NOT upload Personally-Identifiable Information (PII) to GEMS • Provide PII onsite • Examples: documents with student names, UICs, information on individual students, IEP (even without name)

42. TIPS FOR REVIEW ITEMS D02-Accurate CPI data: • Teacher class lists for specified PSNs (see D01). Provide onsite only. Do not upload to GEMS. • List of Students showing Special Populations category • From DISTRICT records (not exported from CTEIS). • Provide onsite only. Do not upload to GEMS.

43. TIPS FOR REVIEW ITEMS D02-Accurate CPI data: • Teacher instructional design document for every PSN identified by OCTE (not printed from CTEIS) • Articulation agreements for PSNs identified by OCTE • Regional list of PSNs that have articulation agreements

44. TIPS FOR REVIEW ITEMS See review checklist for recommended evidence. Findings of noncompliance are common in: • D03-CPI Reports/CPI Activities • D04-Added Cost Quality Data • Tip: provide counts of days of instruction on calendars and minutes of instruction on schedules.

45. TIPS FOR REVIEW ITEMS See review checklist for recommended evidence. Findings of noncompliance are common: • D05-Placement data (follow-up survey) Tip: Remember to provide evidence for: • Total placement • Related placement • CPIs used for program improvement • CPIs shared with staff

46. TIPS FOR REVIEW ITEMS D06-CPI Report availability Tip: Provide web links to reports for all districts in the region D07-Data security Tip: Provide district-level policy documents for all districts in region. Tip: Agendas, meeting minutes or memos can be used to demonstrate that policies and procedures were shared with staff.

47. TIPS FOR REVIEW ITEMS D08-Previous monitoring review • Only provide evidence if requested from OCTE after desk review • Tip: Review prior TRAC Report to avoid repeat findings.

48. Naming Evidence Files In GEMS • Label each file uploaded to GEMS with: • Review item code (example: “D02”) • PSN (for PSN-specific evidence) • -or- District (for District-level evidence) • -or- CEPD (for CEPD-level evidence) • Description: (example: “InstructDesign”) EXAMPLE FILE NAME: Example: ReviewItemCode_PSN_Description PSN example: D02_12509_InstructDesign Building example: D04_Randolph_MasterSchedule District example: D04_DetroitPS_DistrictCalendar CEPD example: D03_CEPD41_AgendaWhereCPIsShared

49. For Evidence Provided Onsite • Don’t forget to use: • A sticky note on a page • Highlight a sentence, paragraph, etc. • Draw an arrow to the sentence, picture, etc. • Provide crosswalks between district documents and or with CTEIS reports, if different

50. QUESTIONS?