Oakland Planning with Data & Kick-Off Webinar September 1, 2011

1. OaklandPlanning with Data &Kick-Off WebinarSeptember 1, 2011 Laenne Thompson Weikart Center

2. Please press *6 to mute and un-mute. If you can hear it, so can we.

3. Please press *6 to mute and un-mute. If you can hear it, so can we. This webinar is being recorded. If you come late, leave early or just want to refresh your memory. You can access the recording afterwards.

4. Please press *6 to mute and un-mute. If you can hear it, so can we. This webinar is being recorded. If you come late, leave early or just want to refresh your memory. You can access the recording afterwards. Be Prepared: This webinar will be participatory. Be ready to be called on.

5. Agenda • Quick Introductions • Using Data to Plan & Implement Program Improvements • Improvement Planning Overview • Submitting Improvement Plans Online (tutorial) • 2011 - 2012 YPQI Kick-Off • Questions and Next Steps

6. Introductions • Who’s here today? • Each site, please share: • The name of your organization • Something new you’ve tried at your site that went well since we last met

7. Youth Program Quality Improvement (YPQI) ASSESS PLAN IMPROVE Conduct SA(YPQI Team) Createimprovement plan (YPQI Team) Carry outimprovement plan (Mgr coaches; Stf do) Conduct EA(External assessor) REPEAT [Collect other data]

8. Improvement Planning Practice • As a group, we’re going to have a practice improvement planning meeting based on aggregate OFCY YPQA data.

9. Score Reports: Keep in Mind • Observation scores represent a snapshot –this has limitations and value. • The overall story is more important than the individual numbers. • What you do with it (how you step up to action) matters. • Reference: ‘How to Interpret PQA Data’ handout

10. Some pointers about YPQA data • Scores of all Fives • Not useful for identifying areas of improvement. • Extreme Variation • Might signify a misunderstanding of what the tool is measuring. • Typical trend • This distribution common as you become more comfortable with the process and the tool. • Program self assessment scores may be lower in subsequent years because raters are being more honest or more critical as they develop their reliability.

11. OFCY External Assessment* vs. National Sample** *n = all OFCY Older Youth, Wellness and Healthy Transitions and Community-Based After School programs listed in SeeChange 2010 – 2011 Interim Report, p. 14; **n = 902 sites

12. Aggregate Data Graph • Things to consider: • What does it mean? • What do you see here? • What does the data tell us?  • Strengths?  • Areas for growth? • What should we do as a response?

13. Aggregate Data Graph • What scores seem lower than you would want? • What item/domain do we want to focus on?

14. OFCY External Assessment* vs. National Sample** *n = all OFCY Older Youth, Wellness and Healthy Transitions and Community-Based After School programs listed in SeeChange 2010 – 2011 Interim Report, p. 14; **n = 902 sites

15. Sample Improvement Plan

16. Sample Improvement Plan

17. Sample Improvement Plan

18. Steps to make an Improvement Plan Set goal & address elements of quality (YPQA items) Implementation: Develop action steps for each goal • Timeline • Evaluate • (Mini) Self Assessment • Staff meeting check in • Observation-Reflection • Other? • Discuss desired support from administrator • Training, information, etc… • Youth Work Methods courses & other further YPQA work **Don’t forget- - Involve staff in the process!

19. More to Consider: Youth Survey Data • How to use this data in your Improvement Plans? • What from the Youth Survey Data stands out as highest priorities for improvement? • What YPQA data does it correlate with? • Does the Youth Survey data help you prioritize which YPQA items to tackle in your Improvement Plan?

20. Discussion • How do you feel about this process? • Questions or potentially challenges about replicating it at your program/site?

21. Submitting Improvement Plans Online (tutorial) • You will receive your login information in an email today from Leah Wallace of the Weikart Center • If you have not received an email with those please contact leah@cypq.org as soon as possible. • These logins will be used to enter both: • The Spring 2011 Improvement Plan (due 9/30) • The Fall 2011 YPQA Self-Assessment data

22. Login at scoresreporter.cypq.org

23. Why do I need to enter my plans? • It allows you to track your goals and your progress in reaching those goals from year to year. • It allows your program leader to tailor professional development opportunities • It helps create an overall picture of what organizations in your city are prioritizing to improve the quality of their programs.

24. YPQI TIMELINE

25. Next Steps & Expectations • Meet with your program team and choose goals for your Improvement Plan • Determine actions steps and monitoring activities for each goal • Finalize your Improvement Plan: enter online by September 30

26. 2011 – 2012 OFCY Grantee Youth Program Quality Improvement Kick-Off

27. YPQI Kick-Off: 2011 - 2012 • Review: What is the YPQA? • What is the YPQI? • Self-Assessment • Next steps and Key Dates • Q/A

28. Quality Matters Transfer: Application of skills in new settings Time: Multiple sessions Setting: Point of service Quality of Instruction & Content Youth Engagement • Proximal skill building • Social emotional • 21st Century • Cognitive • Short-term outcomes • Achievement • Healthy behavior • School retention

29. YOUTH PROGRAM QUALITY ASSEMENT (YPQA)

30. The Youth PQA is a research validated instrument designed to assess the quality of youth programs for the purposes of accountability, evaluation, and program improvement. The instrument has been used in a wide variety of settings including after-school, community-based, camp, drop-in, and mentoring. What is the YPQA?

31. The Pyramid of Program Quality Youth are emotionally and physically safe. Youth are supported by staff and engaged in hands-on skill building Youth work together and experience a sense of belonging Youth have access to high level developmental experiences: planning, setting, goals, making choices and reflecting Plan Make choices Reflect Lead and mentor Be in small groups Partner with adults Experience belonging Encouragement Reframing conflict Skill building Session flow Active engagement Welcoming atmosphere Psychological and emotional safety Program space and furniture Emergency procedures Healthy food and drinks Physically safe environment

32. The YPQA is… • …rooted in idea of Positive Youth Development • …content-neutral • …a measurement that guides learning and professional development • …a tool to measure what happens at the point-of-service, where staff and youth interact

33. Higher scores at higher domains are associated with higher levels of youth engagement • Youth self-reports of: • Belonging • Interest • Challenge • Learning

34. High quality instruction provides youth with opportunities to practice emerging social and emotional skills… • Efficacy • Communication • Empathy • Problem Solving … that support success in adolescence and early adulthood.

35. Lower scores are associated with youth disinterest

36. WHY THE YPQI?YOUTH PROGRAM QUALITY IMPROVEMENT

37. Theory of Change: Multiple Levels of Settings This is the youth at your programs Point of Service Setting Quality Instruction & Proximal Child Outcomes

38. Theory of Change: Multiple Levels of Settings This is your organization and staff Organizational Setting Continuous Improvement Practices for Site Teams

39. Theory of Change: Multiple Levels of Settings What you do here…. Organizational Setting Will improve the experiences for kids here Point of Service Setting

40. Benefits of YPQI • Creates a common language of quality • Emphasizes positive youth development • Provides a framework for professional development

41. What’s new for 2011 - 2012? SELF-ASSESSMENT AND THE YPQI

42. This is the PQA Self Assessment Process ASSESS Standardized Assessment of Instruction IMPROVE Training for Instructional Skills IMPROVE Coaching and Performance Feedback PLAN Team-based Planning with Data

43. Assess The PQA allows you to observe what actually goes on in your youth programs? How often do you have the time or energy to simply observe a program led by another staff? How often do you rely on what you think happens in your programs?

44. What does the Assess Stage look like? • Team Selection • Team Training • Planning the Data Collection • Collecting Data • Team Scoring Meeting

45. Team Selection • Select 1-3 people from your organization to be on the self-assessment team • Make sure to include frontline staff • Attend a Youth PQA Basics Training

46. Observations and Consensus • Plan to observe 2-4 program offerings • Observe at least 15-20 minutes, but try to observe the entire offering • Come together with all your data and come to consensus scores for the entire program based on that data. • The conversation is the most important part of this process!

47. Data Entry You will enter your final scores into an “online scores reporter” data base. This will allow you to make reports and compare your data form year to year.

48. Plan Looking at the results of your assessment gives you a chance to make concrete plans. What are your program strengths? What do you want to improve?

49. Planning with Data This workshop will allow you to look at your assessment data in depth and create improvement plans based on it.

50. Improve During the improve phase you will implement your plans. What kind of professional development do you need? What additional resources? What changes to the curriculum?