Quality Matters: Building Capacity and Investment in Youth Program Quality

1. Quality Matters:Building Capacity and Investmentin Youth Program Quality The Center for Youth Program Quality

2. Objectives • Understand how quality defined and measured by the Youth Program Quality Assessment and other quality assessment tools • Learn about quality improvement systems currently implemented in numerous statewide and place-based networks

3. Systems for QualityLeveraging existing “change” resources Professional Development Local Evaluators Accountability Measurement POS Quality Access to key developmental and learning experiences

4. Outcome Areas Ages after-school Times of Day/Year Quality & Reach… Looking into the developmental white space At best, schools fill only a portion of developmental “white space.” Who fills the rest? And what is the “locally appropriate mechanism for monitoring theavailability, accessibility, and quality of programs…” in school and out? Saturating communities with “ample programs” requires improving the quality and reach of all the systems, settings and programs that touch young people’s lives. Selective replication and improvement are not sufficient. Developmental “White Space” school

5. The Landscape of Youth Programs Children’s Services in LA County SOURCE: Margaret Dunkle

6. The Ready by 21™ Quality Counts Framework Domains Elements Strong Policy / Leadership Horsepower Decision-maker engagement & coordinating structures Aligned policies for quality accountability and improvement Shared vision, strong demand, active family/youth involvement Strong, Stable Program Base Healthy program landscape (distribution & focus) Cross-system program data base/info source Cross-system convening/coordination mechanisms Capacity to Assess & Improve Programs Buy in re definitions, quality standards, accountability requirements Widely adopted assessment and monitoring procedures Adequate assessment and improvement training/TA capacity Capacity to Recruit, Train, Retain Workforce Cross-system provider networks and communications Accurate data on workforce (skills, supports, recruitment, retention) Professional development opportunities/incentives

7. What is program quality? ??? inputs inputs outcomes inputs youth program Another way to say it: • What do we want to see in high quality youth programs? 7

8. Defining Quality Several ways to organize: • Readin’ + ‘Ritin + ‘Rithmatic (old-school) • Affect + Active Learning + Metacognition (Education) • Relatedness + Autonomy + Competence (Psychology SDT) What kids need… Relationship + Task + Increasing Complexity What adults should do… Content Therapeutic process

9. Our Quality Construct: The Pyramid of Program Quality Plan Make choices Engagement Reflect Lead and mentor Be in small groups Partner with adults Interaction Experience belonging Encouragement Reframing conflict Supportive Environment Skill building Session flow Active engagement Welcoming atmosphere Psychological and emotional safety Safe Environment Program space and furniture Emergency procedures Healthy food and drinks Physically safe environment

10. SAE System Accountability Environment PLC Professional Learning Community POS Point Of Service Engage Interaction Viewing Quality in a Systems Context Support Safety

11. Daily supervision & support by site coordinator Community Continuity Pre-session planning Frequent staff meetings Learning Features Data & Information Performance Feedback External monitoring Lesson & curriculum review Youth engagement & content relevance Focusing Features Management Priorities/Values Purposeful relationships Youth voice structures Defining Quality What can management do?

12. Defining quality at the system levelCurrent Regulatory Models Miss the POS Source: (1998).The NSACA Standards for Quality School-Age Care.There are thirty-six keys of quality and 144 total standards 6 standards Engagement 4 standards Interaction 33 standards Supportive Environment 49 standards Safe Environment 1 standard Youth Organizational Voice 19 standards Professional Learning Community 32 standards Other

13. The Youth Program Quality Assessment (YPQA) • Validated observational assessment tool • Measures quality at the point of service • Assesses frequency and access to key developmental experiences • Can be used to assess progress over time • Content-neutral for use across settings, ages, systems

14. Structure of the Youth PQA Form BOrganizational Interview Ask questions, write, score (2 hours) Form AObservation Watch, write, score (3 hours) Program Offering 1 Program Offering 2 Organization Program Offering 3 Program Offering 4

15. Sample Item from the Youth PQA “Domain” “scale” “item row” The Youth PQA consists of 7 Domains (4 in A, 3 in B) 30 Scales (18 in A, 12 in B); 103 item rows (60 in A, 43 in B)

16. What POS Quality Looks Like on the Ground Occurred For All Occurred For Some Did Not Occur • Sample of nearly 600 different youth workers • Parallel findings in schools research The Center for Youth Program Quality

17. Does it Work? Findings from Several Samples • POS quality-outcomes findings: • Supportive environment related to: Attendance • Interaction related to: Interest in program • Engagement related to: Sense of challenge, sense of growth, school-day reading, school-day suspension • Note: No offerings get to high engagement without high support and high interaction • Quality Improvement (YPQI) Findings • Scores increase from pre to post • Scores increase in the targeted areas more • Management practices are related to quality change (Vision, Feedback, Continuity)

18. SAE System Accountability Environment PLC Professional Learning Community POS Point Of Service How we think about DDCI- People change not programs Prochaska, J.O., & DiClemente, C.C. (1982). Transtheoretical therapy toward a more integrative model of change. Psychotherapy: Theory, Research and Practice, 19(3), 276-287. Maintenance Repeat cycle Action Implementation & coaching Preparation Planning with data Contemplation Quality assessment Individual Change Model Organizational Context The Center for Youth Program Quality

19. Competencies for Youth Development Practitioners Emergence of POS quality culture Spend time planning & prepping activities Improve practice systematically "Team focused on youth experience" mentality Intentionality in POS quality practice Icebreakers and inclusion Cooperative grouping strategies Targeting learning edge – scaffolding, engaging youth in planning & reflection, active learning Providing opportunities for youth voice and leadership

20. Competencies for Management • Knowledge/understanding of positive youth development methods individually and integrated as the High/Scope participatory learning approach; • Conduct reliable performance assessment based on observation and validated measurement rubrics; • Conduct performance coaching based on strengths based transmission of performance data; • Lead a staff team through a data driven quality improvement process

21. System Level Outputs • Accountability policy with high returns in staff buy-in and learning; • Common language and terminology supporting focus on quality at the point of service; • An integrated professional development investment that is integrated over time as a sequence of PD experiences, integrated across levels of organization and professional roles, and integrated with ultimate program purposes in positive youth development;

22. Outcomes • High quality programs • Better staff retention and retention of better staff • Greater impact on youth development and learning

23. `` ` l Systems for QualityAccountability Policies in Places • YPQA is part of state and county accountability policies: • Cross sector (DHS& DOE) snapshots: Iowa, Washington, Arkansas • Statewide 21st Century: Michigan, Maine, Minnesota, Rhode Island, New Mexico, • Cities and Counties: Rochester, Detroit, Grand Rapids, Palm Beach Rochester Grand Rapids Washington* M i n n e a p o l i s Chicago New York e t r o i t Iowa Indianapolis Rhode Island m b u s Georgetown Divide Columbus St. Louis Kentucky Oklahoma Nashville Austin West Palm Beach County

24. Designing Quality Improvement Systems (QIS) Mostly Mangers Managers and direct staff TOTs for quality assess, coaching, and youth work methods (f,g) SystemCapacity Use of on-line dashboards and training (d) External quality assessment (a) Quality Advisor (e) Coaching & Training External Quality Report with Norms Self-Assessed Quality Report Targeted youth work methods training for direct staff (h) Use of on-line dashboards and training (d) Quality coaching by managers (i) Program Staff Skill & Knowl Self-assessment of Quality (b) Planning with Data (c) Phase 2: Impact & Sustainability Phase 1: Readiness & Capacity

25. Defining the Purpose of Your QIS Lower Stakes Program Self-Assessment Rough data toget staff thinking anddiscussing programquality in the contextof best practice Less time Less money Impact internal audiences (the creative middle) Higher Stakes ExternalAssessment Precise data forinternal and externalaudiences for evaluation,monitoring, accountability,improvement, reporting More time More moneyImpact internal and external audiences

26. Columbus IndianaPhase 1: Building Local Capacity POSPoint Of Service SAESystem Accountability Environment PLCProfessional Learning Community STEP 2a Self-assessment STEP 1 Decide to build system STEP 3 Plan for improvement STEP 4 Carry out plan STEP 5 Measure change STEP 2b External assessment August 26 Youth PQA Basics January 27-28Ext Assessment October 8 Planning with Data Improvement Plan Annually Program SA Ext Assessment Observe-Reflection Planning with Data Opt Phase 2 Method Workshops Quality Coaching August 25 Quality Matters Presentation

27. Questions about… Purpose? Process? Pilot? Next Steps…