Quality Service Review

1. Quality Service Review Organizational Learning for System Improvement

2. Mission Statement The Philadelphia Department of Human Services’ mission is to provide and promote safety, permanency, and well-being for children and youth at risk of abuse, neglect and delinquency

3. Organizational Chart

4. Performance Management and Accountability (PMA) • Division of PMA started in 2009 • Mission: The division of Performance Management and Accountability will support system improvement by monitoring and evaluating the efficiency, effectiveness, and availability of our services both internal and external • Five Pillars: DIM, QI, PREP, PBC, and Research

5. Quality Improvement (QI) • Case File Review • Quality Visitation Review • Quality Service Review • Child Fatality Review • ChildStat

6. Quality Service Review The Quality Service Review (QSR) is a practice improvement approach designed to assess current outcomes and system performance by gathering information directly from families, children and service team members

7. Quality Service Review History • Mid 1990’s: The first QSR protocol developed • Other states that use the QSR process • Alabama • Utah • Wisconsin • Fall of 2009: Pennsylvania and Philadelphia management observe Utah’s QSR process • January 2010: PA’s OCYF adopts the QSR process and selects Philadelphia to participate in the first QSR pilot process • June 2010: Philadelphia begins internal QSRs

8. Philadelphia’s Local QSR Process • Occurs six times a year • Small sample of randomly selected cases • Stratified by service area • Congregate Care • Children with Special Medical Needs • Adoption • General Foster Care (Foster and Kinship Care) • Treatment Foster Care • In-Home Services Array • Older Youth (SIL/TLP)

9. QSR: Gathering Information • Review teams • Brief review of the case file • Initial interview with DHS worker and supervisor • Interviews with stakeholders and families • Focus child/youth and biological family • Foster parents and group home parents • Private provider workers • Attorneys • Mental health, educational, and medical personnel • Focus Groups

10. Difference between Traditional Reviews and the QSR Traditional Review Were visits timely? Is the child’s IEP in the file? Was there a referral to a mental health service? Quality Service Review Are children currently safe? Is the child progressing well academically? Is the child functioning well emotionally?

11. Difference between Traditional Reviews and the QSR Traditional Review Is there a current plan in the file? Did the parents sign the plan? Is there a written assessment? Quality Service Review Is the plan likely to lead to permanence? Do the parents feel that the plan is their own? Does the assessment accurately identify underlying needs?

12. QSR: Scoring Process • Pennsylvania QSR Protocol • 9 Child/Youth and Family Indicators • 11 Practice Performance Indicators • Scoring Process • 6 – Optimal • 5 – Substantial Acceptable Range • 4 – Fair • 3 – Marginal • 2 – Poor Unacceptable Range • 1 – Adverse

13. QSR: Feedback • Case specific feedback • Final interview with DHS worker and supervisor • Written Case Story • Debriefing • Aggregate system-level feedback • Findings Present Meeting • Findings Present Report

14. Data Analysis • Aggregate scores from all 12 cases • Comparison of acceptable and unacceptable scores with all QSR cases • Content analysis • Case Story recommendations • Focus Groups • Trends and recommendations

15. Findings Present Report • Demographic information • Indicator ratings • Child/Family • Practice Performance • Comparison with overall QSR ratings • Case Stories • Recommendations

16. Findings Present Meeting • All QSR reviewers • DHS management • Private provider representatives • Leaders from the courts • Child advocate and parent attorney offices • Representatives from the mental health, educational, and medical systems

17. Findings Present Graph

18. Overall Trends: Strengths • Safety • Physical health • Engagement of child/youth and substitute caregiver

19. Current Improvement Initiatives • Increase the use of Family Finding and Family Group Decision Making to support family engagement and culturally appropriate practice • Use of specific teamings at key case intervals (e.g. Permanency Action Teamings) to improve planning, permanency, and teaming indicators

20. Tracking Recommendations • Suggested improvements • Submitted to the Commissioner and her cabinet for approval • Assigned to a responsible person • Quality Improvement tracking system • Used to track recommendations from all QI reviews • Recommendations entered into a database • Monthly updates recorded to track individual implementation • Quarterly reports published to track overall implementation

21. Response to the QSR from DHS Staff • “This process allows the child, family and team members to be seen and heard rather than just read about. This allows for a better understanding of actual service delivery and needs.” -DHS Worker • “I like that the QSR takes more than just the case record into account. By speaking to all parties a person is able to get a better understanding of the case.” - DHS Supervisor

22. Response to the QSR from Outside Stakeholders • “I like being heard and feeling that providers are recognized for the service that is being provided. I appreciated the reviewers coming out to our facility making it convenient for me to participate. The reviewers did a great job making me comfortable to share my honest opinions.” – Provider Worker • “I think this process of review is very important to make improvements and give better service.” –Medical Personnel

23. Come Observe! • Interested in observing the QSR process? • Contact Brian Clapier (Brian.Clapier@phila.gov) • Pennsylvania QSR Protocol • http://www.pacwcbt.pitt.edu/Resources/PA%20QSR%20Protocol%20Version%201%200.pdf

24. QUESTIONS Contact Information: Susan.Kinnevy@phila.gov Brian.Clapier@phila.gov Allison.Thompson@phila.gov Cynthia.Brown@phila.gov