Treatment Perceptions Survey ( TPS ) DMC-ODS

1. Treatment Perceptions Survey (TPS)DMC-ODS

2. Why are Counties/Providers Administering the TPS? • Address the data collection needs for the Centers for Medicare and Medicaid Services (CMS) required evaluation of the DMC-ODS demonstration • Fulfill the county’s External Quality Review Organization (EQRO) requirement related to conducting a client satisfaction survey using a validated tool • Support DMC-ODS Quality Improvement efforts

3. How was the TPS Developed? • Adult version based on San Francisco County’s Treatment Satisfaction Survey • Youth version based on Los Angeles County’s Treatment Perception Survey for Youth • Input from stakeholders (e.g., county BH administrators, DMC-ODS evaluation Advisory Group, EQRO, DHCS, Youth System of Care Evaluation Team at Azusa Pacific University )

4. When Do Counties Administer the Surveys and Submit Data to UCLA?

5. Who Should be Surveyed? • Youth (12 -17 years old) and adult (18 years old and older) clients who present in-person and receive face-to-face services at providers within the county during the survey period • Includes clients receiving face-to-face services outside the office (e.g., field-based settings) during the 5-day survey period • Treatment settings: OP/IOP, Residential, OTP/NTP, Detox/WM (standalone), partial hospitalization

6. Who Should Not be Surveyed with the TPS? • Clients who do not receive face-to-face services during the survey period • Clients in immediate crisis (e.g., emergency situation)

7. TPS Forms • 14 questions plus demographic items on the TPS Adult form • 18 questions plus demographic items on the TPS Youth form • 13 languages - English, Chinese, Spanish, Tagalog, Vietnamese, Russian, Arabic, Korean, Armenian (Eastern and Western), Cambodian, Farsi, and Hmong • 1-page and 2-page (large font) versions

10. Survey Form Preparation • Accurate information is required for UCLA to prepare program-level summary reports for your county. • Fill in the information by using: • the online fillable feature before printing the Adult and Youth TPS forms (PDFs); or • a black or blue pen

11. Survey Form Preparation, cont’d • Print survey forms from PDF files that can be found on the TPS website. Please do not photocopy the survey forms. • Print in black/white on plain white paper. • Use both sides of the page (double-sided) when printing the 2-page forms. Do not staple or use paper clips to attach the pages.

12. How Should the Survey be Administered? • Offer survey forms to all youth (12-17 years old) and adult clients receiving face-to-face services during the 5-day survey period. • Clients need complete only one form during survey • Whether the client completes the survey form will in no way adversely affect the services s/he receives. • Inform clients that the survey is anonymous. They should not write their names on the forms. • Be sure clients use a black or dark blue ball point pen. • If the client makes a mistake on the form, s/he should draw an “x” over the incorrect entry.

13. What About Client Confidentiality? • Direct service staff must not be present while the client completes the survey. • A non-clinical staff person, consumer advocate, or volunteer can help the client complete the survey form upon request by the client. • Staff are not to influence how a client responds or deny a client the opportunity to complete the survey. • Clients are to place completed survey forms directly into a ballot-type survey form collection box or large envelope.

14. Client Confidentiality, continued • Clients receiving services outside the office during the survey period should fill out a survey form and seal it in an envelope (provided by staff). • Staff should deposit the envelope into the survey collection box or large envelope with the other completed forms upon returning to the office. • Agency staff should package the completed forms for delivery/shipping to the designated county administrator who is coordinating the survey.

15. Special Note: Client Comments • It is the responsibility of the program/county Clinical Director/ Supervisor to review the client comments (prior to sending the forms to UCLA) for anything that may need immediate attention. • If sending electronic data to UCLA, please do not include the client comments.

16. How Should We Submit Paper TPS Forms Data to UCLA for Scanning? • Compile completed survey forms from provider network. • Complete and email the Shipping Form to UCLA. (UCLA will prepare UPS shipping labels and order supplies, if needed.) • Pack the forms in a box/envelope and complete the Cover Sheet. • Use new boxes and envelopes. • Be sure to seal/tape boxes well to prevent damage during shipping. • Send the box(es)/envelope(s) to UCLA.

17. Shipment Form

18. Cover Sheet

19. How Should We Prepare and Submit Electronic TPS Data Files to UCLA? • Compile completed survey forms from provider network. • Inform UCLA that your county will be submitting data files rather than paper forms for scanning. • Follow the instructions posted on the TPS website for preparing and submitting electronic data • Codebook • Uploading files to your county’s folder in UCLA Health Sciences Box

20. Data Scanning, Analysis, and Reporting • UCLA will scan and analyze the data, and prepare county- and program-level summary reports. • Reports prepared within 3 months after receipt of completed survey forms. • County access to images of client comments. • UCLA will provide counties with access to their raw data files.

21. County-Level Summary Report: Sample Tables

22. County-Level Summary Report: Sample Tables

23. County-Level Summary Report: Sample Table

24. County-Level Summary Report: Sample Table

25. County-Level Summary Report: Sample Figure

26. County-Level Summary Report:Sample Figure

27. Program-Level Summary Report:Sample Comparison to County Figure

28. What is the UCLA Health Sciences Box? • It is a secure, HIPAA-compliant file-sharing platform. • Each county will have a separate folder. • UCLA will send an invitation to individuals identified by the County Administrator to set up a free Box account. • County folders will contain county- and program-level summary reports, raw data files (and images of client comments). • Counties that scan/enter the data locally will be able to upload their data files into their folders in Box to UCLA. • Box accounts expire 6 months after they have been created, so counties may want to download the files.

29. FAQs • Survey Administration • Survey Forms • Survey Data Submission/Analysis/Access, and Reporting Link: http://uclaisap.org/ca-policy/assets/documents/TPS/FAQs.pdf

30. What Information is Posted on the TPS Website? • MHSUDS Information Notice No: 17-026and TPS Instructions (Adult), and MHSUDS Information Notice No: 18-032 and TPS Instructions (Youth) • Survey forms • PDF files • Teleform Definition Files • FAQs • Instructions for shipping forms to UCLA (e.g., UPS shipping labels, supplies, cover sheet) • Instructions for submitting electronic data files (e.g., codebook) • Instructions for using UCLA Health Sciences Box (HIPAA compliant data sharing platform) • Sample County- and Program-level summary reports (TPS – adults) TPS website link: http://www.uclaisap.org/ca-policy/html/client-treatment-perceptions-survey.html

31. Be sure to check the TPS web site periodically for updates: http://uclaisap.org/ca-policy/html/client-treatment-perceptions-survey.html • Contact Cheryl Teruya (cteruya@mednet.ucla.edu) if you have questions or feedback.