Outcome-Based Management Program

1. Outcome-Based Management Program Training Presentation September 8-12, 2008

2. Introductions and Icebreaker Please say the following about yourself: • Name • Club • Position • One positive thing you’ve seen with a member or the club while working at BGCC

3. Agenda

4. Part 1: Outcome-Based Management Program Overview

5. What Are Outcomes? • Outcomes are the desired results, both short term and long term, that our members will experience by attending our clubs and programs • Short term program results that are small contributions towards long term outcomes • Examples • Academics: Increased likelihood of being promoted grade levels • Character: Greater knowledge of positive vs. negative behaviors and attitudes • Health: Higher participation in sports activities outside of the club Intermediate Outcomes Long Term Outcomes • Program end results, often difficult to measure, that target core assumptions and behavior, and are built from short term outcomes • Examples • Academics: Academic achievement and attitudes that lasts a lifetime • Character: Avoidance of detrimental and dangerous behavior • Health: Maintaining a healthy lifestyle through smart nutrition and behavior These outcomes, while somewhat abstract, all contribute to the core mission of the Boys & Girls Clubs of Chicago

6. How Are Outcomes Achieved? • Outcomes are achieved by using various inputs and processes, and can be visualized by using an outcome logic model Outcome Logic Model

7. What Is The Outcome-Based Management Program? • The Outcome-Based Management (OBM) program will allow us to collect information to see how club attendance impacts member outcomes in terms of academics, health, and character We will use data from KidTrax… …to create OBM reports that can be communicated to stakeholders and help improve programs! …and match it with data provided by CPS… ADA by club / program Grades • CPS grades by club ADA / length of membership • CPS attendance by club ADA / length of membership • Character development by club ADA / program attendance • Leadership development by club ADA / program attendance • Health attitudes by club ADA / program attendance Length of membership Pre / post test results ISAT Scores Attendance Time spent by club / program School Promotion Suspension Records Survey results Outcome-Based Management Reports KidTrax Database CPS IMPACT Data

8. What Are The Benefits Of Creating The Outcome-Based Management Program? • OBM program will have wide ranging benefits at both the club level and the executive level Improve Programs and Operations Earn More Money Improve Our Members Outcome reports lead to… Outcome reports lead to… Member-level data and tracking lead to… …more information to work individually with members… …program assessments and benchmarks… …quicker, easier prep for grant proposals… …and allow us to share best practices and improve programs! …and improve their academics, character, and health! …and allow us to get more money! Remember, this is a long term initiative and benefits won’t be recognized overnight

9. How Can We Ensure This Will Be Successful? • Directors and staff members need to understand the following factors that are key to making this program a success • Directors and staff first need to understand why we’re developing this program to make sure it’s done right! Understand the benefits of OBM Ensure accurate and timely data entry • The outcomes are only as good as the data that’s entered! Understand KidTrax and its functionality Regularly use the metrics and KidTrax • To take full advantage of the OBM program, directors and staff need to know how to input the KidTrax data and run new reports! • Reviewing data on a regular basis will help you understand the power of OBM! This program is NOT intended to make your job more difficult. Instead, it will be designed to be simple, and result in more accurate data and better results

10. Test Your Knowledge – OBM Program • Which of the following is NOT an example of an outcome? • Members improve academic performance • Members attend Power Hour twice per week • Members have greater self esteem and self confidence • Members have a more active lifestyle Answer: B

11. Test Your Knowledge – OBM Program • The Outcome-Based Management (OMB) program • Utilizes data from a number of sources, including KidTrax and CPS • Will generate reports that will help improve our programs • Is the responsibility of everyone to be successful • All of the above Answer: D

12. Test Your Knowledge – OBM Program • Which of the following is NOT a benefit of the OBM program? • Improve programs and operations • Earn more money • Increase staff support • Improve our members Answer: C

13. Test Your Knowledge – OBM Program • For the OBM program to be successful, everyone will be responsible for • Making sure the members are attending all programs • Relying on their Club Directors to perform the majority of activities • Understanding the benefits and ensuring good use of KidTrax • Holding more programming than in previous years Answer: C

14. Part 2: Data Collection Requirements

15. Data Collection Processes • The data collection requirements will cover four processes related to attendance tracking and survey / test administration Attendance Surveys / Tests Club Attendance Tracking Selected Program Attendance Tracking Outcome Measurement Toolkit Administration Selected Survey and Test Administration • Tracking member attendance at clubs, including entry and exit times • Tracking member attendance at selected programs • Administering survey tool developed by BGCA on a regular basis • Administering surveys / tests developed by BGCA / clubs on a regular basis Description • Used with other metrics to see how club attendance impacts members • Used with other metrics to see how program attendance impacts members • Used with club and program attendance to analyze changes in attitudes and behaviors • Used with program attendance to analyze changes in attitudes, behaviors, and knowledge Usage

16. Club Attendance Tracking Requirements and Details Key Requirements • Every member must check in and out at front desk to participate in club activities • Club staff must scan members into KidTrax on a daily basis • Club staff must ensure that member entry and exit times are properly inputted into KidTrax Other Details • Responsibility:Front Door Volunteer / Assigned Club Member / Staff Member • Please identify a person who is trained / familiar w/ KidTrax and able to assist others entering data • Frequency: Daily • Batch Scanning: Only permitted for school-based clubs by using sign-in sheet, unless there are emergency or special circumstances • Members who enter 30 minutes after the club opens or leave 30 minutes before the club closes must have their times recorded on the sign-in sheet and be entered into the system separately • Front door resource at stand-alone clubs should use ID scanning or manual scanning in Trax Attendance Manager • Best Practices: Many clubs have created binders containing backup IDs, printed out member bar codes, or created new sign-in sheets with Time In / Out to use as backups

17. Choose Your Own Adventure – Club Attendance Tracking Front Desk, Stand-Alone Club Shoo him into the club without checking in? • Uh oh, guess we won’t be able to track that member! • You’re sitting at the front desk, texting w/ your friends, when a member walks in. You ask for his ID card, but the member forgot it! You panic and start breathing heavily. Do you…  Stop, breathe, then perform a manual KidTrax scan? • Good choice! Back to texting, until… Front Desk, School-Based Club Decide to wait a couple days to enter this data – this isn’t my club, right? • We’re one team – you should be helping out! • …you’re mysteriously transported to a school-based club a couple hours later. All the members have left and they’ve put their exit times on the sign-in sheet. One member had to leave 45 mins early, but other than that, everyone left when the club closed. Do you… Take the initiative and batch scan everyone in w/ the same entry / exit time? • You’re acting quick, but you need to include exit times too!  Wait until tomorrow morning to batch everyone except the member that left early? • Even though it’s not done the same night, you’re doing the right thing!

18. Program Attendance Tracking Requirements and Details Key Requirements • Every child must check in and out with the Program Instructor to participate in selected programs and activities • Club staff must enter program attendance within two days of program session • Batch scanning is acceptable for data entry Other Details • Responsibility:Program Instructor (work w/ Directors to get appropriate KidTrax training / assistance) • Frequency: Up to 2 days after program session • Attendance Sheets: Instructors should utilize attendance sheets provided in program planning kits • Participating Programs: On-Going Programs • SMART Programs • Power Hour • College Bound • Keystone Club • Torch Club Quarterly Programs • SMART Moves • SMART Girls • Street SMART • Passport to Manhood • Money Matters • Career Launch • Youth for Unity • Triple Play • Skill Tech / Digital Suite

19. Choose Your Own Adventure – Program Attendance Tracking  Make note of it on the attendance sheet and not to give her credit for attending? • She’s not here for long, so she doesn’t get credit • As the Program Instructor, you’ve started your lesson for the day and have all the members pumped! Suddenly, 10 minutes into it, one of the participants runs out of the room to take a phone call and doesn’t come back. Do you… Shrug it off and check her in normally– she got the gist of the lesson, right? • If her phone call’s more important, she shouldn’t be included in attendance Ignore it until your Program Director catches you? • Ignoring it only makes it worse! • Program’s over and you’re wiped out. You know you have two days to put the attendance into KidTrax, but you never learned how to use the system – spent too much time playing Guitar Hero w/ the members. Do you…  Talk with your Club Director about learning to use KidTrax? • The quick, sensible answer – good choice! Batch scan it at the end of the program with the Membership Clerk? • If he knows how to enter data, he’ll be happy to help you out up to 2 days after the session

20. Outcome Measurement Toolkit Administration Requirements and Details Key Requirements • Toolkit must be completed by the same members 3x per year during time period in October, January, and May • A set number of members must use toolkit over this period • All answers must be inputted into KidTrax Other Details • Responsibility:Club Director / Program Director / Club Manager / TEEN Reach Coordinator • Frequency / Timeline : 3 times per year (allot ~2 week; must be completed by October 24, January 18, and May 16) • Clubs may allot extra time to input the toolkits into KidTrax, but must be uploaded by dates above • Suggested administration dates: October 13-24, January 7-18, and May 5-16 • Administration: Members must complete the appropriate survey toolkit for their grade level and should be tested at least twice per year • TEEN Reach: Participants are required to use the toolkit only once per year (suggested month: January) and must complete the separate Responsible Youth section • Expectations: Clubs are expected to survey 70% of their projected ADA goals for that year

21. Choose Your Own Adventure – OMTK Administration  Schedule computer time over 3 weeks (2 won’t be enough) and schedule testing by age groups? • The earlier you can plan to administer the toolkit, the easier it will be! • As the Club Manager, it’s the first week in January and you’ve been busy planning for the new year. You look at your to-do list and *BAM* it hits you that you haven’t done anything to plan for the OMTK. Uh oh. Do you… Ignore it and feign ignorance – it’s not like anybody’s checking • Ignoring it will only get you comments on your annual review  Work with your TEEN Reach coordinator to ensure all participants have taken the OMTK? • Your TEEN Reach coordinator will appreciate it! • You’ve scheduled the OMTK administration, but know there are also TEEN Reach requirements that you’ll need to meet. Do you… Pray that everyone takes it and the Illinois Alliance doesn’t notice? • You don’t want to lose funding, do you?

22. Selected Survey and Test Administration Requirements and Details Key Requirements • Selected pre / post surveys and tests must be administered to all members 2x per program at the beginning and end of program • Surveys / tests must be administered each time the program is run • All answers must be inputted into KidTrax Other Details • Responsibility:Program Instructor (work w/ Directors to get appropriate KidTrax training / assistance) • Frequency: 2 times per program (during first and last day of instruction) • Administration: Staff may administer tests either electronically or by paper, but they are responsible for putting the information into KidTrax • Participating Programs*: • SMART Moves • SMART Girls • Street SMART *These requirements do not change any guidelines for other programs (e.g., 21st Century)

23. Choose Your Own Adventure – Selected Survey and Test Administration Call up the service center to have them send it to you? • You don’t want to bother them with that, do you? • As a Program Instructor, you’re preparing materials for your first class and you realize that you completely forgot about the pre-test. You don’t have access to computers, so you have to do it by paper. Do you…  Access the BGCC intranet and print it out from the OBM section? • The quick, easy, sensible solution! Sulk for a day, then spend the next two days curriculum planning, trying to put the test out of mind • Don’t despair – your participants will do great! But first get those tests into KidTrax… • You’ve collected all of the tests and it looks like you have some work ahead of you (average score: 4 out of 10). Frustrated, you go home for the night and…  Make sure that you enter the test data in KidTrax 1-2 days after it’s given, then work on the lesson plans • There’s still plenty of time for planning – entering data in KidTrax only takes a few minutes!

24. Enforcement and Staff Impact • To make sure we meet these new requirements, the service center will monitor clubs’ data on a regular basis and take appropriate actions to ensure data is entered accurately and timely Enforcement Staff Impact Data Collection Process • These requirements are expected to be met by all staff members and will be a part of the performance review process • Clubs that are not in compliance will have a letter sent to the Club Director and Board President • Staff disciplinary action will be taken for non-compliance in accordance to existing policies and practices Reviewed by service center on an on-going basis to ensure data entered properly Club Attendance Tracking Reviewed by service center on an on-going basis to ensure data entered properly Selected Program Attendance Tracking Reviewed by service center after toolkit administration period to ensure clubs are capturing appropriate % of members Outcome Measurement Toolkit Administration Reviewed by service center on a quarterly basis to ensure that program survey being administered to all participants Selected Survey and Test Administration

25. Requirements Summary Key Takeaways • The key to this program is data – make sure it’s captured accurately and timely! • Everything must be entered into KidTrax – no exceptions! • The Data Collection Requirement documents provide additional details, process flows, and best practices – review them for more information! Suggested Practices • Make data collection a priority and try to get into a routine (i.e., enter data right as you get in or right as you leave • Club Directors / Managers, review requirements with the staff and put it on the weekly agenda • Plan to schedule a staff meeting within the next week to review these requirements Many of these requirements and processes should be familiar, helping to make this a smooth transition

26. Test Your Knowledge – OBM Requirements • Batch attendance entry is permitted under all of the following normal scenarios EXCEPT • Entering club attendance at a school-based club • Entering club attendance at a stand-alone club • Entering program attendance at a school-based club • Entering program attendance at a stand-alone club Answer: B

27. Test Your Knowledge – OBM Requirements • Which of the following statements about program attendance is true? • Program Instructors may enter attendance information up to a week after the session • Batch scanning of program attendance is NOT permitted • Program attendance must be entered into KidTrax for all club programs • Instructors should utilize attendance sheets that are part of the program planning kit Answer: D

28. Test Your Knowledge – OBM Requirements • It is required that the Outcome Measurement Toolkit be administered to • All members • 50% of projected ADA • 70% of projected ADA • As many members as possible Answer: C

29. Test Your Knowledge – OBM Requirements • Compliance with these requirements may result in all of the following EXCEPT • Increase in the number of club members • Exemption from disciplinary actions related to OBM • Annual incentives for staff members and clubs • Positive marks on performance reviews Answer: A

30. Part 3: Immediate Action Items and Resources

31. CPS Member and Parent Consent Forms • Each member who attends CPS will be required to fill out consent forms based on their age for us to obtain their CPS information Age <12 Ages 12 - 17 Age >18 Parent Consent Form Parent Consent Form Member Consent Form Member Assent Form Once completed, send forms to the service center with membership applications Please encourage members to complete and submit forms to the club within ten (10) days of their distribution

32. Student ID Number Collection Process • Clubs must use school rosters and report cards from local CPS schools to allow staff to enter student ID numbers on the consent forms if they are not provided • CPS student ID numbers can be found in three places: • Student Rosters • Student ID Cards • Report Cards • Club Directors should work with school principals and administrators to collect student rosters • A letter will be provided by the service center to help explain the importance of the program and how it can assist the school in improving its students’ performance • If available, clubs may also utilize report cards that were collected last year Student ID Number Collection Student ID Number Entry Staff will need to assist members by entering their CPS student ID number on the consent form if they are not able to provide it Note that this will be a one-time process and the service center will follow-up later in the fall to obtain ID numbers that are still missing

33. Survey Best Practices • There are a number of best practices you can use for administering surveys and tests to members • Prepare all materials prior to the scheduled survey. If using hard copies, keep in mind kids often respond better to surveys printed on colored paper • Leave enough time to set up and arrange the room appropriately prior to the members’ arrival • Introduce yourself to the members as they arrive and give a brief description of your purpose for being there and administering the survey. Sample script: • I am here to ask you to help make our clubs better. The survey your are about to take is being used to get your ideas about our club. It is not a test and there are no right or wrong answers — we want to know what your experiences and opinions are. Take your time and be sure to answer each question based on what you really think. If you don’t understand a question, please raise your hand or come find me and I can help you. The survey should take you only 8 to 10 minutes. (May differ depending on survey length).” • Request that members attempt to answer every question so their opinions and experiences can be included • Let members know you are here to help them if they have any questions • While members are completing the survey, quietly walk around the room (do not hover), answer questions and make sure things are running smoothly • When youth finish the survey, thank them for their time and provide incentives, if appropriate Source: The Colorado Trust, ” Toolkit for Evaluating Positive Youth Development”

34. BGCC Intranet Resources • The BGCC Intranet will have many resources available, including BGCC Intranet Pathway: Downloadable Documents > Outcome-Based Management Program Resources Available • Program Surveys / Tests (should be used if administering via paper copy) • Outcome Management Toolkit • Parental / Student Consent Forms • Requirement Documents • KidTrax Instruction Guides • Presentations • Other Forms and Guides This will be your one-stop shop for all things OBM!

35. Conference Call and Meeting Schedule • To support the training you have today, we will hold a conference call and be present at the club manager meeting next week to answer any additional questions and concerns • Tuesday, September 16 • 11:30 am – 12:30 pm • Dial In: 888-998-2663 • Passcode: 6235253 Conference Call Club Manager Meeting • Wednesday, September 17 • 12:00 pm – 2:00 pm • Location TBD We will also distribute a Frequently Asked Questions (FAQ) document based on your feedback!

36. Contact Information • Please contact the following people if you have any comments or questions regarding the program Primary Contact (Until September 19) • Louis Levine (Deloitte), (312) 513-8166, llevine@deloitte.com Additional Trainers / Contacts • George Colone (McCormick): (773) 271-8400 x 4307, gcolone@bgcc.org • Khari Humphries (Holy Angels): (773) 268-4819, khumphries@bgcc.org • Dawn Jimenez (Park Manor): (312) 520-6731, djimenez@bgcc.org • Sidney Johnson (King): (773) 638-5464, sjohnson@bgcc.org • Jim Pottinger (Cotter): (773) 348-1866, jpottinger@bgcc.org • Erin Rochford (Valentine): (773) 927-7373 x4713, erochford@bgcc.org • Juan Saenz (Logan Square): (773) 342-8800 x4210, jsaenz@bgcc.org • LaTrice Smith (GSC): (312) 235-8007, lsmith@bgcc.org • Katie Huckaby (GSC): (312) 235-8008, khuckaby@bgcc.org • Ezohn Smith (GSC): (312) 235-8006, esmith@bgcc.org

37. Test Your Knowledge – Action Items • Members between the ages of 12 and 17 should receive which of the following forms? • Student Consent Only • Parental Consent Only • Student Assent Only • Parental Consent and Student Assent Answer: D

38. Test Your Knowledge – Action Items • Staff should administer surveys and tests related to the OBM program • Printed out from the KidTrax Survey Manager • Found on the BGCC intranet • Provided by TEEN Reach coordinators • Part of the Program Planning Kit Answer: B

39. Test Your Knowledge – Action Items • Which of the following is NOT a best practice for administering a survey? • While participants are completing the survey, quietly walk around the room to make sure things are running smoothly • Provide incentives for members to complete the survey • Do not require members to complete all questions in the survey • Prepare all materials prior to the scheduled survey Answer: C

40. Break

41. Part 4: KidTrax Training and Refresher

42. Manual Scanning (Trax Attendance Manager) • Purpose: Track club attendance as soon as member enters / exits stand-alone club • Situation: When member forgets his or her card and there is immediate access to Trax Attendance Manager at the front desk • Responsibility:Front Door Volunteer / Assigned Club Member / Staff Member Refer to Manual Scanning Instruction Guide

43. Batch Attendance Tracking (KidTrax) • Purpose: Track club attendance at school-based sites and program attendance after session completed by Program Instructor • Situation: After Program Instructor runs session, will need to record attendance for the following programs: On-Going Programs • SMART Moves • Power Hour • College Bound • Keystone Club • Torch Club • Responsibility:Program Instructor • Career Launch • Youth for Unity • Triple Play • Skill Tech / Digital Suite • Quarterly Programs • SMART Girls • Street SMARTS • Passport to Manhood • Money Matters Refer to Batch Attendance Instruction Guide

44. Survey Setup (KidTrax Survey Manager) • Purpose: Setup Outcome Measurement Toolkit, survey, or test to collect results from members in KidTrax • Situation: Staff needs to setup survey / test for member to either take directly on the computer or be entered into KidTrax by a staff member • Responsibility:Club Director / Program Director / Club Manager / TEEN Reach Coordinator / Program Instructor Refer to Survey Setup Instruction Guide

45. Survey Administration (KidTrax Survey Kiosk) • Purpose: Administer Outcome Measurement Toolkit, survey, or test in KidTrax to collect results from members • Situation: Staff needs to administer survey / test to members directly through the computer or access surveys and enter data on behalf of the members • Responsibility:Club Director / Program Director / Club Manager / TEEN Reach Coordinator / Program Instructor Refer to Survey Administration Instruction Guide

46. Survey Results (KidTrax Survey Manager) • Purpose: Track which members have completed a survey and review results of the assessment • Situation: Staff needs to review this information during the testing period to ensure all members have completed the assessment and review results after testing completed • Responsibility:Club Director / Program Director / Club Manager / TEEN Reach Coordinator / Program Instructor Refer to Survey Results Instruction Guide

47. Q&A