2014 National Outcomes Survey Site-Level Survey A dministrator Training

1. 2014National Outcomes Survey Site-Level Survey AdministratorTraining

2. National Outcomes Survey: Overview(Training Guide, page 4) • Measuring Outcomes is essential! • National Outcomes Survey measures indicators of: • Academic Success • Good Character & Citizenship • Healthy Lifestyles • Member perceptions of the Club • The survey works in tandem with your NYOI data submissions

3. National Outcomes Survey: Overview(Training Guide, page 4) • Two versions (youth and teen) • Two languages (English and Spanish) • Two options for administration method (paper booklet or web) • Additional modules on web-based surveys • Voluntary and confidential • Parental consent is required

4. Parental Consent • Consent is required • Passive or active • Templates provided by BGCA • Distributed at least 2 weeks prior to survey administration • Keep copies of the survey on file for parents to review at the Club.

5. Consent Management Template (Training Guide, page 18) • Keep a record of which members are allowed or are not allowed to take the survey. • Determine the easiest way to track this for your Club. • This example shows lists members who do not have parental consent to take the survey – best for Clubs using passive consent.

6. Overview: Timeline of Responsibilities (Training Guide, pages 5-6) Responsibilities BEFORE Survey Administration, January & February 2014 Club Site Responsibilities  Manage consent lists Attend survey administration training by Org or watch BGCA training video  Keep surveys on file or at front desk for parental review  Complete a Survey Administration Plan (p. 17)  Prepare Member ID information (see tips on p. 8)  For sites using web-based survey: Test each computer for access, firewall blockage, etc.

7. Overview: Timeline of Responsibilities, (Training Guide, pages 5-6) Responsibilities DURING Survey Administration, March 2014 Club Site Responsibilities  Administer Survey  For sites using paper booklet surveys: Seal and return ballot boxes to SRBI as soon as survey administration is complete.

8. Overview: Timeline of Responsibilities, (Training Guide, pages 5-6) Responsibilities AFTER Survey Administration, April -June 2014 Club Site Responsibilities • Retain consent lists • Return all sealed ballot boxes by April 4, 2014 Returning paper booklet surveys: Drop off boxes at any FedEx location OR Call 1-800-GOFEDEX and request a ground shipment pickup. Consider coordinating all sites’ return for your organization.

9. The Basics (Training Guide, page 7) When do I survey?: March 3-31, 2014 (Clubs that use the web survey have until April 4!) YOU determine the 2 weeks that work best for you Who do I survey?: As many young people as possible during those 2 weeks Diversity of youth important About 80-100% of ADA How do I administer the survey?: Internet or Paper Booklet versions, indicated at registration What else?: Trained survey administrators necessary Surveys take 35-45 minutes Can be administered in a typical program period or special time slot Survey in small groups, 15-20 members at a time Youth and Teen surveys administered separately Use Daily Checklists to ensure all materials are organized for each day

10. Preparing Membership Number Lists(Training Guide, page 8) Critical step: Members must fill in their date of birth and membership number! • Make this process easier by preparing this info ahead of time • Organize this info logically so that it’s easy to distribute • Check the Survey Resource Hub for instructions on how to create a member list from your member management system with this info TIP! Consider having teens available to help younger members enter this information

11. Other Preparations & Member Assistance (Training Guide, p. 10) Members will need help during the survey, so set yourself up for success! Familiarize yourself with survey items Learn how to provide help when a member does not understand a question • Repeat the question, • Reorder the words in the question, then • Define the words. Do not elaborate by giving personal or program-specific examples. Learn answers to some frequently asked questions Q: Why should I answer these questions? Q: Why do you ask the same question over and over? Q: Who will see what I answered? Believe in the importance of the survey

12. Reading Surveys Aloud (Training Guide, page 9) Survey Instructions should be read aloud to every group taking the survey • reinforces importance, voluntary & confidential nature • provides standard instructions for everyone Instruction scripts are provided in your training guide (web - p. 13; paper – p. 16) Survey questions should be read aloud for all youth, ages 9-12 • Paper Booklet version– Questions are read by trained survey administrator • Web-based version – Questions are read via online audio recording (headphones) or by trained survey administrator (no headphones)

13. Reading Surveys Aloud, cont’d(Training Guide, page 9) Pacing for reading questions > In general, you should be able to read about 4 questions per minute > However, adjust your pace to the group’s comfort level Reading Answer Scales > Each time a new answer scale appears, read the scale aloud for 2-3 questions > After that, repeat the scale in your head to maintain a consistent pace for reading questions > Approximately every 5th question, read the answer scale aloud to remind members of what the scale choices are.

14. Pacing Example

15. Test Computers for Web Survey (Training Guide, page 5) TIP FROM THE FIELD: “Do a check on each computer being used to avoid any problems on day one of the survey. It plummets morale both for the kids and the staff to not be able to efficiently get kids logged on and going.  One day of mishaps can result in kids not going in and the lead staff feeling deflated and unenthusiastic.” Sites using the web-based survey should test that the survey will work on each computer that members will use for the survey. The below is a test link; it is not the real survey. http://confirm.opinionport.com/wix/p8091960.aspxClick through the screens to determine if you have an access problem. Log your computer tests using the template provided on p. 19 of your training manual.

16. Sample Computer Test Log, (Training Guide, page 19)

17. Web-Based Survey (Training Guide, pages 11-13) Web Survey Checklist Each day, the survey administrator should have the following:  Consent lists  Membership number lists/index cards  Survey URL  A printed copy of each survey (youth and teen) WHY?? If a young person asks you a question about a particular survey item, you will want to reference your printed our version instead of reading the question on his/her screen to promote privacy. • Headphones • Web Survey Directions & Script

18. Survey privacy is important!

19. Web-Based Survey (Training Guide, pages 11-13) Web Survey Directions Do not survey more than 15-20 members at a time. Only survey members that have consent. Survey teen and youth members separately as survey questions/content differ. Before members enter: Consider taping folders or cardstock to the sides of monitors to give members a sense of privacy. Direct each computer’s web browser to the URL After members enter: Seat one member to one computer; no sharing computers. Distribute membership numbers. Members will need to put their membership numbers on the survey. Read the web survey instructions script aloud, even if you are using headphones for audio.

20. Paper Booklet Survey Materials (Training Guide, page 14) • Paper surveys are shipped to the organization • Each site box includes: • A printed copy of each survey (youth and teen) • Survey administration guide • Paper survey booklets • Survey privacy envelopes • The “Ballot Box” • Do not photocopy surveys!

21. Paper Booklet Survey Materials: The Ballot Box (Training Guide, page 14) • The Ballot Box provides members with survey confidentiality AND acts as a return shipping box. • The box will include stickers with instructions on • how to fold to construct the box, • where to insert the surveys, and • how to close the box for return shipping. CAUTION: Only seal the box at the end of the survey administration period. Once the box has been sealed, it is ready to be returned.

22. Paper Booklet Survey (Training Guide, page 15) Paper Booklet Survey Checklist Each day, the survey administrator should have the following:  Consent lists  Membership number lists/index cards  Surveys  Privacy envelopes  Ballot box • Blue or Black Pens (no pencils, no other color ink) • Paper Booklet Survey Instruction Script

23. Paper Booklet Survey(Training Guide, page 15) Paper Booklet Survey Directions • Make sure that you have everything that is listed on the paper booklet survey checklist. • Do not survey more than 15-20 members at a time. • Only survey members that are on the consent list. • Survey teen and youth members separately as survey questions/content differ. • When you are ready to begin: • Space out your members to discourage youth from looking at each other’s responses • Distribute membership numbers. Members will need to put their membership numbers on the survey. • Distribute survey booklets, pens (blue or black) and privacy envelopes. • Read the paper booklet instruction script aloud

24. Survey Incentives(Training Guide, page 20)

25. Survey Administration Plan Template (Training Guide, page 17) Each site should complete a survey administration plan.

26. http://srbireports.com/bgca/splash.htm

27. Final Considerations (Training Guide, page 21) • Make sure members understand why you are doing the survey • Don’t administer the survey right after school • Think about ways to make this a positive experience • Make sure each young person has privacy • Keep good track of consent lists

28. Next Steps 1. Complete a Survey Administration Plan 2. Manage consent lists • Final lists compiled one week before survey 3. Print out and organize membership numbers • Do this a few days before survey • Instructions available on Survey Resource Hub 4. For sites using web-based survey: Test survey URL for access, firewall blockage, etc.