Post-school Data Collection: Training Interviews

1. Post-school Data Collection: Training Interviews Potsdam Institute for Applied Research (PIAR) At SUNY Potsdam, J. Pat Turbett, Director Center for Change in Transition Services (CCTS) At Seattle University, Cinda Johnson, Principal Investigator

2. New York Potsdam Institute for Applied Research (PIAR) At SUNY Potsdam J. Pat Turbett, Director 315-267-2718 piar@potsdam.edu

3. New York State Urban-Suburban-Rural 700+ districts ~3.3 Million Students… ~1.1 Million in NYC ~25,000 exits Special Education each year ~8,000 from NYC 2005-2006 SPP#14 includes 107 + NYC (73 districts had 25 or fewer in their survey pool)

4. Central Interviewing/ Call Center What works well (strengths) • Efficient to train and supervise staff • Consistent, onsite, training and supervision • Interviewing 7 days a week: morning-evening • Security • Simplifies mid-survey modifications • Improves interviewer and respondent objectivity • Computerized onsite management tools • Consistent messages/T.A. to districts

5. Training to interviewers ***Voluntary – Confidential – Accuracy*** • Required Online Course (short) : SUNY RF • Conducted by PIAR Staff • At hire, and regularly throughout survey cycle • Provide and Review an Interviewers Manual • Expectations/ Responsibilities/ Schedules • Orientation to the Computer based surveys • Practice – Practice – Practice

6. Training to interviewers ***Voluntary – Confidential – Accuracy*** • Protecting respondent rights • Initiating the survey • Consent • Strategies for objective interviewing • Consistent survey administration • Disability awareness and appropriate accommodation strategies

7. Survey Formats • Computer assisted phone survey: • CASES Software • Web-based survey • Mail Survey • English • Spanish • Large Print

8. Contact Rates • Statewide: 65% • New York City: 52% • Rest-of-State: 69% • Districts with 10 or more in the survey pool: • 27% to 90% • ED: 52% • Minority: 50% Dropped Out: 42%

9. Reporting data • All data are securely stored and backed up onsite • Automated and customized reports • Preliminary reports are prepared for NYSED • Final Reports are generated for NYSED • Demographic Representativeness • Response Rates by Demographic Characteristic • Statewide, District & Sub-population Reports

10. Limitations and areas of improvement • Interviewers are calling all over the state and may not understand all local references. • Respondents do not know SUNY/interviewers and are sometimes suspicious or reluctant to participate. • We have a limited number of interviewer stations and equipment • It is difficult to find interviewers who speak every language spoken by respondents/families.

11. Limitations and areas of improvement: General • Out-of-Date Contact Information provided by districts is more of a factor than refusals • Students with more stable lives are more likely to be interviewed during the federal survey cycle. • Improvement: testing earlier contact and surveying to improve response rates among at risk youth

12. Washington State Center for Change in Transition Services www.seattleu.edu/ccts Seattle University Cinda Johnson, Principal Investigator 206-296-5888 cinda@seattleu.edu

13. Washington State: Census Data • All districts, all leavers • Districts with high schools: 248 • Graduates and drop-outs: 5,000 plus • 80% contact rate with all leavers

14. Post-school Outcome Data • 12th year of post-school data collection • Consistent survey with district input • Paper/pencil to web-based survey • Districts use data for LEA application for funds • Districts develop goals for program improvement

15. Training for interviewers • Assist districts in developing system for data collection • Training in person, video, teleconferencing and web-based • Expectations, responsibilities, timelines • Training to the computer-based surveys • Voluntary and confidential

16. Training to interviewers • Protecting respondents’ rights (IRB) • Assuring consent • Identifying respondent • Consistency of interview • Timing the calls and initiating the survey (at least 3 attempts at different times of day) • Accommodation strategies, LEA identifies

17. Training Activities • Director’s meetings (fall and spring) • Conferences • State department of education monthly bulletins • Email list serve • CCTS website

18. Post-school Outcome Survey • Web-based survey • Training modules online • Research coordinator at CCTS • District manager • District users

19. Reporting data • All data are securely stored and backed up onsite • Reports provided for SPP, state, districts and agencies • Disaggregated by disability, gender, ethnicity and race, urban to rural, educational region, county. • Comparison to state level data and by year.

20. Suggestions for Improvement • Assure that emergency contact information is up to date and reliable. • Be prepared to offer resources and information to former students and families. • Share post-school data beyond special education personnel.

21. Conclusion: The Top Ten… Things you should never do as an interviewer!

22. Top Ten… • Interview the wrong person. • Make up questions and codes of your own. • Talk to other people while on the phone (other than people you should be interviewing). • Talk so fast that even you don’t know what you are asking. • Tell someone how much you like them or their answers.

23. The Top Ten, continued… • Tell your friends how silly someone’s answers were; it might have been them you were talking to. • Tell an anonymous participant you like being their neighbor and once dated their brother. • Laugh at any answers. • Accept offers of dates from participants (or propose one). • (Okay, top 11!) Swear or use slang or make sexual comments (unless they are in the survey).