240 likes | 385 Views
Trying Out New Survey Techniques – Lessons Learned . 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the Program Director June 25, 2013. The Local Hazardous Waste Management Program in King County.
E N D
Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the Program Director June 25, 2013
The Local Hazardous Waste Management Program in King County A regional partnership to protect and enhance public health and environmental quality by reducing the threat posed by the production, use, storage and disposal of hazardous materials.
Program Partners • 37 Cities and 2 tribes in King County • King County Water & Land Resources • Public Health--Seattle & King County • Seattle Public Utilities • King County Solid Waste
King County, Washington 1.92 million residents 799,000 households 62,000 businesses 2,134 square miles 14th most populous county in the U.S.
2012 Survey Goals • Updated snapshot of residents’ attitudes,awareness and behavior about household hazardous materials and waste. • Update on who is using our collection services, and how satisfied they are with the services. • Better understanding of why people are not using the facilities and how to better reach and motivate them to dispose of hazardous wastes properly.
Considerations • Last surveys conducted 5 years ago. • Need for statistically valid samples. • Need to align with increasingly diverse demographics. • Particular concern with input from South King County.
Approach Three complementary surveys: • General survey of residents throughout King County • Focus groups of special populations in South King County • Survey of collection customers
Decision to Try New Methods • General residential survey using address based sampling instead of random-dialed phone calls • Oversampled the non-Caucasian population in South King County • Supplemented with focus groups
What is Address Based Sampling? • Obtain a random sample of addresses in the area you are surveying. • Mail a letter or postcard to the selected addresses. • Provide information about how to respond. • Analyze the responses you receive. • Potentially can adjust the sample.
Final Survey Design: • Worked with consultant to design an approach that would meet survey goals. Survey much shorter than 2007. • Letter and paper survey mailed to 10,000 randomly selected households. • Multiple ways to respond: • mail back paper survey (postage provided) • On-line survey in English or Spanish • Toll-free phone number in English or Spanish
Final Survey Design: • Oversample of 300 non-Caucasian households in South King County. • Telephone calls to targeted phone numbers (cell and land line) • Monitored responses to ensure demographic alignment. • Built in contingency for random dialed phone calls if needed to get a demographically aligned sample.
High Level Findings (n=1,270) • Generally people are using the collection sites in their region. • Most people say they don’t have hazardous waste. • Slightly under half say they know about sites; about one third say that they have used them. • Strong support for importance of government providing these services from both collection site users and non-users.
South County Focus Groups • Selected 4 non‐English speaking communities based on demographic data: • Spanish‐speaking • Vietnamese • Filipino • (Tagalog‐speaking) Korean Focus Group • Korean • Conducted 5 focus groups (two Hispanic) recruited by and facilitated by a multi‐cultural, multi‐lingual team.
South County Focus Groups • Focus group leaders worked to recruit a diverse mixture of people in each 15 person focus group. • Conducted 43 individual interviews in the African American community. • General survey questions were modified for focus groups and interviews. • Materials were translated into the relevant language by each focus group facilitator.
Focus Group Findings • Results can’t be generalized due to small sample size. • Communities differ in their approach and level of concern. • Common perception that small amounts of hazardous waste improperly disposed don’t make a difference. • Widespread confusion about what is hazardous. • Confusion about solid waste disposal and hazardous waste disposal.
Focus Group Findings • The Spanish community members did not know about the services that we provide, despite our two year Hispanic outreach campaign. • The elderly were unaware of the home collection services we provide. • Generally that services are inconvenient and should be expanded. • The results help illuminate some of the responses in the other surveys.
Lessons Learned • Project was too complex with too many moving parts. • Project management schedule was too ambitious. • Both consultants sought revised budgets to cover core activities. • There were problems with communications, oversight and work quality with both consultants. • Schedule regular face to face meetings with consultant at key checkpoints. • Make sure the consultant understands what level of statistical analysis you are looking for and other expectations. • Make sure consultant includes adequate time and budget for writing the report.
Lessons Learned - Next Time • Don’t schedule the General Survey and the Focus Groups at the same time. • Develop a realistic project schedule. • Make sure that the consultant includes adequate time and budget to cover: • Needed meetings • Analyzing the results • Writing the report • Schedule regular face to face meetings with the consultant at key checkpoints.
Lessons Learned - Next Time Make sure that the consultant understands: • The type and level of analysis that you expect. • They need to translate technical presentation of statistics into commonly understood terms. • They are expected to meet standard report requirements. • An outline of the report must be approved prior to drafting it. • The first draft should not have grammar or spelling errors and should be close to final.
Lessons Learned – Next time • Develop focus group questions before or after the general survey questions, not at the same time. • Work closely with the consultant to ensure that concepts are accurately translated and are asked consistently. • Interview consultants to be sure that consultant team members have adequate English proficiency and an understanding of the content that we are trying to collect.
2012 Surveys and Key Findings: http://lhwmp.wordpress.com/category/publications/
Questions? For more information: LizTennant@kingcounty.gov 206-284-7974