170 likes | 279 Views
Cost Effective Evaluation Strategies. Robin Kipke Tobacco Control Evaluation Center rakipke@ucdavis.edu November 19, 2009. Topics We’ll Cover. Adapting existing instruments Saving time in data collection Tapping into volunteer labor Using TCEC resources. Caveat.
E N D
Cost Effective Evaluation Strategies Robin Kipke Tobacco Control Evaluation Center rakipke@ucdavis.edu November 19, 2009
Topics We’ll Cover • Adapting existing instruments • Saving time in data collection • Tapping into volunteer labor • Using TCEC resources
Caveat • TCEC is not advocating for cutting evaluation corners that result in a decline of rigor or quality of data! • Cost-cutting strategies should only be used as the context of other factors allow • Always think first about what data will be convincing to your target audience
Adapt Existing Instruments • Not always necessary to reinvent the wheel • Adapt other data collection instruments for your project’s purposes • Start with high quality instrument • Add questions you need data on • Delete unnecessary questions • Make sure it will collect what you need
Instrument Sources • TCEC website has model instruments on limited number of objectives • Contact TCEC to access repository of instruments created/used by other projects • Ask other projects working on similar objectives to share their instruments
Statewide Sites Download instruments from statewide sites • CA Health Interview Survey – http://www.chis.ucla.edu/questionnaires.html • CA Healthy Kids Survey – http://www.wested.org/cs/chks/view/chks_s/17?x-layout=surveys • CDC – http://cdc.gov/tobacco/data_statistics/surveys/ • YTPS on STORE Website – http://www.tcsstore.org/stages/1_document/t_yps.pdf
Saving Time on Data Collection • Ask to have your questions added to data collection activities being conducted by other departments • Pay attention to what’s going on within your county or agency • Find out when other projects/divisions will be collecting data • Learn which population groups will be targeted by data collection and see how that fits your needs
Adding Your Questions • Obtain a copy of the instrument to see what questions are already being asked • Insert your most important questions • Add transitions as needed to preface your question topics • Examples: • County’s annual Healthy Kids survey • Maternal & Child Health survey (every 5 years) • Needs assessments (use of Prop. 63 funds, adult health survey)
Some Limitations • Think about data sources – who will be reached/excluded? • Will not be a pre-/post measure of same people, so cannot measure intervention effect • Will be limited to space allowed • Access to analysis – may need raw data
Saving Time on Data Collection • Distribute questionnaires/materials via high traffic locations • Back-to-school packets • Church service programs • Grocery store flyers • Classroom teachers • Entertainment tickets (e.g. movie theaters) • Organizational newsletters
Other Distribution Avenues • Online surveys (Survey Monkey) • Invite participation via email, social networking media (Facebook, Twitter) • Your organization’s website (and those of coalition members, agencies that serve target population)
Other Distribution – Limitations • With each distribution outlet, think about who will it reach/not reach? • How to address representativeness • Use screening questions to avoid duplication • Loss of control over administration • Potentially lower response rate
Saving Time on Data Collection • When low population density makes collecting a sufficient POS sample difficult, a series of focus groups (with purposive samples to ensure representativeness) may suffice • Caveat: First find out if the data will be convincing enough for your purposes
Tapping into Volunteer Labor • Develop relationships with local universities, high schools, programs or clubs • Recruit students looking for research experience to do data collection • Offer training, internships, course credit, other incentives
Tapping into Volunteer Labor • Make use of community volunteers and coalition members • To field test instruments/materials for appropriateness to data sources • To translate instruments/materials into other languages • To collect data • To provide access to other distribution points
Utilize TCEC Resources • Online coalition satisfaction survey administration and analysis service • Additional model data collection instruments on a variety of topics will be developed in coming months • Model instruments will be translated into other languages (Spanish, Vietnamese, Hmong and Russian) depending on need • We’re working with other statewide projects to develop tools that will help you use and share your data more effectively for your policy work
Contact TCEC For individualized assistance with: • evaluation planning • development of data collection instruments • advice on analyzing & interpreting data • guidelines for reporting & sharing info