90 likes | 225 Views
Janeen Buck Jeffrey Butts October 23, 2002. National Youth Court Center Evaluation Workshop. National Youth Court Seminar on Funding and Evaluating Teen Courts - Indianapolis, IN. 5.0 Data Collection and Measurement . Data Qualitative vs. Quantitative Sources Measurement
E N D
Janeen Buck Jeffrey Butts October 23, 2002 National Youth Court CenterEvaluation Workshop National Youth Court Seminar on Funding and Evaluating Teen Courts - Indianapolis, IN
5.0 Data Collection and Measurement • Data • Qualitative vs. Quantitative • Sources • Measurement • constructing questions • surveys and questionnaires • Data collection • logistics and strategies • legal considerations
Benefits Problems 5.1 Data Sources and Methods • Often in database form • Measures of official recidivism and program compliance • Allows for large samples • Database structures can be complex • Restricted access • Recidivism measures are biased Official Records (police, court, etc.) Face-to-face interviews Written surveys • Rich, personal data • Measures of subjective perceptions and attitudes • Useful for theoretically oriented evaluations • Very expensive • Requires prof. experience • Increases respondent resistance and attrition • Incentives ($$) needed • Can be highly objective • Measures of subjective perceptions and attitudes • Good response rates • Requires good design • Requires prof. experience • Limited range of variables
5.2 Measurement Issues • Relevance and precision • Simplicity • Balance • “Road-test” measures
Question Wording is an Art Strongly Don’t Strongly Disagree Disagree Know Agree Agree Question #1:I can always talk to my teachers about problems with my school work. SD D DK A SA Problem:Terms such as "always" and "never" create response bias. If respondents can talk to teachers 90% of the time, which answer should they choose?
Question Wording is an Art Strongly Don’t Strongly Disagree Disagree Know Agree Agree Question #2:Community work service and restitution payments are effective with youth. SD D DK A SA Problem:Which part are respondents answering, community service or restitution? They may agree with one and not the other, but the evaluator will never know. The question becomes useless from a research point of view.
Question Wording is an Art Strongly Don’t Strongly Disagree Disagree Know Agree Agree Question #3:The courtroom is hectic during teen court. SD D DK A SA Problem:Question employs a negative image. Respondents may think their courtroom is busy, but not “hectic.” Researchers won’t know what a “disagree” response actually means.
5.3 Data Collection, 5.4 Consent • Developing a plan • MOUs • Informed consent • What? • Why? • When?
6.0 Working with an Evaluator • Objectives and expectations • Selection process • Request for Proposal (RFP) • “Goodness of fit” • Products and services