100 likes | 197 Views
Objectives of Session Three. Focus Group and Survey Assignment Formative Design Proposal Construct Validity Reliability in Data In-class Data Exercise. Focus Group/Survey Project. Select topic and moderator(s) ½ hour – no penalty if you do not finish your guide
E N D
Objectives of Session Three • Focus Group and Survey Assignment • Formative Design Proposal • Construct Validity • Reliability in Data • In-class Data Exercise
Focus Group/Survey Project • Select topic and moderator(s) • ½ hour – no penalty if you do not finish your guide • Short ice-breaker, background on topic, and questions • Be prepared with follow-up questions • Handout relating to survey design
Sources for Formative Design Proposal • Conduct an internet search using keywords for your program area of interest and evaluation terms • Points of Light Foundation -http://www.pointsoflight.org/ • United Way - http://national.unitedway.org/ • Electronic Policy Network -http://movingideas.org/ • Evaluation Center -http://www.wmich.edu/evalctr/index.html • Michigan Non-profit link - http://comnet.org/index.html • Community Foundation of CNY - http://www.cnycf.org/nonprofit/grantees.cfm
Construct Validity • Does the measure capture the construct of interest? How well does the measure capture the process at hand? -Face validity -Content validity -Convergent v. Discriminant validity
Achieving Construct Validity • Propose different possibilities for valid measures or operationalizations of the following concepts: • Quality of an individual’s diet • Customer satisfaction with visit to the Dept. of Motor Vehicles • Racial or ethnic identity • Physical limitations
Reliability • Free of error • Consistency in measurement and in coding • Data cannot be valid unless it is reliable, but reliable data can be invalid.
Internal and External Validity • Internal Validity:Does the design allow us to reach causal conclusions? • External Validity: Are your findings generalizable?
Improving Validity and Reliability of Data • Pretest your questions or data collection techniques • Use widely trusted measures • Correlations between coders/respondents • Develop multiple measures • Use qualitative findings to check quantitative findings and vice versa
In-Class Data Exercise • Data collected from Open Cupboard, nonprofit that opened 10 food pantries 6 months ago • What can the data tell us about service delivery? • Different measures of initial implementation and service delivery -- no impact or outcomes • Frequencies • Means and variance • Correlation • Which measures are most valid? Most reliable?