680 likes | 3.62k Views
Data Gathering Procedures. Two dimensions on which we can distinguish data gathering procedures The degree of contact between the researcher and the participant The extent to which participants are given an opportunity to respond. Data Gathering Procedures.
E N D
Data Gathering Procedures • Two dimensions on which we can distinguish data gathering procedures • The degree of contact between the researcher and the participant • The extent to which participants are given an opportunity to respond
Data Gathering Procedures • Dimensionalizing direct contact / response-gathering procedures
Direct Contact / Direct Response • Different methods of interaction • Whether or not measure was disguised in its intent • The degree of structure imposed on participants’ responses
Direct Contact / Direct Response • Cross-tabulation of interaction types between researcher and participant More Structured Less
Direct Contact / Direct Response • The status of disguised measures • Peak of popularity was in 1950s and 1960s • A period when social sciences were obsessed with negative attitudes and behaviors like prejudice, obedience, conformity • Techniques emphasized control and structure, couple with deception • Techniques were probably oversold by researchers of the day • Less useful and predictive than non-disguised measures • “If you want to know something about someone, ask them; they might just tell you” (Kelly, 1955)
Direct Contact / Direct Response • Non-Disguised measures • Questionnaires and interviews • Means of questionnaire administration • Self-administered • Group-administered • Mail out • Means of interview administration • In-person interview • Focus-group interview • Telephone interview • Degree of structure • Open-ended questions • Close-ended/forced choice questions
Direct Contact / Direct Response • Advantages and disadvantages to interviews and questionnaires Disadvantages Advantages
Direct Contact / Direct Response • Rating and Likert Scales • Rating scale: All things considered, how satisfied or dissatisfied are you with your life as a whole right now? 1 2 3 4 5 6 7 Completely Completely Dissatisfied Satisfied • Likert (Likert-type) Scale: All things considered, I believe that the death penalty should be reinstated ___ Strongly disagree ___ Moderately disagree ___ Slightly disagree ___ Neutral ___ Slightly agree ___ Moderately agree ___ Strongly agree
No Contact / No Response • Unobtrusive measures • Physical trace measures • Erosion and accretion • Selective deposit and selective survival • Archival measures • Allow for longitudinal analysis • Archival sources actually exist • Lower cost • Less influenced by reactivity • Selective deposit and survival
Direct Contact / No Response • Observational techniques • What is an observation? • “The selection, provocation, recording, and encoding of that set of behaviors and organisms in situ which is consistent with empirical aims” (Weick, 1968)
Direct Contact / No Response • Methods of observational coding • What, who, and when to observer • What – Behavior sampling • Who – Subject sampling • When – Time sampling
Direct Contact / No Response • Methods of observational coding • Behavior sampling • Composite behavior • Syndrome • Simultaneous behaviors • Subject sampling
Direct Contact / No Response • Methods of observational coding • Time sampling • Why is time important? • Behavior is expressed as a sequence of events • Observational data is gathered over time • Repeated observations are necessary • The need to be able to differentiate individual differences
Direct Contact / No Response • Time sampling techniques • Semi-continuous sampling • Partial-interval sampling • Divide observation period into equal time intervals • Score behavior interval by interval • 1 for occurrence, 0 for non-occurrence • Overestimates prevalence • Whole-interval sampling • 0 for non-occurrence over entire interval; 1 for any occurrence • Underestimates prevalence
Direct Contact / No Response • Time sampling techniques • Discrete sampling • Ad libitum sampling • Record behaviors on a non-systematic, informal basis • Susceptible to observer bias • Some behaviors attract attention • Momentary sampling • Observe behavior at specific, predetermined point in time • 1 if behavior occurred at that time, 0 if it did not occur • Unbiased estimate of prevalence
Reliability of Observational Coding • Interobserver agreement indices • Smaller / Larger Index • Divide smaller of two values by larger of two values • Observer 1: 9 vocalizations • Observer 2: 10 vocalizations • Smaller/Larger Index = 9/10, 90% • Problem: • Observer 1: 0 1 0 1 0 1 • Observer 2: 1 0 1 0 1 0 • S/L = 3/3 = 100% • No individual observation agrees
Reliability of Observational Coding • Interobserver agreement indices • Percent agreement • Number of agreements divided by the number of agreements plus the number of disagreements • Observer 1: 0 1 0 1 1 1 0 1 0 1 • Observer 2: 0 1 1 1 1 0 0 0 0 1 • # of agree = 7, disagree = 3 • % agreement = 7 / (7+3) = 70% • Problem – inflation by chance
Reliability of Observational Coding • Interobserver agreement indices • Cohen’s Kappa • Discounts expected chance agreements • (Po – Pc) / (1 – Pc) • Po = proportion agreement • Pc = proportion expected by chance
Reliability of Observational Coding Percent agreement & Cohen’s Kappa S: Success in barrier crossing F: Failure in barrier crossing R: Refusal in barrier crossing Percent agreement: # agree / (# agree + # disagree) Example: # agree: 7 (S) + 4 (F) + 3 (R) = 14 # disagree: 0( S) + 2 (F) + 3 (R) = 6 Percent agreement: 14 / (14 + 6) = 70%
Reliability of Observational Coding Percent agreement & Cohen’s Kappa S: Success in barrier crossing F: Failure in barrier crossing R: Refusal in barrier crossing Percent agreement: 70% Cohen’s Kappa: (Po – Pc)/(1 – Pc) Po: Proportion agreement observed Pc: Proportion agreement expected by chance
Reliability of Observational Coding Percent agreement & Cohen’s Kappa Percent agreement: 70% Cohen’s Kappa: (Po – Pc)/(1 – Pc) Po: (7 + 4 + 3) / 20 = .70 Pc: ((7*7) + (6*8) + (7*5) / (20*20) = (49+4*+35)/400 = .33 Kappa: (.70 - .33) / (1 - .33) = .5522