1 / 30

Measurement of Social Capital: Recall Errors and Bias Estimations

Measurement of Social Capital: Recall Errors and Bias Estimations. Kuo-hsien Su, National Taiwan University Nan Lin, Academia Sinica and Duke University. Change in number of positions accessed from wave I to wave II (N=2,707 respondents). No change : 12%. Decrease : 52.9%. Increase : 35%.

Download Presentation

Measurement of Social Capital: Recall Errors and Bias Estimations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Measurement of Social Capital: Recall Errors and Bias Estimations Kuo-hsien Su, National Taiwan University Nan Lin, Academia Sinica and Duke University

  2. Change in number of positions accessed from wave I to wave II (N=2,707 respondents) No change : 12% Decrease : 52.9% Increase : 35%

  3. Differences between the sets of accessed positions during two interviews may reflect…

  4. Motivations • Measurement instability poses a serious challenge to the study of network changes. • Need a clear measurement or better understanding of the possible sources of error. • The two periods panel survey provided an opportunity (1) to model factors associated with changes in accessed position (2) to detect whether the respondent forgot a subsequently/previously named contact .

  5. Prior research • Forgetting is a pervasive phenomenon in the elicitation of network contacts. • Research on forgetfulness has been disproportionately based on name generator instrument. • Little research on the reliability of position generator.

  6. Tasks

  7. Data • Social Capital Project: the Taiwan Survey, conducted in late 2004 and 2006 • Consists of 1,695 men and 1,585 women aged 20-65.

  8. Problem of Non-response Wave I 2004 N = 3,280 Wave II 2006 N = 2,710 Re-interview = 82.6% Non-response = 17.4%

  9. Table 1. Characteristics of the follow-up and non-response sub-sample

  10. Three types of research designs (Brewer, 2000).

  11. Limitations of our data • Our survey was not designed to examine forgetting specifically. • No recognition data or objective records to compare with. • Two years interval is too long: Test-retest design is usually within a very short time interval.

  12. Revised method C: Comparison of accessed positions elicited in two separate interviews How many years have you known this person? 2004 2005 2006 Wave I Wave II Whether the respondent forgot a subsequently named contact? Forgetting = (Contact mentioned in wave II but not mentioned in wave I) AND (duration >= 3 years) Assumption: durations reported in wave II are more or less accurate.

  13. Coding scheme for tie changes

  14. The distribution of length of relationship of forgotten ties (N=4,332 dyads, 7.3%) The average duration of ties forgotten is 13 years

  15. How much does the respondent forget? Unique = 51.1% approximately 15% of forgetting

  16. Distribution of respondents by number of ties forgotten (N=2707 respondents) 35.6% of the respondents did not forget any ties 64.4 %of the respondents failed to mention at least one contact, with an average of 1.6 forgotten ties per respondent. These numbers suggest that forgetting a contact was not a rare occurrence.

  17. Analytical Strategies Model predicting “forgetting” Analysis for the effect of forgetting on estimates of accessibility • What factors are associated with forgetting? • Unit of analysis: person-contacts dyads • Model : Multilevel logit • Whether “forgetting” affects estimates of network resources ? • Unit of analysis: person

  18. Sample • A multi-level logit approach • The models estimate the odds of “forgetting” versus “not forgetting”; the reference population consisted of all contacts mentioned in the first interview (2004).

  19. LEVEL 2 LEVEL 1 Data structure The multi-level approach requires us to transform the individual-based data to person-contacts observations. Positions nested within individuals The final sample consists of 2,682 respondents and 28,343 person-contact dyads.

  20. Variables • Level 2 (respondent level): • Age • Years of schooling • Marital status (married) • Employment status (employee) • Occupational prestige score • Size of daily contact

  21. Variables • Level 1 (ties level): • Type of relationships • Group into six categories: kin, neighbor, school tie, work-related ties, friends, indirect tie • Length of relations (in years) • Closeness • Gender homophily • Status difference • Status distance = absolute difference between respondent’s prestige score and contact’s prestige scores • Status disparity = respondent’s prestige score – contact’s prestige score

  22. Descriptive statistics (individual level)

  23. Descriptive statistics (dyad level)

  24. Multi-level model predicting “forgetting”(level-2 model)

  25. Multi-level model predicting “forgetting” (Level-1 model)

  26. Multi-level model predicting “forgetting” (Level-1 model)

  27. Findings • Recall error may not be random. • Forgetting is more likely among weak ties. • How does recall error affect the estimation of network-driven indices?

  28. Table 4. Discrepancy between “true” (corrected) and “observed” (raw) network resources indices Because forgetting is more likely among weak ties, position-generator underestimate embedded network resources.

  29. Table 5. Correlations between “true” (corrected) and observed (raw) network resources indices at wave I (N=3,272)

  30. Conclusions • Forgetting a contact was not a rare occurrence; • Recall error is largely nonrandom. • Status difference appears to govern the recall process. • Position generator systematically underestimates network-driven resource indices.

More Related