Surveying Migrant Households: A Comparison of Census-Based, Snowball, and Intercept Point Surveys

1. Surveying Migrant Households: A Comparison of Census-Based, Snowball, and Intercept Point Surveys David McKenzie World Bank (with Johan Mistiaen)

2. Motivation • Increasing research and policy attention being given to migration • But few detailed and representative surveys of households of migrants exist • Many specialized surveys of migrants are non-probability samples, making it hard to generalize from them => Paper reports on experiment in Brazil to compare three different survey methods

3. Context • World Bank survey of Brazilians of Japanese descent (Nikkei) • Approximately 1.2-1.9 million of them among Brazil’s 170 million population • Japanese law allows third-generation and below unrestricted access to labor market • Estimated 265,000 Brazilian migrants in Japan, sending approximately $US2billion in remittances annually. • Survey methods used here equally applicable to attempts to survey migrants in destination countries.

4. Surveying methods • In most countries, surveying migrants is survey of rare element • Three approaches used in practice: • Stratified sampling using population census • Chain-referral methods such as snowball sampling • Sampling at locations where immigrants tend to cluster. Time and space/intercept sampling. => Question as to how well other methods perform compared to random sample.

5. Our experiment • Compare three different methods in Brazil • Use the same questions and same survey firm • Wish to compare characteristics of households with and without migrants, estimate proportion of households receiving remittances and with migrants in Japan • Context: population predominately urban; crime a concern => Communications with community organizations, use where possible of Nikkei interviewers.

6. Method 1: Stratified random sample • Restrict to Sao Paulo and Parana provinces where 80% estimated to live (still 47 million people) • Use 2000 Brazilian Census to classify households as Nikkei or not (approximately) • Census doesn’t ask ethnicity, instead asks: race (yellow), place of birth, whether individual has lived elsewhere in last 10 years • Second complication: only 10% long form of census asks this.

7. Method 1: Stratified random sample • Steps: • Select 75 census tracts, with stratified random sampling • Carry out door-to-door listing of households in these census tracts • Return and administer questionnaire to households identified as Nikkei • Follow-up round to re-try and interview those who refused on first set of interviews, use shorter interview if refuse longer one.

9. Listing • Letters sent out to 150 Nikkei associations with bases in the areas chosen, to explain survey • Census tract averages 301 houses • 42 interviewers in Sao Paulo and 24 in Parana used for 50 and 25 tracts. • Three attempts made to interview household; proxy-reporting used if no one at home.

10. Summary of listing • Listed 22, 539 dwellings • Among these, detected 839 Nikkei households – 528 interviewed in person, 311 by proxy-reporting • Initial phase of interviewing interviewed 247 Nikkei households • Returned and interviewed another 156 (45 long interview, 111 short interview) • Found some of those identified as Nikkei by proxy were not • Estimate in the end: • Parana: 76% interviewed, 12% refuse, 10% absent • Sao Paulo: 45% interviewed, 31% refuse, 17% absent

11. Comparing households in 2 phases • Main difference is that households that required the additional effort to survey are much more likely to refuse to say how much they receive in remittances. • Look similar in many other respects to those who answered first time.

12. Snowball survey • Contacted 25 Nikkei organizations and asked each to provide 3 names as seeds. • Targeted sample of 300 households • Received 67 seed names • Two main problems: • Some of seed households refused • Many households refused to provide referrals • In the end have sample of 100 households, 60 seed households, 40 referrals.

13. Intercept survey • Consulted with local researchers, Nikkei organizations, and Sudameris officers to select broad range of locations which Nikkei community frequents • Chose 9 fixed points, and 6 events • Sports club, metro station in Liberdade neighborhood, two Feiras (Sunday marketplaces), hospital, grocery stores, language school, outside branch of bank. • Japanese film event, large cultural festivals, Japanese food festival, Japanese art exposition, Christmas concert and music festival. • Short questionnaire – 62 questions, 7 minutes.

14. Intercept survey • At each location, 2 interviewers used, one to count number passing through location, one to interview • Interviewers there for 129 total hours over 2 weeks (~8.5 hours/location). • Each person asked how often in past 2 weeks had frequented any of the other locations – used to reweight answers. • Only 19% had visited only one location – on average individuals had visited 3.18 out of 15 locations during the two week period. • Weight:

17. Fourth method? • Use phone book to select individuals with minority names? (Osili with Nigerians in Chicago) • 90% have landline phone (most of rest have cellphones) • With intermarriage, 74.8% have recognizably Japanese surname • Only 24% appear in the phone book • Only 20% in phone book with Japanese name!

18. Comparison of different methods • Hypotheses: H1: intercept and snowball households will be more closely connected to Nikkei community than random sample. H2: weighting intercept survey should bring it closer to random sample H3: snowball and intercept surveys will overestimate proportion of households with migrant experience. H4: refusal rates for questions about remittances will be higher for intercept survey, since they take place in public location.

19. Results: Connection to Japan

20. Results: Migration and Remittances

21. Remittances • Proportion refusing to report how much they receive is much higher (82%) in intercept survey than in household survey (31%). • Fear of crime likely to lead to more reticence in public places.

22. Do they give different results in regressions • Probit regressions to consider determinants of being a return migrant • Stratified and intercept survey give quite similar results • Snowball gives quite different picture – gives different answers to education and gender selectivity of migrants.

24. Comparison of costs • Total cost per household interviewed: • Stratified survey: US$212 ($2 per dwelling listed; $80 per interview) • Snowball survey: US$100 • Intercept survey: US$30 • Recall survey lengths differ: • Stratified and Snowball: 36-page questionnaire, 1 hour to complete • Intercept: 3-page questionnaire, 7 minutes to complete

25. Conclusions • Snowball and Intercept methods oversample individuals more closely connected to community • Intercept method does seem to give results closer to random sample when reweight data – suggests multiple location sampling important • Snowball, often seen as cheapest and easiest, may be just as difficult to carry out in some applications.

26. Conclusions • Choice of survey method does make large and significant differences to estimates of means of interest, and to regression analysis. • Cost considerations may suggest: • Intercept survey for exploratory analysis, or where target population attends community locations • Stratified survey for representative analysis.

27. Discussion: when are these methods applicable • Methods here easily applicable in migrant destinations (e.g. Salvadoreans in D.C.) • Are methods applicable in sending countries when group is not ethnically different? • Listing based methods clearly are • Snowball used by sociologists • Intercept – festivals, transportation hubs, money-transmitting branches, churches, social support networks, …