Alternative Sampling Frame: Combining Directory Listings with Cell Phone Exchanges

1. Who Needs RDD? Combining Directory Listings with Cell Phone Exchanges for an Alternative Sampling Frame Presented at AAPOR 2008 New Orleans, LA May 16, 2008

2. Thomas M. Guterbock TomG@virginia.edu James M. Ellis jme2ce@virginia.edu Abdoulaye Diop adiop@virginia.edu Kien Le tkl7b@virginia.edu John Lee Holmes jlh2r@virginia.edu CSR—University of Virginia www.virginia.edu/surveys . . . A unit of the Weldon Cooper Center for Public Service

3. The Research Problem:RDD under threat Are there good alternatives?

4. RDD under threat Random Digit Dialing involves a certain degree of inefficiency Costs of this extra effort justified by completeness of coverage (at least until recently) Recent trends raising these costs: decreasing density of working numbers, increasing rates of non-contact, and rising rates of refusal Advent of cellular phone only households diminishes completeness of its coverage 4

5. A “New Norm?” • Dual-frame “RDD+Cell” has arisen in response to these challenges • traditional list-assisted RDD sample with RDD of working cellphone exchanges. • To screen or not to screen? • And, if not, how to weight? • But other dual frames may also be worth exploring . . .

6. Proposed Alternative: EWP+Cell • EWP+Cell = • “Electronic White Pages” + Cell Phone RDD • Promises considerably greater efficiency and cost savings over RDD+Cell • especially for specific, small geographic regions • or areas not co-extensive with any set of telephone Area Codes. • EWP+Cell fails to cover: unlisted landline households that have no cell phone • We will examine: How big a problem is that?

7. Data Source:2006 National Health Interview Survey permits estimations of the size of . . . • the non-covered segment • demographic characteristics • health characteristics • degree of coverage bias

8. What did we find?A Preview • Surprisingly little coverage bias to be expected from EWP+Cell • Potential cost savings from EWP+Cell compared to RDD+Cell

9. A brief review of the research Not much literature or research compares directory-listed samples with list-assisted, landline RDD samples Consequently, the degree and nature of the differences between listed and unlisted households is not established. 9

10. Older studies (before 2002) Most found only slight differences in substantive results between EWP and RDD frame samples Some efficiency gains in smaller geographic areas Some differences in demographics noticed And in 2007, Zogby announced plans to rely on EWP over RDD phone samples, citing lack of substantive differences in results. 10

11. Recent studies on EWP vs RDD: Substantial differences shown • Unlisted rates are higher for: • Blacks, Hispanics • Lower income • Renters • Single people • See: Guterbock, Diop • and Holian (2007)

12. From 3 segments to 5

13. The universe of U.S. telephone households, 2006

14. RDD samples cover all landline households, listed or not RDD Cell-phone- only households are excluded

15. Cell phone samples include some that are also in the RDD frame Landline- only households are excluded Cell phones

16. RDD and Cell samples overlap,yield complete coverage RDD CELL + LANDLINE 52.0% CELL ONLY 16.6% LANDLINE ONLY 31.4% Cell phones All percentages are from 2006 NHIS data.

17. We need also to consider: listedness • Some landlines are listed in the residential directory or Electronic White Pages [EWP] • LLL = Listed Landline • Some landline households are unlisted • ULL = unlisted landline • LLL and ULL may or may not also have a cell phone in the household • Cell phones are unlisted by definition • Result: five segments of the telephone universe . . .

18. Five telephone segments 2 CELL + ULL 17.7% 3 ULL ONLY 14.2% 1 CELL ONLY 16.6% 4 CELL + LLL 34.3% 5 LLL ONLY 17.2% See table I All percentages are from 2006 NHIS data.

19. Five segments differ, sometimes sharply

20. Segments differ on key demographics See table II All percentages are from 2006 NHIS data.

21. Segments differ on key health questions See table II All percentages are from 2006 NHIS data.

22. Our analysis deals withthree sampling frames: • EWP • List-assisted (landline) RDD • Cell phone RDD

23. We examine 4 sampling designs: • 2 Single frame designs: • EWP only • Landline RDD

24. We examine 4 sampling designs • 2 Dual Frame designs: • EWP+Cell • RDD+Cell

25. Three design contrasts RDD+Cell is the base for all comparisons • It includes the full universe of phone HH We will compute coverage bias for each contrast: • EWP vs. RDD+Cell • RDD vs. RDD+Cell • EWP+Cell vs. RDD+Cell

26. Formula for coverage bias Ῡ= mean for full population ῩC = mean for covered cases ῩU = mean for cases not covered U = cases not covered N = all cases

27. Contrast I: EWP vs. RDD+Cell telephone samples

28. The universe of U.S. telephone households Ῡ

29. EWP sample excludes unlisted landline and cell-only 2 EXCLUDED CELL + ULL 17.7% ῩU 3 EXCLUDED ULL ONLY 14.2% 1 EXCLUDED CELL ONLY 16.6% 4 CELL + LLL 34.3% 5 LLL ONLY 17.2% U/N = .485 EWP All listed landline phones ῩC

30. Coverage bias table:EWP vs. RDD+Cell See table IV All percentages are from 2006 NHIS data.

31. Coverage bias table:EWP vs. RDD+Cell See table IV All percentages are from 2006 NHIS data.

32. Contrast II: RDD vs. RDD+Cell only telephone samples

33. RDD samples cover all landline households, listed or not RDD ῩC ῩU

34. RDD fails to cover 16.6% RDD CELL + LANDLINE 52.0% CELL ONLY 16.6% LANDLINE ONLY 31.4% U/N = .166 Cell phones All percentages are from 2006 NHIS data.

35. Coverage bias table:RDD vs. RDD+Cell See table V All percentages are from 2006 NHIS data.

36. Coverage bias table:RDD vs. RDD+Cell See table V All percentages are from 2006 NHIS data.

37. Contrast III: EWP+Cellvs. RDD+Cell telephone samples

38. RDD+Cell covers all phone households RDD CELL + LANDLINE 52.0% CELL ONLY 16.6% LANDLINE ONLY 31.4% Cell phones

39. EWP + Cell Sample Design EXCLUDES ULL- ONLY Cell EWP All listed landline phones

40. EWP + Cell excludes ULL-only households EXCLUDED: 3 ULL ONLY 14.2% 2 CELL + ULL 17.7% U/N = .142 1 CELL ONLY 16.6% ῩU 4 CELL + LLL 34.3% 5 LLL ONLY 17.2% ῩC

41. Coverage bias table: EWP+Cell vs. RDD+Cell See table VI All percentages are from 2006 NHIS data.

42. Coverage bias table: EWP+Cell vs. RDD+Cell See table VI All percentages are from 2006 NHIS data.

43. Summary of 3 contrasts

44. 3 contrasts: 2006 estimates (ῩC ) See table VII

45. 3 contrasts: 2006 raw bias (ῩC -Ῡ) See table VIII

46. 3 contrasts: 2006 percent bias See table IX

47. Changes in CoverageNHIS 2003 - 2006

48. unlisted landline only cell phone only Changes in telephone status over time

49. Changes in percent bias over time: RDD vs. RDD+Cell (NHIS data)

50. Changes in percent bias over time: EWP+Cell vs. RDD+Cell (NHIS data)