The Role of Over-Sampling of the Wealthy in the SCF

1. The Role of Over-Sampling of the Wealthy in the SCF Arthur B. Kennickell Federal Reserve Board Opinions are those of the presenter alone and do not necessarily reflect the views of the Federal Reserve Board or its staff.

2. Thanks! • Organizers, esp. Andrea Brandolini • FRB, SOI and NORC colleagues • Interviewers • Respondents

3. LWS Process • Create conceptual categories • Create “harmonized” (procrustean?) variables for a number of countries • Not everything is measured everywhere • Some things are measured differently

4. Apples vs. Oranges

5. Kumkwats vs. Tangerines

6. Harmonize Samples? • Samples are bedrock of a survey • Determines what a survey represents • Basis of scientific inference • For surveys with a common population definition, representational differences should be ones of efficiency • What if definition is different? • Sample vs. participants • Nonresponse probably not purely random • Compensating adjustments to weights differ

7. Fruit Baskets

8. Understand and Document • Differences may be due to structural differences in economies • Institutions • Behavior • Differences may be artifacts of different types of measurement • Questions in surveys (instruments) • Survey samples and nonresponse

9. Theorem • There is never an acceptable substitute for thinking about what you are doing. • Corollary: Don’t count on remembering the details: document, document, document.

10. Survey of Consumer Finances • Designed to measure wealth • Detail on the “Primary Economic Unit” • Summary information on other units in the household • Framed approach to measurement • Disaggregated information on assets and liabilities, with supporting detail • Strong emphasis on instrument development, training and quality control • Leave it to other exercises to evaluate the equivalizing of variables in LWS • Focus on sample

11. SCF Sample • Dual-frame design • Area-probability and List Samples • Area-probability sample • Geographically based sample of addresses • Key elements of stratification to achieve balance • Multi-stage, with clusters (tracts) as penultimate stage • Postal address sequences usually used to order the final stage • All units selected with equal probability • Robust coverage of the nation and good representation of behaviors that are broadly distributed

12. List Sample • List sample is a sample of named units (individuals/couples) used to identify households • Based on and selected from statistical records derived from tax returns by Statistics of Income (IRS) • Two-stages of selection • Accepts high-level geographic selections of AP sample • Otherwise, entirely independent • Selected using “wealth index” derived from blend of two models using income and associated data • One, a simple grossing-up of capita income • Other based on modeling of measured wealth in preceding survey • Multiple years of income data to smooth fluctuations • Over-samples wealthy units • Robust coverage of the wealthy, but not whole population

13. Role of Over-Sampling • Increases efficiency of estimates affected by upper tail of the wealth distribution • Makes possible study of relationships that would be too thinly represented in an AP sample alone • Means of detecting and correcting for nonresponse bias

14. Evaluating Over-Sampling • Compare AP sample alone with AP + list sample • Use AP weights calibrated to key population margins • Use combined sample weights with full calibrations in the separate and combined samples • LS determines the shape of the upper tail of the wealth distribution • Allows direct evaluation of the role of type of over-sampling used in the SCF

15. LS as % Combined Samples, 2004

16. Narrowly Held Assets • Of approximately 400 observations with direct holdings of bonds, only ~10% were AP cases • Of approximately 1,500 observations with direct holdings of publicly traded stocks, only ~37% were AP cases • ~25% of weighted total value attributable to AP sample

17. Shares of Total Net Worth, 2004

18. Nonresponse • Present in virtually all surveys • If distribution of a variable for nonrespondents differs from that for participants  bias • Wealth surveys cover a particularly sensitive topic: respondent resistance • Systematic components to nonresponse • E.g., NR increases with capital income and decreases with age and charitable contributions • Central area of research for SCF • LS provides means of correction

19. (AP+LS)-AP, 2004 NET WORTH $2.7M/74% $13,600/4.3% $3,500/3.9% $250/1.9%

20. Effect on Wealth Distribution • High-wealth list sample cases “displace” top of the AP wealth distribution downward • Similar pattern if add synthetic case to AP sample with weight of 1% of population and wealth at 99 percentile of combined distribution • Actual effects more subtle

21. Special Difficulties • More complex interviews • Requires more sophisticated instrument • Training more important • More difficult to contact and gain cooperation • “Gatekeepers” • Interviewers often have an incentive to avoid difficult cases • More expensive • Same problems without an oversample, but much less visible

22. Thanks