Foresight, Insight. Hindsight US Statistical Observations

1. Foresight, Insight. HindsightUS Statistical Observations Fritz Scheuren NORC University of Chicago

2. Reminder on Definitions • Hindsight reflecting on the past –Personally/Collectively • Insight, where it all comes together, like this Conference • Foresight future seeing or shaping, also familiar but bears discussion

3. A Small Statistical Corner • US Official Statistics • Censuses and Surveys • Administrative Records • Focus on lived experiences, ala Deming

4. Times They are a Changing • Relevance of our Discipline? • Responsiveness of Statistics? • Information Age? • Misinformation Age? • Service Partnerships?

5. Children’s Teaching Game? • High? • Low? • You’re Too Slow! • How Avoid Being Too Slow?

6. But Change is Accelerating! • Is our Discipline Keeping Up? • Certainly Computationally! • Tool/Theory Building too! • Practice Slower? • Organizational Issues?

7. How To Keep Up/Catch Up? • Google (of course) • Metadata Revolution -- Still more Promise than Practice • Meta-Analytic Reuse -- Still Often Too Hard

8. High-Clockspeed Trend(use of cell phones, portable devices)

9. Responsiveness to Trends • How Well Do We Play? • High? • Low? • You’re Too Slow!

10. Response Times to Trends(organizational clockspeed = rate at which an organization introduces new products, adopts new production processes, or reorganizes itself; Sources: Charles H. Fine, 1998; David W. Rejeski, 2003)

11. Technological Mega-Trends • Faster and Faster Computing (Slower for Official Statistics) • Descriptive to Analytic • Randomization-based to Model-based • Producer Dominated to Customer Shared

12. Typical Grief Response to Change Still Often True • Denial • Anger • Bargaining • Depression • Acceptance

13. Examples of Hindsight, Insight, Foresight • Nonresponse Circa 1980 • US Census Taking Circa 1990 • Paradata Circa 2000 • Visualization Circa 2010 • Next Steps Together?

14. Nonresponse Hindsight Example

15. 40 Year CPS Income Trend • Insignificant Missingness in 1962 • Now Nearly Half of Interviews have Some Missingness • About a Third of the Amount is Imputed • But Still using the Same Basic “Hot Deck” Methods Today

16. Greater Bias Concerns • Possibility of Greater Nonresponse Bias • Potentially More Income Understatement • Also Characteristics of Poor Blurred

17. Variance More Understated • Growing Variance Not Directly Reduced • Rubin’s Multiple Imputation Solution Still Not Used in CPS • Remains Descriptive Rather than Analytic

18. Paradata Modeling Insight Example

19. Meta-Data Revolution • Applying Computing to Documentation and Training • Including Measurement Process or Paradata • Achieving Full Systems Thinking

20. Unify Meta/Paradata • Bringing All Survey Aspects together electronically • Sharing with All Stakeholders • Breaking Down Barriers between Departments

21. Unify Meta/Paradata • Bringing All Survey Meta-Data together electronically • Sharing with All Stakeholders • Breaking Down All Barriers Between Departments

22. Manage System as a Whole • Not Just Conformance to Requirements Quality • But Total Fitness for Use Quality • From Sampling/Nonsampling to Total Survey Inference • Record linkage Example

23. Total Systems Thinking • Turning Sample “Models” • Into Full Survey “Models” • Using Paradata and Experience • Politz-Simmons Example

24. Visualization Foresight Example

25. Complex Survey Graphics • Clustering and Weighting Distort • Analytically these can be “solved” Approximately • Design Effect Example

26. Restoring Visual Metaphor • Inverse Sampling Algorithm • Works for Many Designs • Satisfactory Analytically • Works Graphically too but not yet Always

27. Simulation Alternatives • Capture Essential (Sufficient?) Conditions • Simulate Graphics Analytically • Retain Real Sample? • Apply Empirical Residuals?

28. Regression Diagnostics • Design-Based and Analytic Alternatives Being Examined Now • Both Have Merits • May Imbed both in an Open-Source, like R

29. Next Steps? Further Considerations and Examples

30. Expected White Swan?

31. Unexpected Black Swan

32. Addressing Evolving Trends