Dueling Estimates: Closing Thoughts

1. Dueling Estimates:Closing Thoughts Jennifer H. Madans National Center for Health Statistics

2. Assumptions • Conflicting estimates are a problem for the Federal Statistical System • They cause confusion among policy makers, the press and the public • They reduce confidence in the data produced by the system and each agency • The producers may understand and accept the causes but the users do not

3. Assumptions • Producers and users would prefer if estimates of what appear to be the same phenomena were similar if not the same • Much attention has been directed toward the problem but results have been unsatisfying • We are still talking about it at this seminar

4. The Challenge • Multiple estimates is a natural result of a decentralized federal statistical system • Focus on maximizing the utility of information for Departmental policy needs • Focus on maximizing quality of each data collection activity • Internal needs trump cross system needs

5. Session Objective • Revisit possible solutions from a data collection life cycle point of view • Acknowledge the benefits and costs of different approaches • Start a discussion about what would be needed to have a more structured, proactive approach to reduce the impact of competing estimates and the associated costs

6. Current Situation • Previous speakers outlined several different approaches • Front end – joint planning and testing with coordinated implementation • Back end -- Explanation of what is being measured; potential source of difference; joint analysis; joint publication

7. Front End Solutions • Costly but hold promise • Increased cost of data collection • Reduced flexibility in collection cycle • Potential sacrifice of survey specific aims to achieve the system’s need for comparability

8. Back End Solutions • Solutions of lowest data collection and staff cost but are probably least satisfying • Will always be needed even if other approaches are used

9. System Solution • Statistical agencies produce multiple estimates of the same thing that generate little attention or confusion • Only a limited number are dueling • For these estimates, agree to designate which are ‘official’ • Official estimates can be a set of related estimates each with specific uses

10. System Solution • Requires more of a change in how agencies relate to each • Limits statistical agency ability to communicate with users, the press and the public • Could have unintended impact of budgets

11. Which Way Forward • System is too complex to embrace one solution • How do we determine what solution works best for different types of estimates? • Not all ‘dueling estimates’ are equally problematic • How do we identify the ones that need most attention?

12. Which Way Forward • When are the costs of minimizing the existence of dueling estimates too high? • Are there other benefits of strengthening ‘System’ objectives in areas of data collection and dissemination of findings?