1 / 31

Agency Capabilities and Performance in Applying BCA

Agency Capabilities and Performance in Applying BCA. Another Modest Proposal for Reform. Richard B. Belzer Regulatory Checkbook Mt. Vernon, VA rbbelzer@post.harvard.edu. History of Federal Practice. Formalized since 1981 Agency responsibility to perform BCA

kale
Download Presentation

Agency Capabilities and Performance in Applying BCA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Agency Capabilities and Performance in Applying BCA Another Modest Proposal for Reform Richard B. Belzer Regulatory Checkbook Mt. Vernon, VA rbbelzer@post.harvard.edu

  2. History of Federal Practice • Formalized since 1981 • Agency responsibility to perform BCA • OMB responsibility to perform peer review • Controversies and conflicts are hard-wired • With Congress and among agencies • Staffing, organization, and reprogramming • Methodological issues and research priorities

  3. ‘CSI: Executive Branch’ • Bureaucratic biology • BCA: A bacterial infection • Agencies have developed antibodies • Program control over / co-option of agency economists • Methodological corruption • Benefits can be found if you look hard enough • Costs can be ignored if you are determined enough • OMB: Dissociative Identity Disorder (ICD-9-CM Dx 300.14) • Performs external, independent peer review of RIAs… • …but advance the President’s agenda… • …without being held publicly accountable for the difference where these functions conflict • Administrative Procedure Act: Placebo therapy

  4. Evidence of ICD-9-CM Dx 300.14 Caveats • Agency (not OMB) estimates • From a draft report Invalid Assumptions • All ‘major rules’ are included • Agencies adhered to OMB Circular A-4 • Estimates of costs, risks, and benefits are unbiased • Uncertainty is unimportant

  5. Agency Use of Science in BCA:April 1998 -- March 2000 • 73 rules, 36 about risk • Embedded bias • 7 (‘judgment’) • 2 (assumptions) • 6 (upper bounds) • 1 (sensitivity analysis) • Uncertainty ignored • 6 (point estimates) • 2 (ranges) • 1 (distribution) • Motive for performing BCA • Statutory reqmt: 100% • EO 12866: 14%

  6. Examples:Major Rules from 1988-1998 • ‘Hazardous Waste Corrective Action’ • C: ~$1 trillion; B: not estimated • ‘Partially-Cooked Beef Patties’ • C: < $10 million; B: not estimated • ‘Seafood HACCP’: impossible benefits • Reduced risks from seafood(‘consumers underestimate true risk’) • Increased consumer confidence in seafood safety(‘consumers overestimate true risk’) • ‘Meat & Poultry HACCP’: imaginary benefits • Assume regulation reduces risk 90%

  7. Examples:Major Rules from 1988-1998 • ‘Hazardous Waste Corrective Action’ • C: ~$1 trillion; B: not estimated; R: ~0 • ‘Partially-Cooked Beef Patties’ • C: < $10 million; B: not estimated; R: ~ 30 deaths/yr • ‘Seafood HACCP’: impossible benefits • Reduced risks from seafood(‘consumers underestimate true risk’) • Increased consumer confidence in seafood safety(‘consumers overestimate true risk’) • ‘Meat & Poultry HACCP’: imaginary benefits • Assume regulation reduces risk 90%

  8. Examples:Major Rules from 2007-08 • 5 Major Rules • PTO ‘Limits on Claims’ • PTO ‘Limits on Continuations’ • PTO ‘IDS’ • PTO ‘Appeals’ • DHS ‘No Match’ • OMB made aware of costs but deemed none of them ‘major’

  9. Examples:Major Rules from 2007-08 • 5 Major Rules • PTO ‘Limits on Claims’ • PTO ‘Limits on Continuations’ • PTO ‘IDS’ • PTO ‘Appeals’ • DHS ‘No Match’ • OMB made aware of costs but deemed none of them ‘major’ $1 billion!

  10. BCAs Can Be Validated,But They Aren’t • Often easier than ex ante analysis • Very limited federal agency interest • Limited external WTP for ‘academic’ work HACCP Implementation Period

  11. Dx 2001:Centralized Review Doesn’t Work • Limited and progressively smaller review staff • Transaction-driven, and hence reactive • Review occurs after decisions have been made • Asymmetrical rules of engagement • Limited enforcement tools • Enforcement has limited utility • Presidential initiatives are exempt • Congressional sensitivity is unbounded • Statutory/judicial deadlines trump OMB review • OMB has limited political and public support

  12. Failed Remedies Persuasion Co-option RIA Guidance Return to sender Suspension of review Rx 2001:

  13. Failed Remedies Persuasion Co-option RIA Guidance Return to sender Suspension of review Possible Ideas Regulatory budgeting ‘Prompt’ letters Publish peer reviews BCA blueprints Integrate BCA & PRA Rx 2001:First Modest Proposal

  14. Failed Remedies Persuasion Co-option RIA Guidance/Cir. A-4 Return to sender Suspension of review Possible Ideas Regulatory budgeting ‘Prompt’ letters Publish peer reviews BCA blueprints Integrate BCA & PRA Information quality Peer review policy Rx 2001:Implemented 2001-09

  15. How Have These Remedies Fared? • Guidance, ‘prompts’, and returns

  16. OMB Review:Lion or Pussycat? • Lion • Reagan I • Bush 41 • Bush 43 (2001 only) • Pussycat • Reagan II • Clinton I & II • Bush 43 (2002-07)

  17. How Have These Remedies Fared? • Guidance, ‘prompts’, and returns • Peer review policy (2005)

  18. How Have These Remedies Fared? • Guidance, ‘prompts’, and returns • Peer review policy (2005) • Covers ‘influential’ scientific, technical, & statistical information used in BCA’ • Good idea, badly implemented • Too much agency discretion • Agencies control charge and reviewers • Reviewers cannot veto • Agency BCAs reviewed by OMB are exempt!

  19. How Have These Remedies Fared? • Guidance, ‘prompts’, and returns • Peer review policy (2005) • Information quality

  20. Information QualityLaw and Guidelines • ‘Information Quality Act’ (2001)44 U.S.C. 3516 note • OMB government-wide guidance (2002) • Reduce error: Pre-dissemination review • Correct error: Administrative procedures • Agency responses • All complied except DHS • Almost all were not serious

  21. Information QualityTerms and Definitions • Utility • Integrity • Objectivity • Substance • “accurate, reliable, and unbiased” • Presentation • “accurate, clear, complete, and unbiased manner” • “within a proper context” • Procedure (transparency, reproducibility)

  22. Substance Transcription Units Excess precision Misapplication Misuse Bias (not policy-neutral) Presentation Caveats omitted Sources omitted Context omitted Data/results censored ‘Best’ models only Bias (not policy-neutral) Types of Information Quality Error Procedural • Non-transparency • Irreproducibility

  23. Types of Bias • Incidental • Not material to decision-making • Unavoidable • No model is perfect, data are limited and expensive • Purposeful • ‘2 + 2 = 5 for large values of 2’ • Best justification given: ‘avoids false negatives’ • Consequence: many false positives • Conflates analytic and decision-making functions • Lets analysts make policy decisions • Fundamentally incompatible with information quality and BCA

  24. An Example ofFreely Advertised Purposeful Bias “EPA’s policy is that risk assessments should not knowingly underestimate or grossly overestimate risks. This policy position prompts risk assessments to take a more ‘protective’ stance given the underlying uncertainty with the risk estimates generated.” U.S. Environmental Protection Agency Office of the Science Advisor. 2004. An Examination of EPA Risk Assessment Principles and Practices. Available: http://www.epa.gov/osainter/pdfs/ratf-final.pdf

  25. Why Biased Risk AssessmentIs So Toxic to BCA • Even if every other methodological problem in BCA is solved… • BCA results are invalid and highly misleading • Baseline risks are overstated • Benefits of regulation are overstated • CE-analysis yields unreliable rankings • Rank order would be correct if bias was constant • Bias is not constant • Tolerating bias incentivizes bias • Some interests are made better off if risk assessment and BCA are discredited

  26. Flexible Principles:Activists & BCA

  27. Flexible Principles:Activists & BCA

  28. Regulatory Checkbook Study of5-Year Agency Performance • Procedural compliance • Timely? • Informative? • Interpretively correct? • Forgiving of limited sophistication? • Substantive compliance • Accurate: Is response true? • Honest: Are legitimate complaints admitted? • Responsive: Is the remedy reasonable?

  29. Agency Performance on the Least Demanding Procedural Index

  30. Why IsAgency Performance So Bad? • Institutional • Externally imposed • Some errors are useful • Acknowledging error increases litigation and compromises enforcement • Legal • WH decision to issue guidance, not promulgate a rule • Limited enforcement of guidance by OMB • No enforcement of guidance by courts • No congressional oversight

  31. What Can the NewSociety for Benefit Cost Analysis Do? • Publish a journal • Some theory • Lots of applications • Publish an external, authoritative manual • OMB guidance documents (1990, 2000, 2003) need improvements that OMB staff cannot make • An external authority is better • Peer review federal agency BCAs • Use information quality levers (insist on objectivity) • Convene genuinely independent panels • M*A*S*H approach (‘meatball economics’)

More Related