Defending your “Arbitrary and Capricious” Samples Bradley M. Casselman , M.S. Senior Statistician

1. Defending your“Arbitrary and Capricious”Samples Bradley M. Casselman, M.S.Senior Statistician

2. The ALJ, Court and other fun waysto spend an afternoon • As we become more and more adept at identifying Fraud, Waste, and Abuse, we are seeing estimated overpayments from statistical sampling rise • Large $$ overpayments are leading to more and more aggressive defense from appellants • ALJ hearings and court appearances are becoming more and more common…

3. The Appeals Process • Providers have the right to appeal decisions made by either the MAC (AC) or the PSC to a first level review, then on to fair hearings, and if necessary, on to an Administrative Law (ALJ) hearing. • PSC’s usually become involved (if notified) at the ALJ level.

4. …arbitrary and capricious… • This has become the “catchphrase” of recent challenges to Statistical Sampling for Overpayment Estimation (SSOE) • Appellants have added due process challenges in addition to standard statistical challenges

5. …arbitrary and capricious… • 1: depending on individual discretion (as of a judge) and not fixed by law <the manner of punishment is arbitrary> • 2 a: not restrained or limited in the exercise of power : ruling by absolute authority <an arbitrary government> b: marked by or resulting from the unrestrained and often tyrannical exercise of power <protection from arbitrary arrest and detention> • 3 a: based on or determined by individual preference or convenience rather than by necessity or the intrinsic nature of something <an arbitrary standard><take any arbitrary positive number><arbitrary division of historical studies into watertight compartments — A. J. Toynbee> b: existing or coming about seemingly at random or by chance or as a capricious and unreasonable act of will <when a task is not seen in a meaningful context it is experienced as being arbitrary

6. …arbitrary and capricious… • governed or characterized by caprice • 1 a: a sudden, impulsive, and seemingly unmotivated notion or action b: a sudden usually unpredictable condition, change, or series of changes <the caprices of the weather> • 2: a disposition to do things impulsively

7. The Guidelines • The CMS Program Integrity Manual (PIM), Chapter 3, Section 10 outlines use of Statistical Sampling for Overpayment Estimation for MAC’s (AC’s) and PSC’s • Additionally, there are criteria under HCFA Ruling 86-1 that apply to conducting an SSOE

8. Lines of Defense • There are a few things that we can do ascontractors to provide the best chance of prevailing at ALJ hearings: • Fully document your process • Conduct a full analysis of appellant statistical expertise • Retain a Ph.D., even if you have to subcontract for one • Make sure lines of communication are open with the ALJ office, the MAC/AC, and the QIC

9. Documentation • Documentation is critical for a successful defense of SSOEs • A list of necessary items can be found in the PIM, Chapter 3.10.4.4 • This is an area of debate, however…

10. Documentation • PIM 3.10.4.4.1 states: • A record shall be kept of the random numbers actually used in the sample and how they were selected. Sufficient documentation shall be kept so that the sampling frame can be re-created, should the methodology be challenged. • Note that the PIM only states that documentation shall be kept so that the sampling frame can be re-created. It does not state that a contractor must “teach” the appellant the computer programming language so that they can re-create the sampling frame

11. Documentation • Your documentation should include: • A concise definition of how the universe of claims was selected • A clear definition of the sampling unit • The random number seed • A listing of the sampling frame with the associated random numbers • Listing of software/programs used to accomplish the random sample • A summary of Universe and Sample statistics • Your credentials and contact information • Date that the sample was selected

12. Documentation • Additional areas of documentation that should be kept, include, but are not limited to: • Documentation from the Investigations unit as to why the SSOE is being conducted in the first place • Documentation from Medical Review if there are workload considerations that may limit the amount of claims reviewed • A worksheet of review decisions from Medical Review along with the associated overpayment amount per sampling unit

13. Analyze the Appellant’s Documentation • If the appellant challenges a SSOE, then there is usually an accompanying document explaining why you are wrong and they are right… • This document should be analyzed thoroughly for any errors, stretched truths, and irrelevancies.

14. Analyze the Appellant’s Documentation • Also, use prior case law to counteract arguments made by the appellant’s expert • Some especially useful cases to look at are: • Pruchniewski v. Leavitt • Ratanasen v. State of California Department of Health Services • Department of Education v. United States Department of Education

15. Analyze the Appellant’s Documentation • Pruchniewski v. Leavitt • This case is intriguing on a few issues • The ALJ, then later the U.S. District Court in FL upheld a sample of size 30 • The retained expert for the appellant, Dr. Michael Intrilligator, is rebuffed in many of his standard arguments, such as: • The fact that there is no such thing as “Generally Accepted Statistical Standards” and this is a term invented by Dr. Intrilligator • The prior sampling appendix numbers are non-binding • OAS and OIG sampling guidelines are not binding to CMS • The fact that the use of the lower limit of a 90% one-sided CI is reasonable and is a valid remedy for larger COV’s • A sample of size 30 was deemed statistically acceptable

16. Analyze the Appellant’s Documentation • Ratanasen v. State of California Department of Health Services • The United States Court of Appeals, Ninth Circuit found that • “the cases addressing sampling cited above make no mention of a statistical ‘floor’ which auditors must exceed in order to guarantee providers due process” when addressing a challenge that a sample was too small.

17. Analyze the Appellant’s Documentation • Michigan Department of Education v. United States Department of Education • The United States Court of Appeals, Sixth Circuit found that: • “There is no case law that states how large a percentage of an entire universe must be sampled.”

18. Retaining a Ph.D. • One useful tool in SSOE defense is retaining a Ph.D. Statistician • Regardless of whether your argument is correct, some will judge based on a M.S. argument vs. a Ph.D. • It is therefore very helpful to have a Ph.D. on your side at a hearing

19. Lines of Communication • This may be the most important aspect of your SSOE defense • ALJs and QICs often communicate results to the AC/MAC if a PSC is involved. • It is imperative to maintain open lines of communication to ensure that you are notified of any hearings. Your chance of success is much greater when you are allowed to present your side!

20. Forum At this time, the remainder of the session will be an open forum to discuss your experiences with the process and any additional tips that may be useful Thanks!