Statistical Sampling in OIG Reviews
Jared Smith, a statistician for the Office of Audit Services, is interviewed by Lisa Wombles, a senior auditor for Audit Services.
Transcript
[Lisa Wombles] I'm Lisa Wombles, Senior Auditor in Springfield, Illinois, speaking with Jared Smith, our office of audit statistician in Washington D.C. Jared, many recent OIG audits have used statistical sampling. Can you explain why?
[Jared Smith] Sure. Statistical sampling gives OIG the ability to cover thousands or even millions of claims in a fair and objective fashion.
[Lisa Wombles] When does OIG use statistical sampling?
[Jared Smith] We apply statistical sampling to many different areas including hospitals, home health, physician services, and durable medical equipment. In general, we consider using sampling whenever it's not possible to review every claim.
[Lisa Wombles] What methods do you use in statistical sampling?
[Jared Smith] Well, each OIG review is unique; so the sampling method we use in each review varies based on different risk factors.
[Lisa Wombles] How do you tailor your sampling methodology?
[Jared Smith] We design a sampling methodology that will do four things. First and foremost, all of our sampling methodologies are designed to be statistically valid. Second, be efficient. Third, produce a sample that's representative of the larger group. And fourth, at the end of the process, we need to produce a valid estimate of any overpayment. Courts have held that the methodology need not be precise or optimal as long as it is statistically valid.
[Lisa Wombles] How do you ensure that the sampling methodology is statistically valid?
[Jared Smith] We evaluate each sample using the appropriate law or regulation. We also use the appropriate statistical formulas to calculate any estimated overpayment. We also document and keep all documents and data related to every sample so that it can be reproduced.
[Lisa Wombles] How do you ensure that your estimates are fair?
[Jared Smith] Regardless of the sample design, we use an estimation method that gives the provider the benefit of the doubt for any uncertainty in the sampling process. As a result, our overpayment estimates will almost always be lower than what we would obtain from reviewing every claim.
[Lisa Wombles] When OIG looks at a sample of claims selected from a larger group of claims, we make an estimate. But we only apply the estimate to that specific larger group of claims from which the sample was drawn, right?
[Jared Smith] Yeah, that's correct. In addition, we reduce our overpayment estimate in order to properly account for claims that are canceled, refunded to the Medicare program, or are otherwise not in error.
[Lisa Wombles] Are there any concerns with using statistical sampling?
[Jared Smith] If done properly, it's an accurate and efficient way to look at a lot of data. Numerous administrative appeal decisions and Federal court cases have concluded that statistical sampling is an appropriate way to calculate any overpayment.
[Lisa Wombles] What about providers? Do they still have a right to appeal when we use statistical sampling?
[Jared Smith] Yes - a provider still has the right to appeal the individual determination of an overpayment through the normal Medicare appeals process. Again, the courts have upheld this.
[Lisa Wombles] What is the alternative to statistical sampling?
[Jared Smith] In order to assess the overall overpayment at a healthcare provider, we would have to review a majority of their claims. Due to the large number of records involved, such reviews would make it much more difficult for providers to gather the necessary supporting documentation and appeal any contested claim. This type of review would not be efficient.
[Lisa Wombles] Thank you, Jared, for sharing such important information about the use of statistical sampling in OIG reviews.
[Jared Smith] Thank you, Lisa.