Department of Health and Human Services
DEPARTMENTAL APPEALS BOARD
Appellate Division
Arizona Health Care Cost Containment System
Docket No. A-19-29
Decision No. 2981
DECISION
The Arizona Health Care Cost Containment System (State), which administers the Medicaid program in Arizona, has appealed a June 26, 2018 determination by the Centers for Medicare & Medicaid Services (CMS) to disallow $19,923,489 in federal financial participation (FFP). Arizona's Medicaid program had received the disallowed FFP for its expenditures on school-based health services, including speech therapy, furnished to Medicaid-eligible children. The disallowance determination is based on findings of an audit conducted by the United States Department of Health and Human Services' Office of Inspector General (OIG).
In this appeal the State raises objections to the OIG's statistical sampling and estimation methods and to the OIG's finding that expenditures for certain speech therapy services were ineligible for FFP. We overrule these objections and therefore affirm the disallowance.
Legal Background
The federal Medicaid statute, sections 1901-1946 of the Social Security Act (Act),
Page 2
plan for medical assistance," which specifies the health care items and services covered by the program. Act §§ 1902(a)(10), 1905(a); 42 C.F.R. Part 435. A state Medicaid program may pay for a health care service furnished by a school to a Medicaid-eligible child if that service falls within a medical assistance category in section 1905(a) of the Act and is either specified in the State plan or covered under Medicaid's Early and Periodic Screening, Diagnostic, and Treatment benefit. CMS, Medicaid and School Health: A Technical Assistance Guide (Aug. 1997), AZ Ex. 1, at AHC 0007-09, 0011-12; Tex. Health and Human Servs. Comm., DAB No. 2187, at 2 (2008).
If applicable federal requirements are met, a state Medicaid program is entitled to FFP – that is, federal matching funds – for a percentage of the program's medical assistance expenditures, including expenditures for covered school-based health services. Act §§ 1903(a), 1905(a); Tex. Health & Human Servs. Comm., DAB No. 2187, at 3. In general, "[o]nly those expenditures for medical assistance made by a state in accordance with the state plan are eligible for FFP." Tex. Health & Human Servs. Comm., DAB No. 2176, at 3 (2008).
Case Background
Arizona's Medicaid program covers school-based health services (services rendered by school employees or contractors) under a sub-program called Direct Service Claiming (DSC). AZ Ex. 6, at AHC 0132; AZ Ex. 2, at AHC 087. DSC's coverage extends only to Medicaid-eligible students who are also eligible for school health services under Part B of the Individuals with Disabilities Education Act (IDEA). AZ Ex. 6, at AHC 0132, 0140; AZ Ex. 2, at AHC 0087. DSC-covered services may include nursing, behavioral health, and physical, occupational, and speech therapy services. AZ Ex. 6,at AHC 0131. Local Education Agencies (LEAs), a term that includes public school districts and charter schools, bill Arizona's Medicaid program on a fee-for-service basis for services they furnish under DSC. Id. at AHC 0132 ns. 1, 3. The State, in turn, claims FFP (on form CMS-64) for payments it makes to the LEAs for those services.
Between 2007 and 2009, the OIG performed an audit of the State's FFP claims for school-based services to determine whether they comported with federal and state requirements. AZ Ex. 6,at AHC 0133-34. The OIG issued a final report of the audit's findings on March 22, 2010. Id. at AHC 0122.
Page 3
The audit examined FFP claims for the period January 1, 2004 through June 30, 2006. Id. at AHC 0134. During that period the State claimed (and received) approximately $124 million in FFP for school-based health services. Id.
To determine whether that sum had been properly claimed, the OIG used statistical sampling and estimation methods. Id. at AHC 0127, 0134-35. The OIG first identified 9,542,367 paid school-based health services associated with the audited FFP claims. Id. at AHC 0134, 0144. The OIG grouped these services into 530,029 "student-months." Id. at AHC 0144. (A student-month was the statistical "sampling unit"
The OIG then obtained and reviewed medical records and other documentation in order to determine whether each of the school-based services in the sample "was allowable [that is, eligible for FFP] in accordance with Federal and State requirements." Id. at AHC 0134-0135. The OIG found that Arizona's Medicaid program had paid for unallowable school-based health services provided during 46 of the 100 sampled student-months, and that the federal government had provided $6,764 in FFP for those services.
Page 4
Based on these results, the OIG calculated a "point estimate"
To account for likely error in the point estimate – the actual value of unallowable FFP in the population was probably higher or lower than the estimate – the OIG constructed a two-sided 90 percent "confidence interval" around the point estimate.
After the OIG issued its final report, CMS agreed to overturn or modify one or more of the report's sample findings based on additional documentation furnished by the State. AZ Ex. 12, at AHC 0216. This action reduced the amount of unallowable expenditures in the sample (as determined by the auditors) from $6,764 to $6,502.55. CMS Ex. 4, at 1. Using the lower figure (rounded up to the nearest dollar), CMS recalculated the
Page 5
population point estimate of unallowable expenditures and constructed a new confidence interval whose lower bound was $19,926,220. Id. at 2.
Based on the revised lower-bound estimate, CMS formally disallowed $19,923,489 in FFP claimed by the State for school-based health services for the 30-month audit period. AZ Ex. 12. (CMS disallowed slightly less than the lower-bound estimate in order to account for a small FFP refund by the State. Id. at AHC 0216.)
The State then filed a request for reconsideration, raising objections to the OIG's sampling and estimation methods. In support of that request, the State submitted a September 2010 report prepared for the State by Milliman, an actuarial consulting firm.
CMS denied the request for reconsideration (AZ Ex. 13), and the State then filed this appeal. With its opening brief the State submitted, among other material, the Milliman Report (AZ Ex. 7), the February 12, 2018 EconLit analysis (AZ Ex. 11), and an April 29, 2019 "Supplemental Report" by EconLit (AZ Ex. 17). In response, CMS submitted a declaration by Jared B. Smith, Ph.D., the OIG's Director of Quantitative Methods (CMS Ex. 1). The State followed up with a reply brief and a June 28, 2019 "Rebuttal Report" by EconLit (AZ Ex. 20).
CMS moved to exclude the Rebuttal Report from the record. The Board denied that motion but gave CMS an opportunity to file a surreply and supporting supplemental declaration from Dr. Smith. The Board also gave the State an opportunity to file a sur-surreply "confined to addressing arguments made in CMS's surreply." See July 19, 2019 Ruling Denying Request to Exclude Appellant's Exhibit 20.
CMS filed its surreply together with a second declaration by Dr. Smith (CMS Ex. 8). The State then moved to strike the portion of the surreply addressing its objection to the OIG's finding that certain school-based speech therapy services were ineligible for FFP. See Appellant's Aug. 15, 2019 Motion to Strike. The Board denied that motion but gave the State additional time to file its sur-surreply. See Aug. 20, 2019 Ruling Denying Motion to Strike. The State then timely filed a sur-surreply and a "Second Rebuttal Report" by EconLit (proposed AZ Ex. 21).
On September 10, 2019, CMS moved to exclude EconLit's Second Rebuttal Report, asserting, in part, that the Board did not authorize the State to file any additional expert reports, and that, in any event, the content of the Second Rebuttal Report is largely repetitive and unnecessary. In response, the State argued, in part, that the Second
Page 6
Rebuttal Report is no more repetitive than Dr. Smith's second declaration and responds to arguments advanced in that declaration. The Board's August 20, 2019 ruling did not expressly state that the State may not submit any additional expert report with its sur-surreply. Although the State did not then ask the Board in advance for leave to do so, the Board has determined that the State nevertheless should have an opportunity to present to the Board additional expert opinion in support of the arguments in its sur-surreply and that our decision-making would be best served by considering the State's last expert report. We accordingly deny CMS's September 10, 2019 motion and admit EconLit's Second Rebuttal Report.
Analysis
The State contends that the disallowance should be reduced on two grounds. First, it submits that the OIG's sampling and estimation methods were "invalid" or inappropriate, resulting in a "significantly overstated disallowance." AZ Br. at 3, 21. Second, the State disputes the OIG's finding that speech therapy services provided during 12 sample student-months were ineligible for FFP. Id. at 11-20. We begin by assessing the State's objections to the OIG's sampling and estimation methods.
1. The OIG's sample-based estimate of unallowable costs is reliable evidence supporting the disallowance.
To determine the extent to which a state Medicaid agency's FFP claims include unallowable (FFP-ineligible) expenditures, CMS may rely upon statistical sampling and estimation when "individual review of the underlying records would be impractical due to volume and cost." N.Y. State Dep't of Social Servs., DAB No. 1394, at 22 (1993); see also P.R. Dep't of Health, DAB No. 2385, at 5-6 (2011) (noting that the Board and federal courts "have repeatedly upheld the use of statistical sampling in calculating disallowances of public funds"). When statistical sampling is used for that purpose, as it was in this case, CMS (as do other federal grantor agencies) typically bases the resulting disallowance determination on the estimate given by the lower bound of a two-sided 90 percent confidence interval. See, e.g., N.Y. State Dep't. of Social Servs., DAB No. 1358, at 45-46 (1992); P.R. Dep't of Health at 4, 7, 9; N.J. Dep't of Human Servs., DAB No. 2415, at 13 (2011) (noting that "[it] has long been standard practice of the [OIG] to use the lower limit of the 90% two-sided confidence interval" in auditing FFP claims). The Board has held this lower bound, if properly derived using valid methods, is "reliable evidence of the amount of unallowable costs charged to federal funds," P.R. Dep't of Health at 9, and "protect[s] [the State] with a 95% degree of confidence from having to"
Page 7
refund more than the true but unknown amount of the FFP overpayment, Ok. Dep't of Human Servs., DAB No. 1436, at 6 (1993).
Because the State challenges the validity or appropriateness of the OIG's sampling and estimation methods, CMS has the burden in this proceeding to show that those methods are "scientifically valid" and yielded "reliable evidence" of the amount of FFP improperly claimed for school-based health services.
- identifying an appropriate sampling unit – the student-month (CMS Ex. 1, ¶¶ 10, 27-28, 30);
- defining a finite target population (the sampling frame), consisting of non-overlapping sampling units (student-months) that had an equal chance of being selected (id., ¶¶ 10, 27-28);
- drawing a simple random sample from the target population using widely accepted statistical software (RAT-STATS) developed by the federal government (id., ¶¶ 7, 10);
Page 8
- using a mean-per-unit estimator or a mean difference estimator,
As noted, the mean difference estimator was calculated by dividing the total amount of unallowable FFP identified across all sampled student-months ($6,724, later reduced to $6,503) by the number of sampled student-months (100). See CMS Ex. 1, ¶ 9 (stating that the estimator was the "average error amount" in the population). a statistic calculated based on the sample findings, to derive an "unbiased""Bias" in the statistical sense means a "systematic tendency for an estimate to be too high or too low." FJC Ref. Guide at 283 (italics added). According to Dr. Smith, "an estimate based on a sample is unbiased if, on average across potential samples [drawn from the population], it will not benefit CMS or a state when compared to a review of all items." CMS Ex. 1, ¶ 9; see also EconLit Rebuttal Report, AZ Ex. 20, at 3 n.3 (stating that "[a]method of estimation is unbiased if the average value of the estimate, taken over all possible samples of a given size n, is exactly equal to the true population value" (citing Cochran, William G., Sampling Techniques (3rd ed.) at 22)). "While an unbiased point estimate exactly equals the actual repayment total [the unallowable amount] on average across potential samples, [the estimate] may be higher or lower than the actual [FFP] overpayment for any given sample" because of random error introduced by the sample selection process. CMS Ex. 1, ¶ 11; FJC Ref. Guide at 243, 246 (noting that an estimate based on a sample will differ from the true population value because of random error) and 296 (defining the term "sampling error," also known as random error). point estimate of unallowable expenditures for school-based health services in the population (id., ¶¶ 9, 10, 14); and - "account[ing] for the uncertainty of the unbiased point estimate" by calculating a two-sided 90 percent confidence interval around the unbiased point estimate (id., ¶¶ 11-14).
According to Dr. Smith, the lower bound, or lower limit, of the 90 percent confidence interval is "designed to produce an estimate [of the FFP overpayment] that is less than the actual overpayment about 95 percent of the time" and "gives [the State] the benefit of the doubt for the uncertainty in the sampling process." Id., ¶ 12; see also CMS Ex. 8, ¶ 9.
The State offers three critiques of the OIG's work. None persuades us that the OIG failed to apply accepted and appropriate methods of audit sampling and estimation, or that the disallowance rests upon on an unreliable estimate of unallowable expenditures in the relevant population.
Page 9
(1) Sample size and precision of the point estimate
The State first contends (in its opening brief) that the size of the sample – 100 student-months – was "too small to yield a result that had any acceptable precision." AZ Br. at 7 (italics and emphasis added).
"Precision" is an attribute of the point estimate. CMS Ex. 1, ¶¶ 11, 31, 43. It is the degree to which that estimate varies across potential samples; in other words, precision captures the uncertainty inherent in the point estimate. Id., ¶ 11 (noting that "[w]hile an unbiased point estimate [of FFP overpayment] exactly equals the actual repayment total on average across potential samples, [that estimate] may be higher or lower than the actual overpayment for any given sample").
In his declaration, CMS's expert, Dr. Smith, acknowledged that "[b]etter precision can be achieved by pulling larger samples" (as well as by using more complex sampling procedures or different methods of measuring the population characteristic of interest). Id., ¶ 11. Dr. Smith also acknowledged that "[p]recision and the general match between the sample and the population is an important consideration when relying on a point estimate." Id. However, Dr. Smith emphasized that precision is less salient in this case because the estimate supporting the disallowance is not the unbiased point estimate but, rather, the lower limit of the confidence interval around that estimate. Id. Dr. Smith explained that any imprecision in the point estimate arising from "sample design and choice of estimation method" is "account[ed] for" in calculating the confidence interval's
Page 10
lower bound,
In response to these statements, the State did not (in its reply or sur-surreply briefs) press its initial claim that the sample size was too small. Nor did the State dispute Dr. Smith's opinion that imprecision in the point estimate is accounted for in calculating the confidence interval's lower limit and "does not impact the confidence associated with the lower limit that CMS is using for the overpayment calculation in this case." CMS Ex. 1, ¶ 31. Indeed, Petitioner conceded that the point estimate's precision has "nothing to do with whether OIG's lower bound is correct." Sur-surreply at 5.
Page 11
limit was disallowed); Ok. Dep't of Human Servs. at 8 (stating that the lower bound of confidence interval gave the state "the benefit of any doubt raised by use of the smaller sample"); N.J. Dep't of Human Servs. at 10 (noting that the state had "not shown any prejudice to it" from the chosen sample design "given that the disallowance was not based on the point estimate"). Accordingly, we find that the size of the sample in this case did not render the population estimate supporting the disallowance invalid, and that any imprecision in the point estimate likely benefitted the State because it widened the confidence interval whose lower bound supports the disallowance.
(2) Representativeness of the sample
The next criticism leveled by the State is that the sample of 100 student-months was not "representative" of the population from which it was drawn. AZ Br. at 7, 10; Reply at 2. The sample was not representative, says the State, because the average amount of FFP paid for services performed during a sampled student-month was $322.12, while the average FFP-paid amount for student-months in the population was only $233.88. Reply at 2, 5; Sur-surreply at 2; see also AZ Ex. 7, at AHC 0158. The State provided evidence that the reason for the discrepancy was a higher-than-expected number of student-months in the sample for which the FFP-paid amounts were between $1,000 and $4,500, a range at the higher end of the frequency distribution of those amounts in the population. AZ Ex. 7, at AHC 0158-60. The State submits that it was necessary to "adjust" for that discrepancy to ensure that the disallowed amount was not "overestimated" but that CMS proffered no evidence that the OIG "considered whether [the] sample was unrepresentative, much less whether it could and should be adjusted." Sur-surreply at 2, 3.
CMS acknowledges that the "average paid amounts are higher in the sample than in the frame" but submits that the lower bound of the confidence interval calculated from the sample is a valid and reliable estimate of actual FFP overpayment amount. Response Br. at 16. In support of that position, Dr. Smith stated that:
- A simple random sample "will tend to differ" from the population and is "not expected to match the population on all dimensions";
- A "key goal of statistics" is to measure the differences between the sample and population and "account for them in a reasonable manner";
- "One well-supported approach for handling the potential differences between the sample and the population is to rely on the confidence interval rather than the point estimate" obtained from the sample; and
- "The confidence interval is designed to cover the population total even in situations where the sample does not match the population."
CMS Ex. 1, ¶¶ 37, 44-45.
Page 12
Dr. Smith further asserted that the average FFP-paid amounts identified by the State "serve as a good test case for the ability of the confidence interval to account for the differences between the sample and the population":
[The State] calls attention to the fact that the average paid amounts are higher in the sample than in the population. This argument must fail if it is possible to use the sample to reliably estimate the very quantity [FFP-paid amounts] that [the State] claims is not represented. In fact, when OIG used the same [estimation] method it used for the refund amount [the FFP overpayment] to calculate paid amounts it obtained a 90 percent confidence interval ($122,949,342 to $217,560,575) that contained the actual paid amount in the population ($123,614,883). As expected, even though the average paid amounts are higher in the sample than in the population, the confidence interval calculated from the sample still captures the correct population total.
Id., ¶ 38 (relying on the calculations in CMS Ex. 4, at 1). In short, says Dr. Smith, the sample in this case is "sufficiently representative" of the population with respect to FFP-paid amounts because the lower limit of the confidence interval calculated for that characteristic includes the total (aggregate) FFP-paid amount in the population, even though the average FFP-paid amount in the sample ($322.12) is larger than the average FFP-paid amount in the population ($233.88). Id., ¶ 44; see also CMS Ex. 8, ¶ 33 (stating that "when the paid amounts in the population are estimated using the sample values, the resulting lower limit is still less than the known paid amounts in the population").
We find Dr. Smith's analysis persuasive. As he suggests, random sampling, if executed properly with respect to a properly defined population, provides substantial assurance that the resulting sample's relevant attributes will sufficiently represent those in the population and lead to unbiased estimates. See CMS Ex. 8, ¶ 3 (stating that under accepted statistical theory, a sample is "sufficiently representative if it is pulled at random from the population to which the inference will be performed"); Colo. Dep't of Social Servs., DAB No. 1272, at 33 (1991) (stating that the "assumption underlying a validly drawn sample is that it is representative of the universe from which it is drawn").
Page 13
Of course, by chance, a random sample will almost certainly not perfectly represent the population across every dimension. The parties agree with the basic maxim that a random draw does not guarantee a sample that is a perfectly representative subset of the population. CMS Ex. 1, ¶ 37 ("Because samples are random, they will tend to differ from the populations that they were drawn from.");AZ Ex. 20, at AHC 0308 (acknowledging that "[r]andom samples cannot ensure representativeness with respect to any characteristic" and that "every sample contains sampling errors"). Consequently, an estimate of the population parameter based on sample data will differ from the value that would have been found for the parameter if the entire population had been surveyed. CMS Ex. 1, ¶ 11. But as Dr. Smith explained, random sampling enables a researcher to apply probability theory to calculate that expected difference (or error), then use that calculation to construct a confidence interval to show the range of values that can be said to include the true population value with a pre-specified level of confidence (absent any bias). CMS Ex. 1, ¶ 37, 44-45; see also FJC Ref. Guide at 240 n.83 (noting that "[r]andomness in the technical sense . . . justifies the probability calculations behind standard errors [and] confidence intervals"). Consequently, if differences between a sample and the population are due to random selection, as they appear to be in this case – the State does not allege that the OIG's methods introduced bias into the sample results – then an error in the point estimate attributable to those differences can generally be "accounted for" in the interval estimates based on the sample. See CMS Ex. 8, ¶ 33 (noting that differences between the sample and the population are generally accounted for through the use of the lower limit); FJC Ref. Guide at 241, 243, 244, 246 (explaining that a population estimate derived from a random sample will reflect some degree of error whose magnitude can be reported in terms of the "standard error" and confidence interval); id. at 296 (defining "sampling error," also called "random error," as the difference between an unbiased sample-based estimate and the true population value that results from the fact that the sample "is not a perfect microcosm of the whole").
The State asserts that the OIG "did nothing to account for the unrepresentativeness of the sample," and that this failure "produc[ed] a lower bound that is incorrect, statistically invalid, and far too high." Reply at 6. These assertions are unconvincing for at least three reasons. First, the State sidesteps Dr. Smith's main point, which is that the confidence interval's upper and lower bounds account for sample-population differences because they are calculated using the expected error, or variability, in the point estimate due to those differences. See CMS Ex. 1, ¶¶ 34, 43 (noting that the interval estimates are determined using a measurement of the point estimate's variability). The State does not cite any statistical text, theory, or formula to rebut that proposition. See Sur-surreply at 4; Reply at 2-3, 6. Nor does the State suggest that the difference between average FFP-paid amounts in the sample and population reflected mistakes by the OIG in defining the target population or selecting the sample.
Page 14
Second, the State does not contend that the disparity between average FFP-paid amounts in the sample and population is so large or unusual that the sample cannot be relied upon to produce valid interval estimates. See AZ Br. at 11 (stating that the "disallowance can be recomputed without a need for further sampling," provided that a different "estimator" is used to extrapolate the sample findings). The State's exhibits include a November 2014 report prepared by Al Kvanli, Ph.D., who stated that he had 33 years of experience "specializing in statistical issues related to government audit applications." AZ Ex. 10, at AHC 0184. Dr. Kvanli noted in this report that the average FFP-paid amount in the sample was 1.64 standard deviations above the population mean for that parameter. Id. at AHC 0187. Dr. Kvanli further stated that while the sample mean was "slightly on the high side," it was not "unusually so," and that he considered "a sample mean more than 2 standard errors from the population mean as being in the 'unusual' category." Id. at AHC 0187-88. The State did not dispute these particular observations, which appear valid on their face. See FJC Ref. Guide at 244 (noting that "[r]andom errors larger in magnitude than the standard error are commonplace," while "[r]andom errors larger in magnitude than two or three times the standard error are unusual").
Third, the State's suggestion that the sample's alleged non-representativeness actually caused the OIG's lower-bound estimate to be higher than it would otherwise have been is unfounded. According to the State, because the average FFP-paid amount for the sampled student-months was 37.7 percent higher than for student-months in the population, the OIG "overstated" the lower bound of the confidence interval for FFP-overpayment amount by the same percentage. Sur-surreply at 2, 5 (citing AZ Ex. 20, at 8 and AZ Ex. 21, at 3-4). The State provided no statistical or other mathematical analysis to back up that particular claim. According to the Milliman report, the difference in average FFP-paid amounts in the sample and population stemmed from a higher-than-expected number of student-months in the sample for which the FFP-paid amount was between $1,000 and $4,500. AZ Ex. 7, at AHC 0158-60. The State does not say which student-months in that upper range included services that the OIG identified as ineligible for FFP, specify the amount(s) of FFP paid for those services, or show that those amounts (if any) had an outsized effect in calculating the mean FFP overpayment for the sample. See CMS Ex. 1, ¶ 45 (commenting that a statistical test run by Milliman provided "no evidence of any potential impact" of the sample-population differences on the confidence interval's lower limit).
Based on the foregoing analysis, we conclude that the sample's alleged non-representativeness did not invalidate, or render unreliable, the OIG's lower-bound estimate of unallowable expenditures in the target population.
Page 15
(3) Appropriateness of the OIG's statistical "estimator"
The State's third main criticism assumes that the second is well-founded. The State contends that in light of the sample's alleged non-representativeness, the OIG should have used – and indeed was "required" to use – a "cluster sample analysis"
We pause to describe the State's ratio estimation approach, which is set out in EconLit's Supplemental and Second Rebuttal Reports (AZ Ex. 17, at AHC 0248, and AZ Ex. 21, at 2). The State calculated the ratio estimator by dividing the amount of FFP that the OIG determined had been paid in error for services performed during the sampled student-months (this amount was originally $6,764 but was reduced by CMS to $6,503 (rounded up from $6,502.55)) by total FFP payments made for those services ($32,212). The resulting quotient – the fraction of total FFP paid for services performed during the sampled student-months that was paid in error (according to the OIG) – is 0.20188. The State multiplied that fraction by total FFP paid for all student-months in the population (approximately $123.6 million) to derive a point estimate of unallowable costs ($24,953,704) and a two-sided 90 percent confidence interval whose lower bound is $11,945,011, approximately $8 million less than the disallowance amount. See AZ Ex. 17, App. B; AZ Ex. 21, at 2.
As noted, the OIG used a mean difference estimator, rather than a ratio estimator, to make its population estimates. CMS Ex. 1, ¶¶ 9-10. The mean difference estimator, which Dr. Smith described as "unbiased" (a characterization not disputed by the State), is the average FFP overpayment amount for the student-months in the sample. That estimator is derived by dividing the total amount of FFP paid in error for the sampled student-months ($6,503) by the number of student months in the sample (100). The OIG multiplied that sample mean ($65.03) by the number of student-months in the population
Page 16
(528,543) to calculate a point estimate and confidence interval with a lower bound of approximately $19.9 million. See CMS Ex. 4, at 2; AZ Ex. 21, at 2.
According to the State and its consultant, EconLit, the ratio estimator "adjusts for the fact that the sample is not representative with respect to the amount paid (because it does not matter if the sample contains an unusually high number of large payments or an unusually high number of small payments)." AZ Br. at 10. The State further asserts that, because the OIG's overall objective was to estimate the absolute "dollar amount of improperly reimbursed dollars," it was appropriate to view the chosen sample as "clusters" of "dollars paid," rather than as "a simple random sample of student-months paid," and that viewing the sample as clusters of dollars paid "dictates the use of a ratio estimator." AZ Ex. 20, at AHC 0305, 0309; see also Sur-surreply at 2 (asserting that the OIG "chose to audit clusters (student-months) of claims, rather than individual claims"). In addition, the State submits that the ratio estimator yields a population estimate that is "more precise" than the estimate derived using the mean difference estimator, and that "[p]recision matters in this case because HHS/OIG's technique increases the risk that a single estimated improper payment amount and its associated confidence limits could be much too high or much too low." AZ Ex. 20, at AHC 0311.
In response to these assertions, Dr. Smith explained that a ratio estimator allows a statistician to account for known "auxiliary" information (here, that information is the amount of FFP paid for a student-month) when estimating the value of the population characteristic of interest (FFP paid in error). CMS Ex. 1, ¶ 33. Ratio estimation, said Dr. Smith, "can improve the precision of the point estimate when the auxiliary information is related to the measure under study." Id. Dr. Smith acknowledged that such improvement is theoretically possible in this case because "total paid amounts for the student-month are likely related to the total error amounts." Id. However, Dr. Smith asserted that it is "not necessary" that the point estimate be as accurate and efficient as possible given CMS's reliance on the lower limit of the confidence interval, CMS Ex. 8, ¶ 9, and that any potential improvement in precision obtained from a ratio estimator does not affect the "reliability" of the lower-bound estimate supporting the disallowance:
[Sample] [d]esigns with worse precision tend to result in more conservative lower limits. The reason for this relationship is that the lower limit is calculated by subtracting a measure of precision from the point estimate. When the precision is worse, the amount subtracted is greater. It follows that at the time the sample was pulled, the choice to use the difference estimator was more conservative than the ratio estimator. The fact that the lower limit of the difference estimator is higher or lower than alternatives calculated for any given sample does not implicate the reliability of the limit.
Page 17
. . . Textbooks commonly refer to the precision of the point estimate; however, they do not refer to the precision of the lower limit. For example, Cochran, 1977, and Thompson, 2012, do not provide any equations for the calculation of the precision of the lower limit and do not mention the concept anywhere in the text. [The State] also has not provided any calculations comparing the precision of the two lower limits. Even if [the State] provided such calculations, it is not clear why they would be meaningful given that the lower limit is meant to provide an estimate that tends to be conservative regardless of the variability of the sample. Moreover, both the difference estimator and the ratio estimator have the same confidence level and thus are likely to be less than the actual overpayment amount the same percentage of time.
The issue with the concept of the precision of the lower limit is apparent in EconLit's statement that the 'precision matters in this case because HHS/OIG's technique increases the risk that a single estimated improper payment amount and its associated confidence limits could be much too high or much too low.' The possibility that the confidence limits are too low works in favor of [the State] because it would result in [the State] repaying a substantially smaller amount to CMS than it was actually overpaid. Moreover, the risk that the limit is too high is captured with the confidence level and is known to be small.
CMS Ex. 8, ¶¶ 13-15 (citations and paragraph numbers omitted); see also id., ¶ 25 (stating the "reliability of the lower limit of the difference estimate can be proven regardless of the properties of the ratio estimate," and thus it is "of secondary importance whether the ratio estimate is in fact unbiased or more precise than the CMS approach for the current population"); CMS Ex. 1, ¶ 31 (stating that "precision is a measure of the point estimate and does not impact the confidence associated with the lower limit that CMS is using for the overpayment calculation in this case").
Dr. Smith made several other points in response to the State's claim that the OIG used an inappropriate estimator. First, he asserted that defining the sampling unit as a student-month (rather than as a cluster of "dollars paid") and selecting a random sample of student-months were valid sampling methods and sufficed to produce an "unbiased" estimate of the relevant population parameter. CMS Ex. 1, ¶¶ 10, 25, 27-28 (explaining that the OIG's sampling unit satisfied accepted criteria for drawing a simple random sample). "[T]he fact that each student-month can be broken down to multiple services or dollar values," said Dr. Smith, "does not undermine the unbiased nature of the sample as long as the total [dollar-amount] for each student-month is finite." Id., ¶ 10; see also CMS Ex. 8, ¶ 21 (stating that defining the sampling unit to match the "quantity measured" (dollars) is "not required by the proofs underlying finite population sampling").
Page 18
Regarding the State's assertion that a ratio estimator was the only appropriate estimator for studies in which the sampling units are "clusters" of payments or of paid claims, Dr. Smith stated that "there is no single estimator that the meets the definition of a 'clustered sample calculation.'" CMS Ex. 1, ¶ 32. "When the divisible sample units [such as student-months] are selected using a simple random sample," said Dr. Smith, "the standard equations used for calculating point estimates and confidence intervals are unchanged by the fact [that] the sample units are divisible." Id., ¶ 30. "For example, the mean difference estimator (the equation used by the OIG) would remain the same whether the sample is called a clustered sample or a simple random sample." Id.
Dr. Smith further stated that the OIG does not use a ratio estimator because it can be "biased," and that "the lower limit of the ratio estimator tends to be less reliable than the lower limit of the difference estimator." Id., ¶ 22. Dr. Smith stated that the ratio estimator's possible bias "is described in Section 6.8 of Sampling Techniques (Cochran 1977)." Id.
Finally, Dr. Smith asserted that the State failed to show that the ratio estimator was actually "unbiased or more precise" than the mean difference estimator for the OIG's audit. CMS Ex. 8, ¶¶ 25-30 (stating, in paragraph 28, that "[w]hile it is possible that the ratio estimate is unbiased for the current data, the point has not been proven and certainly could not have been shown at the time that the sample was designed").
Having carefully considered Dr. Smith's declarations and the EconLit reports, we are unpersuaded that the OIG used an incorrect or inappropriate estimator to project its sample findings to the population. Key assertions by EconLit lack foundation, allowing Dr. Smith's opinions on the subject, which we find facially plausible, to stand unrebutted. For example, EconLit cited no literature, statistical theory, or evidence of accepted norms of audit sampling to support its suggestion that it was necessary for the OIG to use a ratio estimator merely because the sampling units could be viewed, or should have been originally defined, as "clusters" of "dollars paid." See AZ Ex. 20, at AHC 0305; see also AZ Ex. 11, at AHC 0199-200 (presenting the argument that the sample data should have been analyzed as a "cluster sample").
Also unfounded is the State's principal claim that a ratio estimator must be used to correct for a non-representative sample.
Page 19
provided copies of these cited pages as attachments to Dr. Smith's second declaration. CMS Ex. 8, at 15-18. As Dr. Smith observed, id., ¶ 18, none of the cited pages mentions non-representative samples or indicates that a ratio estimator must be used to address potential differences between the sample and the population. Those pages, Dr. Smith stated, merely confirm that "precision improvements [in the point estimate] are possible when there is a close relationship between the available auxiliary information [FFP paid for a student-month] and the quantity of interest [FFP paid in error]." Id., ¶ 19.
In addition, the State failed to substantiate its assertion that the ratio estimator is unbiased for the population of student-months defined by the OIG. The parties apparently agree that a ratio estimate is unbiased if two conditions (specified in Cochran, Sampling Techniques (3rd ed. 1977)) are met: (1) the "relationship between the paid amounts and error amounts is a straight line through zero"; and (2) "the variance of the paid amounts is proportional to the variance of the error amounts." CMS Ex. 8, ¶ 26; see also AZ Ex. 20, at AHC 0306 (quoting from Cochran text). EconLit asserted that "[b]oth of these conditions are true for the population in the instant matter," AZ Ex. 20, at AHC 0306, but we see nothing in the record to back up that assertion.
Page 20
that this relationship confirms that, "on average, disallowed payments are a fixed percent of claimed payments."
The State contends that "[t]he question [for the Board] should be which lower bound [estimate]" – the mean difference estimate or the ratio estimate – "bears the closest relationship to the unknown 'actual overpayment,' not whether CMS may think some other number is 'conservative' or 'acceptable' enough for its purposes." Sur-surreply at 6 (italics added). If one applies that standard – together with the principle that the "point estimate" is ordinarily the best estimate of the true-but-unknown population parameter of interest given the relevant study design (see CMS Ex. 1, ¶ 12) – then the lower-bound of the confidence interval determined by the OIG (approximately $19.9 million) is preferable to the State's lower-bound estimate ($11.9 million) because the former is closer to either party's point estimate of the "actual overpayment" – CMS's point estimate based on the adjusted mean difference estimator ($34.3 million), and the State's point estimate based on the ratio estimator ($24.9 million). AZ Ex. 21, at 2.
The State also contends that its lower-bound estimate is "more conservative" than the OIG's, Sur-surreply at 6, but the estimate cannot be considered conservative given the State's failure to verify the ratio estimator's unbiasedness. Furthermore, even assuming for the moment that the State's lower-bound estimate is more conservative than the OIG's (and the State has not shown that its estimate is more conservative), CMS was under no obligation to choose the more conservative of two conservative estimates of the actual FFP-overpayment amount given that either estimate is likely to be less than the true amount of unallowable costs in the population. See CMS Ex. 8, ¶ 16 (noting that "the lower limits of the ratio and difference estimator will vary in their exact value" upon repeated sampling but that one "can expect that both [estimates] will tend to be less than the actual overpayment amount in a majority of cases"). Given that the OIG's lower-bound estimate was derived using valid sampling and estimation techniques and is unbiased, the State cannot prevail merely by touting a lower interval estimate derived from a sample statistic (the ratio estimator) whose suitability has not been adequately
Page 21
demonstrated. Cf. N.J. Dep't of Human Services at 10 (noting that statistical results from a given "sample design" can be valid even if the design is not "the most appropriate one").
Finally, we note that despite its assertion that "precision matters" in choosing the estimator, the State failed to provide any statistical calculations or analysis to support its claim that the population point estimate generated by the ratio estimator was actually more precise than the point estimate yielded by the mean difference estimator. See AZ Ex. 20, at AHC 0311 (asserting, without supporting statistical analysis, that the ratio estimator is actually "more precise" and not just "potentially" more so). Dr. Smith stated that Cochran, Sampling Techniques (3rd ed. 1977) "defines the inequality that must hold in the sampling frame for the ratio estimate to be more precise than the difference estimator (referred to here as the 'mean per unit' estimate)." CMS Ex. 8, ¶ 29. "To argue that the ratio estimate is certainly more precise than the difference estimator," said Dr. Smith, "EconLit would have to show that the sample provides incontrovertible evidence that the [cited] inequality holds for the population, not just the sample." Id. However, EconLit's reports do not, as Dr. Smith accurately observed, "mention the inequality" described in the Cochran text, "much less demonstrate with certainty that the inequality holds for the population or could have been proven at the sample design phase." Id.
2. The State failed to carry its burden to show that certain expenditures for speech therapy were allowable.
We now consider the State's objection to the OIG's finding that certain school-based speech therapy services were ineligible for FFP.
Title 42 C.F.R. § 440.110(c) establishes conditions under which a state Medicaid program may receive FFP for "[s]ervices for individuals with speech . . . disorders."
(i) Has a certificate of clinical competence from the American Speech and Hearing Association[;]
(ii) Has completed the equivalent educational requirements and work experience necessary for the certificate[; or]
(iii) Has completed the academic program and is acquiring supervised work experience to qualify for the certificate.
Page 22
Id. § 440.110(c)(2).
The OIG found that school-based speech therapy services provided during 12 sampled student-months were ineligible for FFP because the services were not provided by, or "under the direction of," a "speech pathologist" in accordance with section 440.110(c).
There is no evidence of any such "acceptance" by CMS, legally binding or otherwise. While Arizona's State plan (as it existed during the audit period) stated that a school-based speech therapy provider had to be a state-licensed "speech-language pathologist," the plan did not state or imply that state licensure sufficed to make a therapy provider's services eligible for FFP. To the contrary, the State plan stated that a state-licensed provider's services would be "covered in accordance with the requirements in 42 CFR § 440.110." AZ Ex. 2, at AHC 0088. That statement is most reasonably read to mean that school-based speech therapy furnished by a state-licensed provider would be covered only if the provider met the federal definition of a "speech pathologist" or if the provider acted under the direction of a speech pathologist. The State does not claim that it has – or that it has ever had – a different reasonable understanding of that particular State-plan language. Furthermore, the legal significance of state licensure is an academic issue in this case because the State failed to submit proof that the therapists whose services are
Page 23
implicated by the audit findings were state-licensed "speech-language pathologists" under section 36-1940.01 of the Arizona Revised Statutes. During the audit, the State gave the OIG copies of some state-issued credentials (including "limited licenses" issued by the Arizona Department of Health Services), but it was the State's obligation to produce those materials in this proceeding if it wished to rely upon them.
There being no allegation, much less evidence, that the questioned speech therapy services were provided "by" persons meeting the federal definition of a speech pathologist, those services are eligible for FFP only if they were provided "under the direction of" a speech pathologist meeting that definition. To prove that the services met that alternative condition, the State proffered letters written by LEA officials who averred that speech therapy delivered to students during the audit period was "monitored," "coordinated," "reviewed," supported, or "supervised" by a speech pathologist. See AZ Ex. 8. All of the letters (with one exception) bear a date in September or October 2010,
The LEAs' letters are insufficient. They contain only bare uncorroborated assertions of compliance with section 440.110(c). They give no details about the questioned services (such as when and to whom they were provided) and fail to identify the speech pathologists allegedly responsible for directing those services. The letters also fail to describe the role that the speech pathologists actually played in the students' care beyond using generalities such as "coordination" or "supervision." In addition, the letters do not indicate that the attesting officials based their statements on a review of treatment files, written program policies and procedures, or other records documenting how the billed speech therapy services were planned and delivered. And the officials do not seek to explain their organizations' apparent inability to produce such records.
A grantee must ordinarily support a claim for FFP in a Medicaid service with appropriate "contemporaneous" documentation – documentation created in the normal course of business around the time that the service was provided or when payment for the service
Page 24
was made.
Even if those assertions are true, they fail to establish compliance with section 440.110(c). In Maryland Dep't of Health and Mental Hygiene, the Board held that the requirement that speech therapy be provided "under the direction of" a speech pathologist (when the speech pathologist is not the direct service provider) "is not satisfied by a showing that the speech therapist worked under the general supervision of a speech pathologist." DAB No. 2090, at 2 (2007). To demonstrate compliance with that condition, a state must proffer evidence that the speech pathologist in some way "directed . . . the provision of services to the particular student." Id. None of the LEAs' letters states that a speech pathologist in any sense "directed" – that is, gave orders or instructions about – the speech therapy provided to specific students. At most the letters establish that speech pathologists generally supervised the therapists who delivered school-based services or provided comparable degrees of oversight or "coordination."
As a last resort, the State contends that the disputed speech therapy services are not subject to section 440.110(c) because they were furnished under Medicaid's Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) benefit. AZ Br. at 18-20; Reply at 12-16. We see no merit in this contention.
The EPSDT benefit provides comprehensive health care services for individuals under age 21 who are enrolled in Medicaid. See 42 C.F.R. §§ 440.40(b), 441.50, 441.56. The Medicaid statute requires that a "State plan for medical assistance" provide or arrange for EPSDT services, as defined in section 1905(r). Act § 1902(a)(43), 1905(a)(4)(B); see also 42 C.F.R. § 441.55. Section 1905(r) defines "early and periodic screening, diagnostic, and treatment services" to include screening, vision, dental, and hearing services – plus any other health care items and services falling within section 1905(a)'s
Page 25
definition of medical assistance that are "necessary" to "correct or ameliorate" conditions discovered during screening, "whether or not such services are covered under the State plan" for EPSDT-ineligible persons.
Contrary to the State's implication, section 440.110 makes no distinction between speech therapy provided under the EPSDT benefit and speech therapy provided under other medical assistance categories. Section 440.110 is found in subpart A of 42 C.F.R. Part 440. Section 440.2(b) of subpart A states that "FFP is available in expenditures under the State plan for medical or remedial care and services as defined in" section 440.110 and other sections of that subpart. In light of that statement of general purpose, section 440.110 is properly read to govern the availability of FFP for any "expenditures under the State plan" for speech therapy. EPSDT services are, as noted, a mandatory state-plan benefit that encompasses "necessary" corrective or ameliorative speech therapy services (regardless of whether such services are available to adult (21 years or older) Medicaid-eligibles). Because EPSDT services are a mandatory state-plan benefit, expenditures for necessary speech therapy services provided under that benefit constitute "expenditures under the State plan" whose eligibility for FFP is governed by section 440.110.
Section 440.110 is also applicable here because Arizona's Medicaid plan (as it existed during the audit period) expressly required compliance with that regulation as a condition for claiming FFP for school-based speech therapy provided under the EPSDT benefit. Paragraph 4.b of Attachment 3.1-A (effective July 1, 2000) of the State plan specified the services, including outpatient speech therapy, generally available to "EPSDT recipients" (persons younger than 21 years). AZ Ex. 2, at AHC 0086-87. Paragraph 4.b further provided that, in accordance with an agreement between the State and the Arizona Department of Education, the State "will reimburse LEAs on a fee-for-service basis for a defined set of Medicaid covered services . . . provided by a qualified school-based provider to students who are Title XIX [Medicaid] eligible and eligible for school health and school-based services pursuant to the Individuals with Disabilities Education Act (IDEA), Part B." Id. at AHC 0087 (italics added). The plan then specified the "reimbursable" school-based services available to qualified students (those eligible under both Medicaid and the IDEA). Id. at AHC 0087-89. Those reimbursable services included outpatient speech therapy services "covered in accordance with the requirements in 42 C.F.R. § 440.110." Id. at AHC 0088.
Section 1903(a)(1) of the Act authorizes FFP for "amount[s] expended . . . as medical assistance under the State plan." Because school-based speech therapy provided as an EPSDT benefit during the audit period constituted medical assistance under the existing State plan, see Act § 1905(a)(4)(B) and 1905(r), and because the State plan called for school-based speech therapy services to be provided "in accordance with" section 440.110, the State's expenditures for those services were eligible for FFP only if they met that regulation's conditions. Me. Dep't of Health and Human Servs., DAB No. 2292, at 10 (2009) ("A State's expenditures are eligible for federal Medicaid reimbursement only
Page 26
if they are made in accordance with the state plan."), aff'd, Me. Dep't of Human Servs. v. U.S. Dep't of Health & Human Servs., 766 F. Supp.2d 288 (D. Me. 2011); La. Dep't of Health and Human Resources, DAB No. 979 (1988) (upholding a disallowance of FFP for transportation services that did not meet state-plan requirements).
Conclusion
For the reasons outlined above, we sustain CMS's June 26, 2018 determination to disallow $19,923,489 in FFP for Arizona's Medicaid program.
Christopher S. Randolph Board Member
Constance B. Tobias Board Member
Susan S. Yim Presiding Board Member