Content created by Assistant Secretary for Financial Resources (ASFR)
Fiscal Year 2022
Released June, 2021
Reporting HHS Agencies: ACF | ACL | AHRQ | ASA | ASPR | CDC | CMS | FDA | HRSA | IHS | NIH | SAMHSA
Administration for Children and Families (ACF)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 3A (ACF) | Classroom Assessment Scoring System (CLASS: Pre-K) | CLASS: Pre-K is a valid and reliable tool that uses observations to rate the interactions between adults and children in the classroom. Reviewers, who have achieved the standard of reliability, assess classroom quality by rating multiple dimensions of teacher-child interaction on a seven point scale (with scores of one to two being in the low range; three to five in the mid-range; and six to seven in the high range of quality); low range is defined as any CLASS review with a domain scoring below 2.5 for purposes of this performance measure. ACF will implement ongoing training for CLASS: Pre-K reviewers to ensure their continued reliability. Periodic double-coding of reviewers is also used, which is a process of using two reviewers during observations to ensure they continue to be reliable in their scoring. |
| 4A (ACF) | The Runaway and Homeless Youth - Homeless Management Information System (RHY-HMIS) | In FY 2015, ACF entered into a Memorandum of Understanding with HUD, SAMHSA, and VA to use Homeless Management Information Systems (HMIS) as primary information technology systems to enter data on clients served by Federally-funded homeless assistance services. Since FY 2015, RHY grantees have been using local HMIS systems to upload de-identified client-level data to the RHY national data repository called RhyPoint. Following each upload, grantee data are validated by RhyPoint and a report is sent to grantees to monitor and improve data completeness and quality. The aggregate data are then cleaned and validated using a set of business rules developed by FYSB to make sure that records are accurate and relevant using a number of logic checks. |
| 7B (ACF) | National Child Abuse and Neglect Data System (NCANDS) | States report child welfare data to ACF through the NCANDS. Each state's annual NCANDS data submission undergoes an extensive validation process which may result in revisions to improve data accuracy. To speed improvement in these data, ACF funds a contractor to provide technical assistance to states to improve reporting and validate all state data related to outcome measures. The Children's Bureau, in ACF, and the NCANDS project team are working with states through national meetings, advisory groups, and state-specific technical assistance to encourage the most complete and accurate reporting of these data in all future submissions. All of these activities should continue to generate additional improvements in the data over the next few years. |
| 7D (ACF) | State Annual Reports | States are required to submit an Annual Report addressing each of the Community-Based Child Abuse Prevention (CBCAP) performance measures outlined in Title II of the Child Abuse Prevention and Treatment Act. One section of the report must "provide evaluation data on the outcomes of funded programs and activities." The 2006 CBCAP Program Instruction adds a requirement that the states must also report on the OMB performance measures reporting requirements and national outcomes for the CBCAP program. States were required to report on this efficiency measure starting in December 2006. The three percent annual increase represents an ambitious target since this is the first time that the program has required programs to target their funding towards evidence-based and evidence-informed programs, and it will take time for states to adjust their funding priorities to meet these requirements. |
| 14D (ACF) | Family Violence Prevention and Services Program Performance Progress Report Form | Grantees submit this data in an aggregated format (non-client level data). When the grantees submit their reports in the Online Data Collection System, there are automatic data validation and error checks that run before the grantees are able to submit their reports. The Family Violence Prevention and Services Act (FVPSA) Office provides a check of each grantee's data by comparing the current year's data to prior years and checking for inconsistencies or typos. The grantee is then given a short amount of time to confirm the submitted data or revise the report. In addition, performance report data are used to inform grant monitoring by state administrators and federal staff. |
| 16.1LT and 16C (ACF) | Matching Grant Progress Report forms | Data are validated with methods similar to those used with Performance Reports. Data are validated by periodic desk and on-site monitoring, in which refugee cases are randomly selected and reviewed. During on-site monitoring, outcomes reported by service providers are verified with both employers and refugees to ensure accurate reporting of job placements, wages, and retentions. All of the grantees use database systems (online or manual) for data collection and monitoring of their program service locations. |
| 22B (ACF) | National Directory of New Hires (NDNH) | Beginning with performance in FY 2001, the above employment measures – employment entry, employment retention, and median earnings gain – are based solely on earnings data obtained from the NDNH. Data are updated by states, and data validity is ensured with normal auditing functions for submitted data. Prior to use of the NDNH, states had flexibility in the data source(s) they used to obtain wage information on current and former TANF recipients under high performance bonus (HPB) specifications for performance years FY 1998 through FY 2000. ACF moved to this single source national database (NDNH) to ensure equal access to wage data and uniform application of the performance specifications. |
| 22G (ACF) | The nFORM (Information, Family Outcomes, Reporting, and Management) Data Collection and Reporting System | Healthy Marriage (HM) grantees use the nFORM system, along with web surveys for clients, to collect data at program exit on clients' attitudes toward marriage, as well as other information. To collect high-quality data, grantees receive ongoing training in data collection best practices through webinars, training videos, infographic-style "cheat sheets," in-person presentations, and bi-monthly virtual office hours. Grantees also receive individualized technical assistance through a ticketing system in nFORM, with assistance provided by email and phone calls. The use of ACASI (audio computer-assisted self-interviewing) surveys further enhances the rigor of the client survey data. Clients complete web surveys on their own, which reduces responding in socially desirable ways and eliminates variation in how interviewers might ask questions. The web surveys also include automated skip patterns so that clients only answer questions that apply to them and "soft checks" that provide an additional prompt for clients to answer questions before skipping them as a way to encourage data completeness. To help clients with low literacy levels answer the surveys on their own, the ACASI surveys include audio that reads questions and the response options to the clients. |
| 22H (ACF) | The nFORM (Information, Family Outcomes, Reporting, and Management) Data Collection and Reporting System | Healthy Marriage (HM) grantees use the nFORM system, along with web surveys for clients, to collect data at program exit on clients' attitudes toward marriage, as well as other information. To collect high-quality data, grantees receive ongoing training in data collection best practices through webinars, training videos, infographic-style "cheat sheets," in-person presentations, and bi-monthly virtual office hours. Grantees also receive individualized technical assistance through a ticketing system in nFORM, with assistance provided by email and phone calls. The use of ACASI (audio computer-assisted self-interviewing) surveys further enhances the rigor of the client survey data. Clients complete web surveys on their own, which reduces responding in socially desirable ways and eliminates variation in how interviewers might ask questions. The web surveys also include automated skip patterns so that clients only answer questions that apply to them and "soft checks" that provide an additional prompt for clients to answer questions before skipping them as a way to encourage data completeness. To help clients with low literacy levels answer the surveys on their own, the ACASI surveys include audio that reads questions and the response options to the clients. |
| 22I (ACF) | The nFORM (Information, Family Outcomes, Reporting, and Management) Data Collection and Reporting System | Healthy Marriage (HM) grantees use the nFORM system, along with web surveys for clients, to collect data at program exit on clients' attitudes toward marriage, as well as other information. To collect high-quality data, grantees receive ongoing training in data collection best practices through webinars, training videos, infographic-style "cheat sheets," in-person presentations, and bi-monthly virtual office hours. Grantees also receive individualized technical assistance through a ticketing system in nFORM, with assistance provided by email and phone calls. The use of ACASI (audio computer-assisted self-interviewing) surveys further enhances the rigor of the client survey data. Clients complete web surveys on their own, which reduces responding in socially desirable ways and eliminates variation in how interviewers might ask questions. The web surveys also include automated skip patterns so that clients only answer questions that apply to them and "soft checks" that provide an additional prompt for clients to answer questions before skipping them as a way to encourage data completeness. To help clients with low literacy levels answer the surveys on their own, the ACASI surveys include audio that reads questions and the response options to the clients. |
Administration for Community Living (ACL)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 8F (ACL) | Protection and Advocacy for Individuals with Developmental Disabilities (PADD) Annual Program Performance Report (PPR). | Outcome data for each fiscal year are reported in PPRs submitted in January of the following fiscal year. Verification and validation of data occur through ongoing review and analysis of annual reports. Data collected in the PADD PPR is validated and verified by comparing the data against parameters of that field and also compared with previous year's data. In case of any outlier data, grantees are asked to verify and/or validate and provide ACL with an explanation and/or supporting documents. |
| ALZ.3 (ACL) | ACL's Dementia Capability System Quality Assurance tool. | Each fall grantees complete the tool to assess improvements in the dementia capability of their long-term services system. Technical assistance liaisons review grantee data for completeness and accuracy. A new on-line system will facilitate grantee completion of the tool, review and analysis. |
Agency for Healthcare Research and Quality (AHRQ)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.3.8 (AHRQ) | Internal AHRQ performance management systems | Tools included in this measure will be made publicly available |
Assistant Secretary for Administration (ASA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.6 (ASA) | https://www.opm.gov/fevs/reports/governmentwide-reports/governmentwide-management-report/governmentwide-report/2018/2018-governmentwide-management-report.pdf - PDF | Office of Personnel Management offers this federal survey as a self-administered web survey of all full-time and part-time employees. OPM weights respondents' data to produce survey estimates that accurately represent the agency population..Office of Personnel Management (OPM) Employee Viewpoint Survey: https://www.fedview.opm.gov/ |
| 2.8 (ASA) | Human Resources Enterprise Processing System (HREPS) and Business Intelligence Information System (BIIS) | Review and validated by OHR Director of Analytics |
| 3.3 (ASA) | Risk Management Framework Portal (RMFP) | The HHS Office of Chief Information Director of Information Security validates these data. |
| 3.4 (ASA) | OGR Biannual FITARA Scorecard | The House Committee on Oversight and Reform, Subcommittee on Government Operations |
| 3.5 (ASA) | PhishMe Solution and PhishMe Report | The HHS Office of Chief Information Director of Information Security validates these data. |
| 3.6 (ASA) | RiskVision: Ad Hoc Reports | The HHS Office of Chief Information Director of Information Security validates these data. |
Assistant Secretary for Preparedness and Response (ASPR)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.4.13a (ASPR) | For all performance measures related to licensure, emergency use authorization, and/or commercialization of medical countermeasures are captured either through approval from appropriate regulatory agencies such as the United States Food and Drug Administration (FDA) and/or associated host country regulatory licensing board. This information is publically available and has gone through rigorous review approval for the safety, efficacy, tolerability and immunogenicity of such medical countermeasure for the advancement of pandemic preparedness and critical lifesaving interventions. During emergency times, Emergency Use Authorization's (EUA) are assigned by the FDA to move forward certain lifesaving technologies in order to meet pandemic preparedness and response timelines. All EUAs are made public on the FDA website (https://www.fda.gov/EmergencyPreparedness/Counterterrorism/MedicalCountermeasures/MCMLegalRegulatoryandPolicyFramework/ucm182568.htm#current) | All data are checked against multiple databases to ensure accuracy and validation of the numbers reported. Contracts awarded and draft requests for proposal for industry comment are negotiated and issued, respectively, in accordance with Federal Acquisition Regulations (FAR) and the HHS Acquisition Regulations (HHSAR). Interagency Agreements are developed with federal laboratories to address specific advanced research questions. Contractors and awardees are required by contract terms and conditions to report on inventions, discovery, and other advancements in the advanced development of medical countermeasures. This information is used for quality assurance and control purposes to ensure data reported is accurate. |
| 2.4.15b (ASPR) | Data sources for performance measure 2.4.15b are collected and reported from the number of executed awards made during the fiscal year as it relates to the advanced research and development of influenza vaccines and broad-spectrum therapeutics. Data sources will include www.USASpending.gov , www.fbo.gov , UFMS, and other government systems. BARDA staff are experts in analysis and report a great deal of accurate and complete data. | All data are checked against multiple databases to ensure accuracy and validation of the numbers reported. Contracts awarded and draft requests for proposal for industry comment are negotiated and issued, respectively, in accordance with Federal Acquisition Regulations (FAR) and the HHS Acquisition Regulations (HHSAR). |
Centers for Disease Control and Prevention (CDC)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 1.3.3a (CDC) | Behavioral Risk Factor Surveillance System (BRFSS) Behavioral Risk Factor Surveillance System (BRFSS), interviews conducted September-June for an influenza season (e.g., September 2011-June 2012 for the 2011-12 influenza season) and provided to ISD from NCCDPHP by August (e.g. August 2012 for the 2011-12 influenza season). Final results usually available by September (e.g. September 2012 for the 2011-12 influenza season). BRFSS is an on-going state-based monthly telephone survey which collects information on health conditions and risk behaviors from ~400,000 randomly selected persons ≥18 years among the non-institutionalized, U.S. civilian population. Numerator: BRFSS respondents were asked if they had received a 'flu' vaccine in the past 12 months, and if so, in which month and year. Persons reporting influenza vaccination from August through May (e.g., August 2011-May 2012 for the 2011-12 flu season) were considered vaccinated for the season. Persons reporting influenza vaccination in the past 12 months but with missing month or year of vaccination had month and year imputed from donor pools matched for week of interview, age group, state of residence and race/ethnicity. The cumulative proportion of persons receiving influenza vaccination coverage during August through May is estimated via Kaplan-Meier analysis in SUDAAN using monthly interview data collected September through June. Denominator: Respondents age ≥18 years responding to the BRFSS in the 50 states and the District of Columbia with interviews conducted September-June for an influenza season (e.g., September 2011-June 2012 for the 2011-12 influenza season) and provided to ISD from NCCDPHP by August (e.g. August 2012 for the 2011-12 influenza season). Persons with unknown, refused or missing status for flu vaccination in the past 12 months are excluded. |
Data validation methodology: Estimates from BRFSS are subject to the following limitations. First, influenza vaccination status is based on self or parental report, was not validated with medical records, and thus is subject to respondent recall bias. Second, BRFSS is a telephone-based survey and does not include households without telephone service (about 2 percent of U.S. households) and estimates prior to the 2011-12 influenza season did not include households with cellular telephone service only, which may affect some geographic areas and racial/ethnic groups more than others. Third, the median state CASRO BRFSS response rate was 54.4% in 2010, and nonresponse bias may remain after weighting adjustments. Fourth, the estimated number of persons vaccinated might be overestimated, as previous estimates resulted in higher numbers vaccinated than doses distributed. |
| 3.2.4b (CDC) | CDC's National Healthcare Safety Network (NHSN) and CDC's Emerging Infections Program (EIP) 's Healthcare-Associated Infections Community Interface (HAIC) activity surveillance for community-onset Clostridium difficile infections (CDI) reduction | NHSN data is validated by the Centers for Medicare & Medicaid Services (CMS) and state/local health departments. EIP data undergoes annual audits to ensure accuracy |
| 3.3.3 (CDC) | National Healthcare Safety Network (NHSN) | Extensive cross-field edit checks are used for validation and incomplete records cannot be reported. Detailed instructions for completion of report forms ensure consistency across sites. Process and quality improvements occur through email updates and annual meetings. |
| 3.5.2 (CDC) | Electronic Laboratory Reporting Repository – automated | The ELR Implementation Support and Monitoring team (collaboration between (NCEZID and OSELS) will analyze data for anomalies. |
| 4.6.2a (CDC) | US Census and Treasury; Alcohol Tobacco Tax and Trade Bureau (TTB), Monthly Statistical Reports, and the Census Bureau Annual Census Estimates | Data is pulled from public reports from US Census and Treasury, and validated through HHS and CDC calculations. |
| 4.11.10a (CDC) | National Health and Nutrition Examination Survey (NHANES), CDC, NCHS | Data are validated by NCHS |
| 4.11.10b (CDC) | National Health and Nutrition Examination Survey (NHANES). | NHANES data is validated by quality control standards. |
| 7.2.6 (CDC) | CDC/NCHS, National Vital Statistics System, Mortality | See http://www.cdc.gov/nchs/nvss/about_nvss.htm. NVSS data are provided through contracts between NCHS and vital registration systems operated in the various jurisdictions legally responsible for the registration of vital events including deaths. |
| 8.B.1.4 (CDC) | National Notifiable Disease Surveillance System (NNDSS) | Data is validated by calculations at CDC based on the format of data transmissions received by CDC. The frequency of calculation and monitoring is at least yearly. |
| 13.5.3 (CDC) | Reported as part of the Operational Readiness Review (ORR) data from 55 PHEP recipients | Quality assurance reviews with follow-up with recipients |
Centers for Medicare & Medicaid Services (CMS)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| MCR23 (CMS) | The Prescription Drug Event (PDE) data | CMS has a rigorous data quality program for ensuring the accuracy and reliability of the PDE data. The first phase in this process is on-line PDE editing. The purpose of on-line editing is to apply format rules, check for legal values, compare data in individual fields to other known information (such as beneficiary, plan, or drug characteristics) and evaluate logical consistency between multiple fields reported on the same PDE. On-line editing also enforces business order logic which ensures only one PDE is active for each prescription drug event. The second phase of our data quality program occurs after PDE data has passed all initial on-line edits and is saved in our data repository. We conduct a variety of routine and ad hoc data analysis of saved PDEs to ensure data quality and payment accuracy. |
| MCR36 (CMS) | Medicare Shared Savings Program Financial Reconciliation Reports; Master Data Management (MDM) System; Integrated Data Repository (IDR); TAP files; CCW claims data; CMS Office of the Actuary (OACT) annual Part A and B expenditure data | Numerator: Model payment actuals for CMS downside risk APMs based on model specific data, such as the number of aligned beneficiaries and annual per beneficiary spending. Denominator: The CMS Office of the Actuary (OACT) actual or estimated annual Part A and B expenditure. CMS staff and contractors provide beneficiary alignment and expenditure data to CMMI. Model teams and contractors use quality assurance measures and data cleaning, including an audit and validation process of the programs that calculate the results to ensure the reliability of the results |
| MIP1 (CMS) | The Comprehensive Error Rate Testing (CERT) Program selects a random sample of Medicare Fee-for Service (FFS) claims from a population of claims submitted for Medicare Fee For Service payment. Complex medical review is performed on the sample of Medicare FFS claims to determine if the claims were properly paid under Medicare coverage, coding, and billing rules. | The CERT program is monitored for compliance by CMS through monthly reports from the contractors. In addition, the HHS Office of the Inspector General conducts annual reviews of the CERT program and its contractors. |
| MIP5 (CMS) | The Part C Error estimate measures the extent to which diagnostic data used in payment is substantiated by medical records submitted to CMS by MAOs. The diagnostic data is used to determine risk adjusted payments made to MAOs. | Data used to determine the Part C program payment error rate is reviewed by several contractors. The Part C program payment error estimate is based on data obtained from a rigorous National Risk Adjustment Data Validation (National RADV) process in which medical records are reviewed by independent coding entities in the process of confirming that medical record documentation supports risk adjustment diagnosis data submitted by Medicare Advantage Organizations for payment. |
| MIP6 (CMS) | The payment error measurement in the Part D program is a rate that measures payment errors from errors in Prescription Drug Event (PDE) records. A PDE record represents a prescription filled by a beneficiary that was covered by the plan. | For the Part D payment error rate, the data to validate payments comes from multiple internal and external sources, including CMS' enrollment and payment files. Data are validated by several contractors. A key data source is CMS' PDE Data Validation process, which validates PDE data through contractor review of supporting documentation submitted to CMS by a national sample of Part D plans. |
| MIP9.1 (CMS) | As part of a national contracting strategy, adjudicated claims data and medical policies are gathered from the States for purposes of conducting medical and data processing reviews on a sample of the claims paid in each State. | CMS and our contractors are working with the 17 States to ensure that the Medicaid and CHIP universe data and sampled claims are complete and accurate and contain the data needed to conduct the reviews. In addition, the OIG conducts annual reviews of the PERM program and its contractors. |
| MIP9.2 (CMS) | As part of of a national contracting strategy, adjudicated claims data and medical policies are gathered from the states for purposes of conducting medical and data processing reviews on a sample of the claims paid in each state. | CMS and our contractors are working with the 17 states to ensure that the Medicaid and CHIP universe data and sampled claims are complete and accurate and contain the data needed to conduct the reviews. In addition, the OIG conducts annual reviews of the PERM program and its contractors. |
| MMB2 (CMS) | CMS Geographic Variation Database (Foundation of the Chronic Conditions Warehouse). This performance measure defines a readmission as a case of a full-benefit Medicare-Medicaid enrollee in fee-for-service who is discharged from an acute care hospital and admitted to the same or another acute care hospital within thirty days from the date of the index admission discharge. The formula is the number of readmissions per 1000 eligible beneficiaries. CMS uses a hybrid method of extracting readmissions data on Medicare-Medicaid enrollees, which incorporates elements of the Partnership for Patients readmission measure and the Medicare Hospital Readmissions Reduction Program (HRRP) measure methodologies (see MCR26 for more information). The methodology differs from MCR26 in that readmission data on all full-benefit Medicare-Medicaid enrollees in FFS is analyzed, as opposed to only those 65 years old and older, in order to capture the experience of those with disabilities under age 65 years. |
Data are validated using parallel coding, reasonableness checks on each file, version-to-version changes by variable and service types, and year-over-year comparisons. |
| MSC5 (CMS) | CMS reports the percentage of long-stay nursing home residents that received an antipsychotic medication with a quality measure (QM) derived from the Minimum Data Set (MDS). | CMS reports the percentage of long-stay nursing home residents who received an antipsychotic medication with a quality measure derived from the MDS. The MDS is the source of the data used to calculate this measure. The MDS is considered part of the medical record. The nursing home must maintain the MDS and submit it electronically to CMS for every resident of the certified part of the nursing home. For this goal, CMS reports the prevalence of antipsychotic use in the last three months of the fiscal year. The numerator consists of long stay residents receiving an antipsychotic medication on the most recent assessment. The denominator is all long-stay nursing home residents, excluding residents with schizophrenia, Tourette's syndrome, or Huntington's disease. Residents are considered to be long-stay residents if they have resided in the nursing home for 101 or more days. The baseline number reflects the prevalence of use in the last quarter of CY 2011. It was selected because it was the last quarter in the pre intervention period. |
| QIO7.3 (CMS) | Nursing Home Compare Data | Data for nursing home compare are validated as part of the process to display nursing home compare 5 star rating scores, and are comprised of Medicare claims data and MDS data. For this measure, underlying data for the 5 star rating were analyzed, and baseline and targets were set to focus improvements on current one star value nursing homes to raise the overall quality of care for nursing homes assessed, specifically one star homes. |
| QIO11 (CMS) | Medicare Patient Safety Monitoring System (MPSMS) | The Agency for Healthcare Research and Quality (AHRQ) National Scorecard data comes mostly from independent clinical chart abstractions of a statistically representative sample of United States Prospective Payment System hospitals. These charts are collected by the CMS Clinical Data Abstraction Center (CDAC) through the Medicare Patient Safety Monitoring System (MPSMS), which uses software-guided chart review of inpatient records performed by nonclinical analysts to identify 21 types of adverse events. We apply the MPSMS methodology to a multi-stage stratified random sample each year acute care hospitals eligible for Medicare's inpatient prospective payment system (IPPS). In addition, the AHRQ National Scorecard draws on two other measurement systems to capture additional adverse event types not captured by the MPSMS: select Centers for Disease Control (CDC) National Healthcare Safety Network (NHSN) and AHRQ Patient Safety Indicator (PSI) measures, which are included in order to generate a more comprehensive set of preventable patient harms. The PSI data are generated from the national Healthcare Cost and Utilization Project (HCUP) database. Over 90% of this dataset is not dependent on coding or coding practices, making it a highly reliable account of patient safety harms occurring on a national scale. AHRQ produces preliminary results using the data available in December following the performance year. Earlier data are used to fill in gaps for preliminary estimates. As all the data for a given year become available, a final number is produced. For example, preliminary results for 2015 are based on quarters 1-3 of 2015 from the MPSMS. The rate for the 2015 Q4 MPSMS data has been estimated as 93 percent of the 2015 Q1–Q3 rate, based on the mean rate for Q1–Q3 compared with Q4 rates for the MPSMS data from 2010 to 2014. Data from HCUP and NHSN included in the preliminary 2015 number are actually from 2014. More details are available at: https://www.ahrq.gov/professionals/quality-patient-safety/pfp/index.html AHRQ and CMS are in the process of updating the MPSMS data source to which new clinical measures were last added in 2005. For example, opioid-related adverse drug events are not currently monitored by MPSMS. An updated baseline is anticipated to accommodate major updates to the measurement system during the performance period. The Quality and Safety Review System (QSRS) is a software system developed for AHRQ to enable hospitals to comprehensively examine patient safety concerns and to support Centers for Medicare & Medicaid Services (CMS) Clinical Data Abstraction Center's (CDAC) chart abstraction operations. Patient safety concerns include patient safety incidents, near misses and unsafe conditions. QSRS will support the users' needs as a surveillance system to conduct retrospective medical record review of some or all of a hospital's discharges in order to identify patient safety incidents. Currently, CMS continues to work with AHRQ and their contractors to develop quality control procedures for the Quality Safety Review System (QSRS) system, especially as it relates to measuring inter-rater reliability. |
Food and Drug Administration (FDA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 291101 (FDA) | Office of Scientific Program Development (OSPD) annual evaluation reports | OSPD produces annual evaluation reports which offer a detailed summary of outcomes, including the number of applications and selections, demographics, research contributions to FDA product centers, and yearly percentage of FDA hires. OSPD creates, maintains, and verifies recruitment and graduation records. |
| 292203 (FDA) | Sentinel Coordinating Center and Active Risk Identification and Analysis (ARIA) system https://www.sentinelinitiative.org/active-risk-identification-and-analysis-aria  |
ARIA is the FDA's post-market safety surveillance system for medical products. ARIA is comprised of pre-defined, routine querying tools that allow ARIA to use customizable parameters to assess data in the Sentinel Common Data Model The Sentinel Coordinating Center will track the number of medical product analyses conducted by ARIA by using the Sentinel query tracking database. FDA updates these data on a continuous basis at the end of every calendar quarter and makes them publicly available online. |
Health Resources and Services Administration (HRSA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 4.I.C.2 (HRSA) | HRSA Bureau of Clinician Recruitment Service's Management Information Support System (BMISS) | BMISS is internally managed with support from the NIH which provides: Data Management Services, Data Requests and Dissemination, Analytics, Data Governance and Quality, Project Planning and Requirements Development, Training, and Process Improvement. |
| 16.III.A.4 (HRSA) | The RWHAP Services Report (RSR). The RSR contains client-level data and enables the Program to un-duplicate the estimated number of people who received at least one RWHAP-funded service within the reporting period. | This web-based data collection method communicates errors and warnings in the built-in validation process. To ensure data qality the Program conducts data verification for all RSR submissions. Recipients receive reports detailing items in need of correction and instructions for submiting revised data. The web system has an arrray of reports available through which the grantees and their funded providers can identify data issues that need to be resolved. In addition, the Program provides technical assistance and training during and after the submission period to address quality issues. |
| 29.IV.A.3 (HRSA) | Reported by grantees through the Program's Performance Improvement Measurement System. | Validated by project officers. |
Indian Health Service (IHS)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 81 (IHS) | IHS Integrated Data Collection System Data Mart | Monthly review of reports for completeness regarding full participation and monitoring of outliers. |
| MH-1 (IHS) | Indian Health Service Performance and Evaluation System (IHPES). | Reports generated from the IHS Performance and Evaluation System (IHPES) are reviewed and verified periodically to assure data quality control and monitor percent change outliers which may indicate error. |
National Institutes of Health (NIH)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| SRO-2.1 (NIH) | Publication, databases, administrative records and/or public documents | https://clinicaltrials.gov/ct2/show/NCT03444714 https://clinicaltrials.gov/ct2/show/NCT03953742 |
| SRO-2.9 (NIH) | Publication, databases, administrative records and/or public documents. | NIAID News Releases:
|
| SRO-2.12 (NIH) | Publications, databases, administrative records and/or public documents. | New publications and resources from awards supporting technology sharing, dissemination, and integration: FY2020 publication or resources from grants: MH111316 [PMID 31760048], NS109520 [PMIDs: 31467423, 31600508], NS109107 [PMIDs: 32989297, 32775539], NS109102 [PMIDs: 32142646,32737465,32699144, 32452384], NS109103 [PMID 31658449], NS109043 [https://open-ephys.org/ ], NS109113 [https://www.addgene.org/collections/brain-initiative/ , PMIDs: 32592656, 32675362, 32358193, 31619497]New publications from PPP awards: NS107709 [PMID 32565222], NS103442 [PMID 32413298], NS100548 [PMIDs: 32456504, 31221553], NS100549 [PMIDs: 32377637, 31730752, 32114787], NS095553 [PMIDs: 32292333, 32497676, 31471470, 31877713], NS107673 [PMIDs: 32158384, 31884187], NS103550 [PMIDs: 32699489, 31352356], NS095554 [PMID 32116233], NS103549 [PMIDs: 32292333, 32114787], NS100544 [PMID 31619492], NS103468 [PMID 31884188], NS100553 [PMIDs: 31879060, 32292333, 32299006, 32082113], NS095495 [PMIDs: 32387921, 31841958, 32103826, 32546753, 32863855, 31444029] |
| SRO-4.9 (NIH) | Grant administrative record and public press release from funded company | UG3DA047707 "Nalmefene Implant for the Long-Term Treatment of Opioid Use Disorder" https://projectreporter.nih.gov/project_info_description.cfm?aid=9778798&icde=52216417 Funding Clinical Trial NCT03810495 https://clinicaltrials.gov/ct2/show/NCT03810495 UG3DA047720 "Evaluation of Safety and Pharmacokinetics of Naltrexone Implant" https://projectreporter.nih.gov/project_info_description.cfm?aid=9778810&icde=52216423&ddparam=&ddvalue=&ddsub= &cr=1&csb=default&cs=ASC&pball1; |
| SRO-5.1 (NIH) | Publication, databases, administrative records and/or public documents | Maxwell AE, Lucas-Wright A, Santifer R, Vargas C, Gatson J, Chang LC. Promoting Cancer Screening in Partnership with Health Ministries in 9 African American Churches in South Los Angeles: An Implementation Pilot Study. Prev Chronic Disease 2019 September. PMID URL Link: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6795066/ Maxwell AE, Lucas-Wright A, Gatson J, Vargas C, Santifer RE, Chang LC, Tran K‡. Community Health Advisors assessing adherence to national cancer screening guidelines among African Americans in South Los Angeles. Preventive Medicine Reports 2020 April. PMID URL Link: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7190749/ Maxwell AE, Sundin P, Crespi CM. Disparities in cancer mortality in Los Angeles County, 1999-2013: An analysis comparing trends in under-resourced and affluent regions. Cancer Causes & Control, 2020December. PMID URL Link: https://pubmed.ncbi.nlm.nih.gov/32964365/ Leon Guerrero RT, Palafox NA, Hattori-Uchima MP, Robinett HR, Vogel CW. Addressing Cancer Health Disparities in the Pacific Peoples of Hawai'i, Guam, and the US Associated Pacific Islands Through Pacific-Focused Research Capacity Building. JCO Glob Oncol. 2020 February. PMID URL Link: https://pubmed.ncbi.nlm.nih.gov/32031449/ |
| SRO-5.3 (NIH) | Publications and grants |
2020 Publications Category 1: ADSP Banner Publications
|
Substance Abuse and Mental Health Services Administration (SAMHSA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.3.19K (SAMHSA) | This data presents national- and state-level data from the Treatment Episode Data Set (TEDS) for admissions and discharges occurring in specified time periods that summarizes demographic information and the characteristics and outcomes of treatment for alcohol and/or drug use among clients aged 12 years and older in facilities that report to individual state administrative data systems. | These facilities are surveyed annually about the nature of treatment received to develop the Treatment Episodes Data Set. Both of these activities were designed and implemented by specialized methodologists to ensure data quality. |
| 2.3.19L (SAMHSA) | National Survey on Drug Use and Health (NSDUH) | NSDUH uses audio computer-assisted self-interviewing to provide the respondent with a highly private and confidential mode for responding to questions in order to increase the level of honest reporting of illicit drug use and other sensitive behaviors. Mental Health Services is defined as having received inpatient treatment/counseling or outpatient treatment/counseling or having used prescription medication for problems with emotions, nerves, or mental health. |
| 2.3.19O (SAMHSA) | National Survey on Drug Use and Health (NSDUH | NSDUH uses audio computer-assisted self-interviewing to provide the respondent with a highly private and confidential mode for responding to questions in order to increase the level of honest reporting of illicit drug use and other sensitive behaviors. Treatment for depression is defined as seeing or talking to a health or alternative service professional or using prescription medication for depression in the past year. |