|
Oregon Health Policy Commission – Quality and Transparency Workgroup
Hospital Reporting
Rationale for Recommendation of APR-DRGs
|
| Criteria for Risk Adjustment Systems for Public Hospital Reporting |
A wide variety of risk adjustment software is available, all differing with regard to requisite data elements, populations covered, the type risk measured (risk of what?), and the time period to which the assessment is applicable. Listed below are key criteria important for the risk adjustment system to be used for this public hospital reporting project.
- Optimized for use with hospital inpatient data. In many cases, risk adjustment systems incorporate inpatient data in addition to outpatient data and/or clinical information such as medical history or laboratory results. While these factors may enhance risk assessment, outpatient data are not routinely collected in Oregon, unique person identifiers are not available to link such files, and obtaining the resources necessary for abstracting clinical information is not currently feasible.
- Applicable to the General Hospitalized Population. Some risk adjustment systems are designed for specific populations such as the Medicaid, veterans, or infants in PICU’s. The selected system must be designed for use with a broad population.
- Produce risk scores at the discharge level. A few systems predict resource use for a particular patient over an entire year. However, existing quality indicators are applied at the discharge, not the patient level, and patient identifiers are not provided in the available data.
- Risk scores must be applicable to mortality, charges, and possibly length of stay measures. Systems designed to assess future resource consumption or very specific outcomes have limited value for this project. Ideally, the selected risk adjustment system will produce different scores for various outcomes.
- Methodology must be transparent. The methods and logic used by proprietary, or “black box” systems, do not meet this transparency requirement.
- Good statistical performance in empirical studies. Risk adjustment must explain a sufficient amount of variation in charges, mortality, and/or other measures.
- Validated for similar applications. In many cases, other organizations have used risk adjustment tools for hospital reporting.
| Overview of Existing Risk Adjustment Systems |
Focusing on criteria 1 through 4 above, many existing risk-adjustment systems are not applicable to hospital discharge data for public hospital reporting. *Indicates systems that are technically feasible given the available data and required product.
RISK ADJUSTMENT TOOL
|
CONSTRAINTS
|
Adjusted Clinical Groups (ACG). Projects resource consumption over time based on morbidity profile obtained from inpatient and outpatient diagnoses.
|
Requires outpatient data, not discharge specific, focus on resource consumption
|
Acute Physiology and Chronic Health (APACHE). Computes a variety of risk scores such as ICU mortality and length of stay for the ICU population based on 17 acute physiologic parameters and other clinical information.
|
Limited to adults in ICU’s, requires extensive clinical information
|
*All Patient Refined Diagnosis Related Groups (APR-DRG). Assess resource use (severity of illness) and in-hospital mortality based on computerized discharge data.
|
No technical constraints
|
*All-Payer Severity-Adjusted DRGs (APS-DRG). Provides severity of illness score (0, 1, 2) based on complications and comorbidities.
|
No technical constraints
|
Chronic Illness and Disability Payment System (CDPS). Estimates annual total expenditures for Medicaid recipients based on hospital and physician claims.
|
Requires outpatient data, not discharge specific, focus on resource consumption
|
Comprehensive Severity Index (CSI). Assess physiologic complexity for a variety of populations, based on disease-specific clinical factors.
|
Requires clinical data
|
*Delta System. Closed system that appears to be based on the APS-DRG.
|
No technical constraints; proprietary
|
Diagnostic Cost Groups/Heirarchical Condition Category (DCG/HCC)1. Projects total expenditures based on hospital or physician claims or encounter records.
|
Not discharge specific, focus on resource consumption
|
*Diagnosis Related Groups (DRGS). Groups each discharge into groups that are similar clinically and for resource intensity using computerized discharge records.
|
No technical constraints, but DRGs are imprecise groupings designed for payment and most ideally applied to the Medicare population.
|
*Disease Staging (DS). Multiple versions are available; all provide complexity/severity scores, and one provides scores specific to total charges/costs, length of stay, and in-hospital mortality. Two versions work with computerized discharge data.
|
No technical constraints;
Stage definitions are open but complex; prediction equations are proprietary
|
MedisGroups. Produces scores for a variety of outcomes including various mortality and length of stay measures using 250+ “key clinical findings”
|
Requires clinical data
|
National Surgical Quality Improvement (NSQIP). Assesses risk for 30-day post-surgery mortality and complications for veterans based on a wide range of clinical data.
|
Limited to veterans, specific to surgical procedures, requires clinical data, risk is very specific to mortality and complications
|
Pediatric Risk of Mortality Score (PRISM). Assesses risk of PICU mortality.
|
Limited to pediatric population in PICU, requires clinical data
|
*Refined Diagnosis Related Groups (RDRG). Refines DRG into 4 levels of severity based on presence of major complications and comorbidities.
|
No technical constraints, but like DRGs, are most ideally applied to the Medicare population
|
| Technically Feasible Risk Adjustment Systems |
A notable point of consensus is that “no risk adjuster is ever perfect” (Iezzoni, 2003). This fact applies to systems based on both administrative and clinical data. While clinical data can provide risk adjustment better able to distinguish among providers, the complexity and resource intensity of clinical data collection can also lead to inaccuracies and inconsistencies that increase noise and bias that counteract advantage gained from incorporating clinical data (Davies, 2001).
Statistical Performance, Validation, Transparency, and Generality. As seen in the table below, among risk adjustment systems based on administrative discharge data, no system consistently outperforms the others in statistical performance (Criteria 6). The systems diverge, however, in the degree of methodological transparency (Criteria 5), the extent to which they have been validated in comparative studies and in practice (Criteria 7), and their inclusion of all patient populations.
|
Risk Adjustment Tool
|
Explained Variance
|
Validation Studies
|
Use By Other Organizations
|
Other Issues |
|
AMI*
|
CABG*
|
Pneu*
|
Stroke*
|
LOS
|
APR-DRG
|
0.20
|
0.07
|
0.10
|
0.10
|
0.14
|
Extensive
|
HCUP
MD, UT, MI, NM
TX, NY, WI, CO (AHRQ)
|
|
APS-DRG
|
**
|
**
|
**
|
**
|
**
|
Scarce
|
HCUP
|
|
Delta System
|
**
|
**
|
**
|
**
|
**
|
Scarce
|
WI
|
Proprietary?
|
Disease Staging
|
|
|
|
|
|
|
HCUP
|
|
|
Mortality
|
0.27
|
0.07
|
0.13
|
0.11
|
|
Moderate
|
|
|
|
Stage
|
0.17
|
0.02
|
0.08
|
0.01
|
0.14
|
Moderate
|
|
|
DRG
|
**
|
**
|
**
|
**
|
**
|
-NA-
|
CMS
|
Medicare population
|
RDRG
|
0.15
|
0.08
|
0.28
|
0.07
|
0.17
|
Moderate
|
CMS
|
Medicare population
|
*In-hospital mortality. AMI (Iezzoni, 1996b); CABG (Landon, 1996); Pneu (Iezzoni, 1995); Stroke (Iezzoni, 1996a); LOS (Iezzoni, 2003)
**Comparable data not available
DRG-Based Systems. With the exception of Disease Staging, the risk adjustment tools listed here are DRG-based severity systems. DRG systems have the advantage of widespread use and comprehension among state data organizations, CMS, and hospitals, in addition to extensive development to accommodate all populations and to improve predictive power for outcomes other than resource use (Davies, 2001). They have also been used in conjunction with smoothing methods, which reduce the effects of random variation in performance measures for small providers in particular (Davies, 2001). Smoothing methods are used in AHRQ’s quality indicator programs.
A disadvantage of DRG-based systems is that major operating room procedures are included as risk-adjusters, even though the decision to perform surgery is under the medical staff’s control, and thus not an exogenous factor. Solutions to this issue are offered, so this limitation is not insurmountable (Hornbrook, 2003). Additionally, while Disease Staging does not have this limitation, its expense, stay, and mortality predictive scales incorporate the DRG medical/surgical splits (Hornbrook, 2003). Therefore, this limitation is common to all of the tools assessed here.
Methodological Distinctions. The points below provide an overview of key distinguishing characteristics related to each system’s methodology, for those systems with open methodologies. DRGs and RDRGs are excluded because they are most optimally applied to the Medicare population.
APR-DRG
- Subclasses are based on 1) the severity level of secondary diagnoses and 2) interactions between secondary diagnoses, age, principal diagnosis, and certain procedures. (Davies, 2001)
- Provides subclasses for severity of illness (resource use and complications) and risk of mortality.
- A noted limitation of the APR-DRG is that some biases still remain in the system in payment applications for neonates; however, this issue has limited impact on the current project. (Hornbrook, 2003)
APS-DRG
- Maintains a dynamic linkage to Medicare DRGs (Hornbrook, 2003); however, CMS contracts with 3M to maintain the DRG system, which will help facilitate integration of DRG refinements into the APR-DRG methodology
- Apply the DRG grouping logic to all patients, including neonates (Hornbrook, 2003)
- Use only diagnosis data (not procedure data) to assign severity subclasses within DRGs (Hornbrook, 2003); however, as noted above, all DRG-based systems implicitly integrate surgical procedures into the classification system.
- Support case-mix grouper research with its open coding system (Hornbrook, 2003); however, this advantage has little relevance to the current project.
Disease Staging
- Will assign the principal diagnosis to the correct disease stage and level, even if the principal diagnosis is a complication of another disease; thus, it can compensate for arbitrary ordering of diagnosis codes and outputs the appropriate disease category rather than treating health problems independently (Hornbrook, 2003)
- Designed to be a measure of disease progression.
Conclusions. Each risk adjustment system has strengths and weakness from all perspectives. Based on statistical performance, APR-DRGs perform equally or better than other systems. From a methodological perspective, areas in which APR-DRGs are weak have limited relevance to this project as well as foreseeable future projects. APR-DRG’s, in comparison to Disease Staging, have the added benefits of DRG-based systems (the universally used DRG system as their basis, and application to smoothing methods) as well as open mortality and resource use prediction logic. Finally, APR-DRGs are the most widely accepted system and have undergone the most extensive comparative and validity testing of any system, resulting in their integration into AHRQ’s quality indicator programs. Therefore, we recommend APR-DRGs as the risk-adjustment system for public hospital reporting.
“The APR-DRGs are by far the most widely used casemix classification system in the U.S. This systems is used in about 1,500 hospitals at this point and is used for payment purposes.” (Hornbrook, 2003). Additionally, the APR-DRG system is used by almost all of the state data organizations and health agencies currently reporting comparative hospital data on resource use and mortality. (Davies, 2001)
Some examples of organizations that have recommended APR-DRG for severity adjustment:
- The Medicare Payment Advisory Committee (MedPAC) recommended APR-DRG for severity adjustment of Medicare reimbursement.
- The Agency for Health Research and Quality (AHRQ) selected APR-DRG to risk-adjust their Quality Indicators for the following reasons (Davies, 2001):
- “Open” system
- Low additional cost to hospitals and states
- One risk adjustment system for resource use and risk of mortality.
- Official recognition by federal and state agencies. Additionally, 3M’s responsibility for maintaining the DRG grouper will facilitate integration of DRG refinements into the APR-DRG methodology.
- Maryland’s Health Services Cost Review Commission selected the APR-DRG grouper for severity adjustment of their Medicaid payment system. The recommendation was developed by a workgroup of hospitals, payers, and Commission staff.
In addition, it should be noted that the 3M has partnered with a number of IT firms who use the APR-DRG system to build more complex third party systems including:
- 3M Health Information Services Health Data Management Products
- McKesson Trendstar and Horizon
- ACS-Midas
- QuadraMed
- Solucient
- Premier Perspective
- HealthShares
- HealthGrades
- University HealthSystem Consortium
Davies SM, Geppert J, McClellan M, McDonald KM, Romano PS, Shojania KG. (2001) Refinement of the HCUP Quality Indicators. Technical Review No. 4. AHRQ Publication No. 01-0035. Rockville, MD: Agency for Healthcare Research and Quality.
Hornbrook MC. (2003) Disease Severity Adjustment Software Evaluation for the Healthcare Cost and Utilization Project. A report submitted to AHRQ.
Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD. Predicting who dies depends on how severity is measured: implications for evaluating patient outcomes. Annals of Internal Medicine. 1995 Nov 15;123(10):763-70.
Iezzoni LI, Shwartz M, Ash AS, Mackiernan YD. Predicting in-hospital mortality for stroke patients: results differ across severity-mesurement methods. Medical Decision Making. 1996 Oct-Dec; 16(4):348-56.
Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD. Judging hospitals by severity-adjusted mortality rates: the influence of the severity-adjustment method. American Journal of Public Health. 1996 Oct;86(10):1379-87.
Iezzoni LI, Shwartz M, Ash AS, Hughes JS, Daley J, Mackiernan YD. Severity measurement methods and judging hospital death rates for pneumonia. Medical Care. 1996 Jan;34(1):11-28.
Iezzoni L, Risk Adjustment for Measuring Health Care Outcomes, 3rd Ed. (2003).
Landon B, Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD. Judging hospitals by severity-adjusted mortality rates: the case of CABG surgery. Inquiry. 1996 Summer;33(2):155-66.
|
