Potential Bias in U.S. News Patient Safety Scores

Potential Bias in U.S. News Patient Safety Scores

Hospitals can get overwhelmed by the array of ratings, rankings and scorecards that gauge the quality of care that they provide. Yet when those reports come out, we still scrutinize them, seeking to understand how to improve. This work is only worthwhile, of course, when these rankings are based on valid measures.

Certainly, few rankings receive as much attention as U.S. News & World Report’s annual Best Hospitals list. This year, as we pored over the data, we made a startling discovery: As a whole, Maryland hospitals performed significantly worse on a patient safety metric that counts toward 10 percent of a hospital’s overall score. Just three percent of the state’s hospitals received the highest U.S. News score in patient safety — 5 out of 5 — compared to 12 percent of the remaining U.S. hospitals. Similarly, nearly 68 percent of Maryland hospitals, including The Johns Hopkins Hospital, received the worst possible mark — 1 out of 5 — while nationally just 21 percent did. This had been a trend for a few years.

What could account for this discrepancy? Could we all really be doing this poorly in my home state and in our hospital, where we take great pride in our efforts to prevent patient harm? After lengthy analysis, it seems quite clear that the answer is no. Instead, the patient safety score appears to have a bias against Maryland hospitals, because the data from our state is incomplete and not consistent with the data reported for hospitals outside of Maryland.

Maryland’s Unique Arrangement

The U.S. News patient safety score rates hospitals on their track record for preventing seven health care-associated patient harms, such as punctured lung, hematoma and pressure ulcer. U.S. News derives this score by identifying Medicare billing claims that include the diagnosis codes for these harms. For this year’s rankings, this analysis used claims data from October 2010 through September 2013.

The differences between Maryland and other states involves how we account for complications that are “present on admission” and therefore are not the result of poor hospital care. The hematoma that a patient suffered in an auto accident, for example, should not be attributed to the care he or she received in the hospital. Since late 2007, hospitals outside of Maryland have been required to add codes to their Medicare billing claims to indicate such present-on-admission conditions or face financial penalties for not doing so.

But in Maryland, we have a longstanding and unique arrangement with Medicare that has allowed us to participate in our state’s pay-for-quality programs instead of the federal program. This essentially requires Maryland hospitals to have two data sets, one we submit to Medicare for billing and one we submit to the state for quality reporting. It wasn’t until October 2014 — after the period analyzed for this year’s U.S. News patient safety score — that that Medicare program started requiring present-on-admission codes from Maryland hospitals. On the other hand, Medicare required these codes from non-Maryland hospitals starting in 2007.

continue reading >>>