Sometimes good intentions have unintended consequences. Take, for example, using rates of blood clots (venous thromboembolisms, or VTEs) after surgery as a quality measure, with higher rates theoretically pointing to suboptimal preventive measures.
Developing blood clots after surgery is a common complication, and the postoperative morbidity and death associated with VTEs are potentially preventable through such measures as anticlotting drugs and getting patients moving around soon after surgery. To assess whether physicians and hospitals are reducing preventable blood clot formation after surgery, the Agency for Healthcare Research and Quality developed a risk-adjusted postoperative blood clot rate measure that was endorsed by the National Quality Forum and has been incorporated into numerous quality improvement programs and public reporting initiatives.
But a study appearing today in JAMA suggests such a measure might be flawed because of “surveillance bias”—which means you might find more of something not because it’s become more common but because you’ve stepped up efforts to find it. In the case of VTEs, the study suggests, some hospitals might find more VTEs in their patients because of increased efforts to look for VTEs, not because the number of blood clots has actually increased.
Researchers using data from 2838 hospitals and Medicare claims data from 954926 surgical patient discharges from 2786 hospitals found that institutions with increasing quality scores had higher rates of using measures to prevent VTE but also higher rates of blood clots. In addition, they found increased hospital VTE event rates were associated with increasing hospital VTE imaging rates.
Lead author Karl Y. Bilimoria, MD, MS, of Northwestern University and Northwestern Memorial Hospital in Chicago, discusses his team’s findings.
news@JAMA: Why did you do this study?
Dr Bilimoria: We heard from a number of institutions saying that no matter what they did, they couldn’t drive down their VTE rates. Here at Northwestern, we’ve worked on this for the past 4 or 5 years, and it seemed we got worse over time.
news@JAMA: Why did it seem to get worse?
Dr Bilimoria: What happened was we were putting so much emphasis on VTEs that we started finding more, and this created a vicious cycle. And we heard this from other top-quality hospitals. How frequently hospitals use imaging to detect a VTE is strongly associated with the number of VTEs they find, which makes them look worse on the rankings. All this suggests is that there is a surveillance bias around the measurement.
news@JAMA: So postsurgical blood clot development should not be a quality measure?
Dr Bilimoria: Everybody wants to measure quality around hospital care and VTE is one of the most common metrics used in public reporting sites and internal sites, so it has huge implications for hospitals. We’re not saying you shouldn’t look for VTEs, but to assess quality [associated with VTE prevention], it may be better to look at process measures or adherence to guidelines.
news@JAMA: Does this surveillance bias have ramifications for patients?
Dr Bilimoria: If you’re a patient and look at these hospital quality rankings and find one has more blood clots after surgery, you may not go there. But that hospital may be more vigilant and have the ability to act better than the one you end up choosing.
news@JAMA: Does the surveillance bias have ramifications for physicians?
Dr Bilimoria: There is a huge potential for adverse consequences. If you get knocked for finding more VTEs, then maybe you look for them less or you use prophylaxis inappropriately in low-risk patients, and that is not without risk.