Nearly 3 decades ago, David Axelrod, MD, then commissioner of the New York State Department of Health (DOH), became concerned by the nearly 5-fold variation in cardiac surgery mortality rates across New York hospitals. Out of this concern, the DOH released the first public reports of surgical outcomes to the New York Times, labeling the performance of hospitals on cardiac surgical mortality rates across the state. After learning that individual surgeon mortality rates were also available, the Long Island–based Newsday sought release of the mortality rates of individual surgeons. The state demurred, arguing that data were less reliable at the surgeon level. Newsday quickly filed a Freedom of Information Act lawsuit to obtain the information and eventually prevailed in court, partly on the argument that patients generally choose surgeons, not hospitals.
This debate—whether it is best to report data on the hospital level only or whether to also report on individual surgeons—continues to this day. The evidence on whether the public or payers use these data to select high performers suggests that for the most part, they do not. But we do know that hospitals and physicians are paying attention. And there is widespread hope that with increasing consumerism in health care and with more patients paying out of pocket for their care, they will increasingly look for these kinds of reports.
That’s part of the reason public reporting is now so pervasive. While public reporting continues to become more common, how to report—whether to include only institutions or to also report on individual physicians—remains controversial. This debate was recently renewed when ProPublica, a highly regarded independent and nonprofit investigative journalist group, used national Medicare data to publish complication rates for hospitals and individual surgeons. It was their decision to name individual surgeons that was most controversial.
Criticism of Surgeon-Level Reporting
Critics of individual surgeon reporting have made a series of important arguments against the practice, each of which is worthy of consideration. And as more states and the federal government continue their efforts towards public reporting, these arguments against naming individual surgeons are gaining traction. But I believe that a careful examination of the literature indicates that moving away from surgeon-level reporting would be a critical mistake.
One common argument is that volumes of surgery for individual surgeons may be too low to be reliable. Small sample sizes can be a problem, but there are ways to address this issue. First, as New York State has done with cardiac surgery, one can aggregate performance across multiple years, thus substantially increasing sample size. Second, given that total volume of similar surgeries is also related to performance on a given surgery, (for example, a surgeon’s performance on esophagectomy improves with the number of other similar surgeries she performs) , one could use a surgeon’s performance across a range of procedures to increase sample size. Finally, publicly reported data could and should be presented with confidence intervals, to highlight the level of imprecision so that those reading the report are aware of the statistical limitations.
Critics of naming individual surgeons also argue that the practice would increase the likelihood that surgeons will avoid the hardest cases, thus reducing access to surgical care for the sickest patients. This is an important concern that has been voiced widely. However, the evidence on the extent to which this occurs is weak and anecdotal. Quite a few studies have tried to examine whether publicly reporting individual surgeon data leads to sicker patients not receiving care, and most have failed to find an effect. And to the extent that a few studies have found a reduction in access, these have been transient, with access returning to baseline within a couple of years of the reporting program.
Finally, and possibly most importantly, people who oppose reporting findings for individual surgeons have argued that much of medical care is now delivered in teams, and that each of the team members matters. They argue that naming an individual surgeon puts undue emphasis on that surgeon when the team’s performance is paramount. This argument is understandable; the importance of team work on surgical care is increasingly recognized. As the leader of the team, the surgeon plays a unique and outsized role, responsible for ensuring effective communication and well-delineated roles among team members, and for fostering a culture that supports team members speaking up when things have gone wrong. So when we publicly report a surgeon’s performance, we are essentially reporting on his or her surgical team’s performance.
Beyond Team Work
But even beyond team work, the empirical evidence of the importance of the individual surgeon’s role is overwhelming. Landmark studies have shown that within the same institution, there are large variations in outcomes across surgeons. Why might this be the case? It’s because outcomes of surgical care are influenced by a multitude of factors for which the judgment, skills, and capabilities of individual surgeons matter immensely.
Possibly the most consequential decision—whether to do the surgery at all—is undertaken primarily by the surgeon. Careful case selection—ensuring that the patient undergoing the procedure is likely to benefit—is critical to good surgical care, and reporting outcomes only at the institutional level would fail to capture the results of that judgment.
A second critical step in determining outcomes is the surgical technique itself. Recent studies have shown that there are large differences in basic surgical capabilities (from dissecting to cutting to suturing), and that highly skilled surgeons have dramatically better outcomes than less-skilled ones. The surgeon’s hands still matter. There are likely other factors that matter as well, but the empirical evidence is clear: picking the right surgeon is at least as important as picking the right hospital.
And that brings us to what is probably the most important reason to report data on individual surgeons: it’s information that patients want. When people seek advice on where to go for surgical care, they ask about the best surgeon for them. Telling the public that the surgeon doesn’t matter, that only the choice of hospital matters, is neither useful nor accurate. And it won’t work, because for patients, undergoing surgery creates deep anxiety, and trusting the judgment of an individual surgeon is paramount. Consumers would reject reports that only showcase institution-level data because they would find it less useful.
Publicly reporting on outcomes of individual surgeons is a high-stakes endeavor. We know from studies that some surgeons stopped practicing or moved away as a result of the New York State cardiac surgery reporting. There are still lingering fears that surgeons will turn away risky patients, although most surgeons have enough professionalism that they are unlikely to deny someone life-saving care just because it might hurt their rankings. But these fears mean we need to work harder at risk adjustment, and possibly create safe harbors for cases that are the highest risk.
But the surgeon remains critically important to surgery, and suggesting that what matters is the institution, not the individual, is both empirically inaccurate and unlikely to resonate with patients. We must address the challenges that come with individual surgeon-level reporting—not shy away from doing it. We must expand our efforts to report performance publicly, including for individual surgeons, not curtail them. Because if the goal is to improve surgical care, we must ensure that the person most influential in the process remains accountable for its outcome.
About the author: Ashish K. Jha, MD, MPH, is K. T. Li Professor of International Health and Health Policy at the Harvard T. H. Chan School of Public Health and a practicing internist at the Veterans Affairs Boston Healthcare System. He received his doctor of medicine from Harvard Medical School and was trained in internal medicine at the University of California, San Francisco. He received his master’s in public health from Harvard School of Public Health. Dr Jha’s major research interests lie in improving the quality and costs of health care. His work has focused on 4 primary areas—public reporting, pay for performance, health information technology, and leadership—and the roles they play in fixing the US health care system. (Image: Aubrey LaMedica/Harvard T.H. Chan School of Public Health)
About The JAMA Forum: JAMA has assembled a team of leading scholars, including health economists, health policy experts, and legal scholars, to provide expert commentary and insight into news that involves the intersection of health policy and politics, economics, and the law. Each JAMA Forum entry expresses the opinions of the author but does not necessarily reflect the views or opinions of JAMA, the editorial staff, or the American Medical Association. More information is available here and here.