Balancing misclassification costs
Michael.Kohn at UCSF.EDU
Mon Dec 16 20:23:14 UTC 2013
Dear Brian et al.
Simplify the situation of a patient presenting to the ED with chest pain. After all testing to refine probabilities, you have determined a probability P that this is acute cardiac ischemia (ACI) and a probability 1-P that it is esophageal reflux (an obvious oversimplification). There are two errors you can make: 1) send the patient home when he has ACI, and 2) hospitalize the patient when he has reflux. Error 1 results in increased mortality risk and other problems commonly subsumed under the letter "B" (which represents the foregone "benefits" of treatment). Error 2 results in increased risk of iatrogenic complications as well as ultimately unnecessary financial/resource costs commonly subsumed under the letter "C". If B = 9C, then you should hospitalize for P > 0.1 (10%), as in your scenario. At the margin, you will be admitting 9 patients unnecessarily (Error 2) for every one that you avoid sending home with ACI (Error 1). There is simply no way to eliminate both types of error. You could eliminate Error 2 by admitting everybody, but only at the cost of countless unnecessary hospitalizations. In an uncertain world, zero tolerance for error is impossible and irrational.
Michael A. Kohn, MD, MPP
Epidemiology and Biostatistics
Attending Emergency Physician
Mills-Peninsula Medical Center
From: Jackson, Brian [brian.jackson at ARUPLAB.COM]
Sent: Monday, December 16, 2013 7:59 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Subject: Re: [IMPROVEDX] Error Rates: Diagnosis--neither numerator nor denominator is known
Suppose a physician judges that a patient has a 10% chance of a serious, treatable condition, and that initiating treatment immediately outweighs the risks. And then suppose that the patient turns out not to have that particular condition. Was the initial action based on a misdiagnosis? Or would the opposite action, namely withholding treatment, have been considered an error? Now imagine that there's a clinical practice guideline that explicitly recommends immediate treatment for this condition provided that the probability is judged to be at least 10%. That seems to put the physician on solid ground, right? But now imagine that the patient died from the side effect of the treatment, and on retrospective review (M&M?) a different expert physician judges that the patient had only a 5% a priori probability of that condition. Now, was it an error?
(Some readers might want to cop out by calling this a question of therapeutic error, but the assumption here is that the treatment decision follows directly from the diagnostic assessment.)
Given that medical diagnosis deals in probabilities rather than absolutes, and that many cases have considerable ambiguity, I'm concerned about the potential consequences of labeling specific incidents as errors. The legal industry boils things down to absolutes, and we've seen how that works. Our goal is to avoid errors, but maybe we can't measure individual errors directly, and it may even be counterproductive to even try to do so. Indirect (process) measurement may well be more practical for most situations. And outcome measures might be best based on estimation over large data sets. In principle, any of these measures could be framed either positively (diagnostic success measures) or negatively (error measures) but based on Dr. Centor's suggestion, use of "error" terminology might be better reserved for use in the abstract, and "success" terminology for specific, labeled settings.
Brian R. Jackson, MD, MS
VP - Chief Medical Informatics Officer, ARUP Laboratories
Assoc. Professor of Pathology (Clinical), University of Utah
500 Chipeta Way, Mail Code 100
Salt Lake City, Utah 84108-1221
phone: (801) 583-2787, extension 1-3191
toll free: (800) 242-2787
email: brian.jackson at aruplab.com
More information about the Test