[EXTERNAL][IMPROVEDX] Balancing misclassification costs & numerators\denominators

Graber, Mark Mark.Graber at VA.GOV
Tue Dec 17 02:40:05 UTC 2013

I just wanted to say that this has been one of the best listserv discussions we've had, so thanks to everyone who's contributed, and hopefully others will join in too.

The discussion recapitulates parts of the debate we had at the DEM meeting on the definitions of diagnostic error.  Its true that we can go round and round if we try to define diagnostic error as a breakdown in the process of diagnosis, because we get into the issues of expert opinion, what is the standard of care, and the problems Brian has identified.  But .... If we define diagnostic error as a noun referring to the outcome itself, things get much simpler.  In this case, the patient in Brian's case
DOES represent a diagnostic error and it doesn't matter what the probability was, or what the guideline said, or what the various experts judged the probability to be.  I know this doesn't clarify what we should do about measurement, and I grant that being able to know with some degree of confidence that the diagnosis was "A" and not "B" can only be done in a few cases, and only in retrospect.

Seems to me that the trick is, as sage Centor has pointed out, is to focus on how to improve the diagnostic process to get the best outcome.  Things like a better physical exam, as Georges is working on, is a major advance; as would be a better test to differentiate ACI from esophageal reflux.  As Michael has pointed out, moving up and down the same curve is just a trade off. We need a better curve!

From: "Kohn, Michael" <Michael.Kohn at UCSF.EDU>
Reply-To: Society to Improve Diagnosis in Medicine <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, "Kohn, Michael" <Michael.Kohn at UCSF.EDU>
Date: Mon, 16 Dec 2013 15:23:14 -0500
Subject: [EXTERNAL][IMPROVEDX] Balancing misclassification costs

Dear Brian et al.

Simplify the situation of a patient presenting to the ED with chest pain.  After all testing to refine probabilities, you have determined a probability P that this is acute cardiac ischemia (ACI) and a probability 1-P that it is esophageal reflux (an obvious oversimplification).  There are two errors you can make: 1) send the patient home when he has ACI, and 2) hospitalize the patient when he has reflux.  Error 1 results in increased mortality risk and other problems commonly subsumed under the letter "B" (which represents the foregone "benefits" of treatment).  Error 2 results in increased risk of iatrogenic complications as well as ultimately unnecessary financial/resource costs commonly subsumed under the letter "C".  If B = 9C, then you should hospitalize for P > 0.1 (10%), as in your scenario.  At the margin, you will be admitting 9 patients unnecessarily (Error 2) for every one that you avoid sending home with ACI (Error 1).  There is simply no way to eliminate both types of error.  You could eliminate Error 2 by admitting everybody, but only at the cost of countless unnecessary hospitalizations.  In an uncertain world, zero tolerance for error is impossible and irrational.


Michael A. Kohn, MD, MPP
Associate Professor
Epidemiology and Biostatistics

Attending Emergency Physician
Mills-Peninsula Medical Center
Burlingame, CA

From: Jackson, Brian [brian.jackson at ARUPLAB.COM]
Sent: Monday, December 16, 2013 7:59 AM
Subject: Re: [IMPROVEDX] Error Rates: Diagnosis--neither numerator nor denominator is known

Suppose a physician judges that a patient has a 10% chance of a serious, treatable condition, and that initiating treatment immediately outweighs the risks.  And then suppose that the patient turns out not to have that particular condition.  Was the initial action based on a misdiagnosis?  Or would the opposite action, namely withholding treatment, have been considered an error?  Now imagine that there's a clinical practice guideline that explicitly recommends immediate treatment for this condition provided that the probability is judged to be at least 10%.  That seems to put the physician on solid ground, right?  But now imagine that the patient died from the side effect of the treatment, and on retrospective review (M&M?) a different expert physician judges that the patient had only a 5% a priori probability of that condition.  Now, was it an error?

(Some readers might want to cop out by calling this a question of therapeutic error, but the assumption here is that the treatment decision follows directly from the diagnostic assessment.)

Given that medical diagnosis deals in probabilities rather than absolutes, and that many cases have considerable ambiguity, I'm concerned about the potential consequences of labeling specific incidents as errors.  The legal industry boils things down to absolutes, and we've seen how that works.  Our goal is to avoid errors, but maybe we can't measure individual errors directly, and it may even be counterproductive to even try to do so.  Indirect (process) measurement may well be more practical for most situations.  And outcome measures might be best based on estimation over large data sets.  In principle, any of these measures could be framed either positively (diagnostic success measures) or negatively (error measures) but based on Dr. Centor's suggestion, use of "error" terminology might be better reserved for use in the abstract, and "success" terminology for specific, labeled settings.

Brian R. Jackson, MD, MS
VP - Chief Medical Informatics Officer, ARUP Laboratories
Assoc. Professor of Pathology (Clinical), University of Utah

500 Chipeta Way, Mail Code 100
Salt Lake City, Utah 84108-1221
phone: (801) 583-2787, extension 1-3191
toll free: (800) 242-2787
fax: (801)584-5108
email: brian.jackson at aruplab.com
web: www.aruplab.com

Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br>
<a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1" target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1</a>

More information about the Test mailing list