Error Rates: Diagnosis--neither numerator nor denominator is known

Ross Koppel rkoppel at SAS.UPENN.EDU
Thu Dec 12 03:27:48 UTC 2013

The discussion on measurement of error and error rates is wonderful and 
What seems to be missing from much of this conversation, however, is 
that reality that we usually have no measures of the numerator -- the 
number of errors;  and we have an often slippery measure of the 
denominator-- number of diagnoses, number of correct diagnoses 
determined by case review and autopsies, number of correct diagnoses 
determined by a brighter doctor, number of correct dx when the patient 
was discharged (sic), number of med orders, number of med orders 
actually administered,  number of opportunities to order, number of 
patients, number of meds administered correctly, etc.
Of course, the denominator and the numerator obviously differ by the 
measure on which we focus:
Some errors are easy to spot, e.g.,  leaving hemostats in the thoracic 
cavity is a classic.
Med prescription errors are very hard to know.  Patients are sick, old, 
have 5 comorbidities and are on 13 other meds.  Bad things happen when 
we do the right thing, good things happen when we do the wrong thing, 
polypharmacy is pandemic, what is ordered may not have been 
administered, no one knows the drug-drug-drug interactions among 13 
drugs, etc.     Case review is dependent on the right Dx (tautology 
alert) and knowing which tests to order, etc.

And then we come to Dx errors. Again, some are easy, but, as 
demonstrated powerfully by this group's insightful conversation, many 
are profoundly hard to determine...especially in the first set of 
iterations.  Some may be impossible. Thus, the  discussion of error 
rates makes me cry for a massive dose of methodological caution and even 
more humility.

None of this is to say the discussion is neither productive nor 
exciting.  Of course not!  But the epistemological issues here are 
beyond most of the problems of science.  One of the reasons I so respect 
medical thinking is because physicians must deal with so many unknowns 
and uncertainties.  My colleague, Renee Fox, noted that one of the first 
things we teach a young doctor is to distinguish between what he/she 
does not know and what Medicine does not know.  With diagnostic error, 
we have both of those factors plus the more vexing problem of what is 
knowable and what is knowable within the constraints of a specific 
patient's life and setting.

Caveat: I'm not a physician, but I have spent the last 47 years focused 
on research methods, measurement of error, and statistics.

Ross Koppel, Ph.D. FACMI
Sociology Dept and Sch. of Medicine
Senior Fellow, LDI, Wharton
University of Pennsylvania, Phila, PA 19104-6299
215 576 8221 C: 215 518 0134

To unsubscribe from the IMPROVEDX list, click the following link:<br>
<a href="" target="_blank"></a>

More information about the Test mailing list