Why is the Bayesian method not employed for diagnosis in practice

Bruno, Michael mbruno at PENNSTATEHEALTH.PSU.EDU
Sun May 27 12:53:26 UTC 2018


Thanks Dr. Bimal for starting this interesting thread, and to Stefanie Lee for sharing that excellent 2006 paper from Blackmore, et al., as well as to Dr. Oldham for his insightful comments.



I think the Blackmore paper is really touching on the topic of "signal detection theory," which is a very useful tool to understand about how useful information is actually extracted from complex data sets (like CT scans) that have extremely high levels of uncertainty built-in. We radiologists practice in a milieu of extraordinarily high undertainty, as I noted in my 2017 review in our Society's official journal, Diagnosis (DOI 10.1515/dx-2017-0006), so this is particularly relevant to us.



I've attached an excellent PDF discussion of signal-detection theory as applied to diagnostic radiology to this message for anyone who might be interested.  While radiologists and others may recognize the use of ROC curves, which are a feature of this theory, they may not be as familiar with the concepts of d', a quantitative measure of how well the alternatives can actually be discriminated from the test, and the concept of "criterion," whereby the interpreter decides how sensitive vs. specific they wish to be.  This was the thrust of the Blakemore article, and his meta-analysis suggested that radiologists by-and-large got it right.  (For David Meyers--the attached PDF is in the public domain, provided by Professor David Heeger of NYU).



Dr. Oldham also got it right, of course, saying that Baysean reasoning is not everything in diagnosis, but I believe that it takes us most of the way.  I very much appreciate Dr. Oldham's analogy that the radiologist is acting as the "expert witness" in a courtroom setting while the treating physician and patient / patient's family serves as the judge and jury, who are charged with ultimately deciding what "truth" is.



Have a terrific Memorial Day Weekend, everybody!



Mike





________________________________
From: James Oldham <james.oldham at HEALTH.NSW.GOV.AU>
Sent: Sunday, May 27, 2018 1:49 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Subject: Re: [IMPROVEDX] Why is the Bayesian method not employed for diagnosis in practice

Dear all
There is a difference between the treating doctor who makes a probabilistic diagnosis and evidence providers such as radiologists who comment on phenomena the observe and interpret it - taking Bayesian prior probability calculations explicitly into their offerings. I think the testing physician and their patients and family as a collective Judge relying on evidence provided by expert witnesses. So this final stage will be informed by Bayesian logic but not ruled by it and certainly not trying to make a prediction- just best information to hand
James Oldham
Child Psychiatrist
NSW

from  James Oldham - sent from my phone. (Please accept apologies odd sentence construction and creative additions courtesy of Apple autocorrect)

On 27 May 2018, at 13:25, Stefanie Lee <stefanieylee at GMAIL.COM<mailto:stefanieylee at GMAIL.COM>> wrote:
An interesting paper from Blackmore and Terasawa, JACR 2006 on the complexity of CT interpretation and how clinical probability may be factored into the decision to call a test result positive or negative:
"In this paper, we present an example of how health utility assessment can be used to guide the optimum interpretation of an imaging test."

"Radiologists have the ability to alter the sensitivity and specificity of their interpretations. For objective positivity criteria, different thresholds can be chosen to consider test results positive. For example, with appendicitis, one important factor in CT interpretation is the size of the appendix. Using a lower size threshold (eg, 6 mm) to consider an appendix abnormal will result in higher sensitivity for appendicitis, at the expense of lower specificity. Using a higher size measurement to consider the appendix abnormal (eg, 8 mm) would result in lower sensitivity but higher specificity. For subjective criteria such as periappendiceal fat stranding, the process is the same but less explicit. Individual radiologists can alter how much increase in the density of the fat is necessary to be considered abnormal "stranding.""
"In general, when a disease is rare and there are substantial costs to a false-positive diagnosis, interpreting a test with high specificity will maximize patient benefit. In contrast, when a disease is common and there are substantial costs for a false-negative diagnosis, interpretation at high sensitivity will maximize utility."
"Our prior meta-analysis indicated that radiologists interpret CT scans with approximately equal sensitivity and specificity. The current analysis indicates that this is an appropriate threshold at the intermediate probability of disease at which CT is commonly used today. If CT is to be used in populations at higher or lower probabilities of disease, then different imaging thresholds will be appropriate."

On 15 May 2018 at 13:18, Jain, Bimal P.,M.D. <BJAIN at partners.org<mailto:BJAIN at partners.org>> wrote:
In this attached paper, I discuss that the prescribed Bayesian method is not employed for diagnosis in practice because probability of a diseases is considered evidence for it in a given patient in this method, which is incorrect as a probability is a frequency in a population. This leads to all sorts of errors in practice which I discuss.
The correct method of diagnosis, which is employed in practice, consists of hypothesis generation and verification in which evidence is assessed by a likelihood ratio which locates it in the given patient of interest.
Please review and comment on this paper.

Thanks

Bimal

Bimal P Jain MD
Northshore Medical Center
Lynn MA 01904.

The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX
Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/







Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine


HTML Version:
URL: <../attachments/20180527/60a469b2/attachment.html> ATTACHMENT:
Name: Signal Detection Theory.pdf Type: application/pdf Size: 208896 bytes Desc: Signal Detection Theory.pdf URL: <../attachments/20180527/60a469b2/attachment.pdf>


More information about the Test mailing list