Kahneman interview - putting a premium on error detection

Bruno, Michael mbruno at PENNSTATEHEALTH.PSU.EDU
Mon Feb 11 22:16:40 UTC 2019

Thanks, Tom.  Yes, that might be illustrative for this forum.

To Jason’s point – I think the dDx is definitely a great cognitive tool.  We can de-bias ourselves just by asking, “what else, how else, why else?”

To Gerrit’s point – absolutely there are myriad “no fault” errors.  Like when we miss the diagnosis completely in an asymptomatic patient, or when there are no lab or imaging findings that can help us.  And all false-negative tests.

Kathryn Shultz’ book is a must-read:  https://www.amazon.com/Being-Wrong-Adventures-Margin-Error/dp/0061176052

She also has a great TED talk:  https://www.ted.com/talks/kathryn_schulz_on_being_wrong?language=en


From: Tom Benzoni [mailto:benzonit at GMAIL.COM]
Sent: Monday, February 11, 2019 1:49 PM
Subject: Re: [IMPROVEDX] Kahneman interview - putting a premium on error detection

== CAUTION: External Sender ==
This message came from outside of Penn State Health & College of Medicine.
Please use caution when opening attachments or links.

Would this list like to see what currently passes for diagnoses?
Since we went Electronic, billing codes are used for dx.

On Mon, Feb 11, 2019, 12:15 Jason Maude <jason.maude at isabelhealthcare.com<mailto:jason.maude at isabelhealthcare.com>> wrote:
This circles us back to the differential diagnosis which is a great trigger to make us stop and think and then, perhaps, realise that we may be wrong. However, it would have to be required for every patient and recorded in the notes to be able to play the role of a trigger. If that happened then it could easily become  a measure!

Jason Maude
Founder and CEO Isabel Healthcare
Tel: +44 1428 644886
Tel: +1 703 879 1890

From: Gerrit Jager <gerrit.jager at PLANET.NL<mailto:gerrit.jager at PLANET.NL>>
Reply-To: Society to Improve Diagnosis in Medicine <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>>, Gerrit Jager <gerrit.jager at PLANET.NL<mailto:gerrit.jager at PLANET.NL>>
Date: Saturday, 9 February 2019 at 17:16
Subject: Re: [IMPROVEDX] Kahneman interview - putting a premium on error detection

I fully agree with Mike, but I like to add another category; the no-fault errors. The sensitivity of a diagnostic test is almost never 100%, e.g. a chest X-ray has too little spatial resolution to detect small nodules and in most patients missing them is an unavoidable misdiagnosis related to the limitations of the test. But abandon chest X-ray and perform CT in every patient is not the alternative.
(It is a different story when the selection of an imaging test is inappropriate.)

So the message is that we should always be aware and prepared that we may be wrong and be willing to reconsider a diagnosis.

This is not easy as excellently described by Kathryn Schultz in her book “Being Wrong, adventures in the margin of error”.  She explains why we are so bad at imaging that we are mistaken.

Gerrit Jager

Op 07-02-19 15:59, Bruno, Michael <mbruno at PENNSTATEHEALTH.PSU.EDU<http://mbruno@PENNSTATEHEALTH.PSU.EDU>> schreef:
Thanks, Nelson,

Yes, I agree—process changes, but also cognitive changes and an accommodation to our basic human biology.  I like to think of errors as falling into three categories: (1) those which are due to faults in our work processes, (2) t hose which are due to faults in our thinking, and (3) those which occur simply because of how we are made.  Each of these will be amenable to different solutions—but only error detection after the fact addresses all three.

While there are a few diagnoses which, if missed, lead to immediate and irreparable harm, most do not.  If we learn to be inclined to expect, search for, and rapidly detect errors we will prevent harm MOST of the time.  In our experience in radiology, we do on regular occasion detect errors which have been present for several hours, such as when a finding is missed overnight but picked up on re-review in the morning.  In virtually all of these, patient harm was averted by correcting the diagnosis a few hours later.

All the best,


From: Nelson Toussaint [mailto:ntoussaint at tamarac.com]
Sent: Thursday, February 07, 2019 8:36 AM
To: 'Society to Improve Diagnosis in Medicine'; Bruno, Michael
Subject: Kahneman interview - putting a premium on error detection

February 7, 2019
8:16 AM


In Aerospace, it is anticipated that this behavior will occur.  Equipment will fail and people will make mistakes!  So, a concept of Accommodation is built into the process to counteract those faults that could lead to serious harm.  In some cases you can detect the errors and in some the consequences are in play before you can confirm a detection.  So where real harm can appear, the Accommodation needs to be somewhat active before/as the fault occurs.

An example is the Ground Proximity Warning System, which estimates if the airplane may soon fly into terrain (navigation fault of the pilot or equipment).  This is only a warning and the pilot must take an action to change course.

Early detection of diagnostic error is good; but it still may not be in time to limit the potential harm.  A more robust process would include methods where the "conclusions" are reviewed before a diagnosis is complete.  One that comes to mind is what many of us use quite often - "what does this seem like to you"; a discussoin amongst involved parties such as the attending physician - radiologist/pathologist – patient.  This Accommodation can serve to cause the “experts” to pause and reasses their findings.  It does not give a foolproof solution, but still allows the experts to continue to form the diagnosis.

This is just a thought, but it seems to me the process changes are the best way to attack this problem in order to get broad participation.  The trick will be how to modify the process without penalizing the many straight-forward cases.

 Nelson Toussaint

ntoussaint at tamarac.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__ntoussaint-40tamarac.com&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=y10B2V6S_hbcG5V4c3Oi8Du1tGNGShAXnQTmFsnkOiU&e=> <mailto:ntoussaint at tamarac.com>

From: Bruno, Michael [mailto:mbruno at PENNSTATEHEALTH.PSU.EDU]
Sent: Tuesday, February 05, 2019 12:41 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<https://urldefense.proofpoint.com/v2/url?u=http-3A__IMPROVEDX-40LIST.IMPROVEDIAGNOSIS.ORG&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=XgKy10ZBkAAspTK9C1C4lvLCgevLg2FPIpnAz8UZ2v4&e=>
Subject: Re: [IMPROVEDX] Kahneman interview - putting a premium on error detection

Yes, thanks, Art!

It seems to me that while some errors are preventable, others may well be inevitable. The perceptual errors we were talking about at that session at DEM are probably biologically rooted (i.e., secondary to neurocognitive brain network functions) and are thus essentially outside of our conscious control. They are an example of an entire class of errors that flow from “how we are made,” our biology and our evolution, and so they will not respond to the usual interventions, such as cognitive de-biasing, adult learning/CME, or even mindfulness.

So it seems to me that, for these types of errors at least, there needs to be a premium placed on early error detection, so that errors can be more promptly detected and corrected before any patient harm is done.  That is the value of double-reading in radiology, and it may be an avenue where AI turns out to be particularly helpful in the years ahead.

All the best,

Michael A. Bruno, M.D., M.S., F.A.C.R.
Professor of Radiology & Medicine
Vice Chair for Quality & Patient Safety
Chief, Division of Emergency Radiology
Penn State Milton S. Hershey Medical Center
• (717) 531-8703  |  6 (717) 531-5737
• mbruno at pennstatehealth.psu.edu<http://mbruno@pennstatehealth.psu.edu> <mailto:mbruno at pennstatehealth.psu.edu>  |

*****E-Mail Confidentiality Notice*****
This message (including any attachments) contains information intended for a specific individual(s) and purpose that may be privileged, confidential or otherwise protected from disclosure pursuant to applicable law.  Any inappropriate use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalty.  If you have received this transmission in error, please reply to the sender indicating this error and delete the transmission from your system immediately.

From: Art Papier [mailto:apapier at VISUALDX.COM]
Sent: Sunday, February 03, 2019 2:34 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<https://urldefense.proofpoint.com/v2/url?u=http-3A__IMPROVEDX-40LIST.IMPROVEDIAGNOSIS.ORG&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=XgKy10ZBkAAspTK9C1C4lvLCgevLg2FPIpnAz8UZ2v4&e=>
Subject: Re: [IMPROVEDX] Kahneman interview

Thanks for sharing!   Great interview with many fascinating threads, and some new thoughts on the randomness of error, and how all error is not due to cognitive bias.  Towards the conclusion we hear that Dr. Kahneman in not a believer in cognitive debiasing…. saying essentially we are too busy making errors to recognize that we are making errors.  He asserts we should be thinking about how we recognize other peoples errors.  In made me think of the session at DEM on perceptual errors in diagnostic imaging and the very positive role of second reads in radiology….whether by humans or AI to recognize errors.  Perhaps either co-decision making, or AI “second opinions” is an area we should all be further exploring.

Art Papier MD
CEO VisualDx
Associate Professor of Dermatology and Medical Informatics
University of Rochester College of Medicine


From: Xavier Prida <dr.xavier.prida at GMAIL.COM<https://urldefense.proofpoint.com/v2/url?u=http-3A__dr.xavier.prida-40GMAIL.COM&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=X71fZJSJh6zGoXvpAHVDy1ayOxc34uNv6Drs-vKQeRE&e=>>
Sent: Sunday, February 03, 2019 8:59 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<https://urldefense.proofpoint.com/v2/url?u=http-3A__IMPROVEDX-40LIST.IMPROVEDIAGNOSIS.ORG&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=XgKy10ZBkAAspTK9C1C4lvLCgevLg2FPIpnAz8UZ2v4&e=>
Subject: [IMPROVEDX] Kahneman interview

On "thinking again"- not changing your mind, error, and bias - the latter two are not always linked.

https://onbeing.org/programs/daniel-kahneman-why-we-contradict-ourselves-and-confound-each-other-jan2019/<https://urldefense.proofpoint.com/v2/url?u=https-3A__onbeing.org_programs_daniel-2Dkahneman-2Dwhy-2Dwe-2Dcontradict-2Dourselves-2Dand-2Dconfound-2Deach-2Dother-2Djan2019_&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=BKmxto84rxjyXIkrbS2yJvF3d8P7A4S0QkLUwfvaDLQ&e=> <https://urldefense.proofpoint.com/v2/url?u=https-3A__onbeing.org_programs_daniel-2Dkahneman-2Dwhy-2Dwe-2Dcontradict-2Dourselves-2Dand-2Dconfound-2Deach-2Dother-2Djan2019_&d=DwMGaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=UrbAzwfML-iEIuCykgw2Fqn20pE6WrN96IDUMbiOHMs&s=3X_IjCf1rsQokLp5q2wzaZqF1pXrJky5MKUV2tVppnQ&e=>


praesent superare odio (rise above)

Xavier E. Prida MD FACC FSCAI

Assistant Professor of Medicine

Program Director Cardiology Fellowship Training

USF Morsani College of Medicine

Department of Cardiovascular Sciences

2 Tampa General Circle

STC 5 th Floor

Tampa, Fl 33606

813 259 0992(O)


Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<https://urldefense.proofpoint.com/v2/url?u=http-3A__IMPROVEDX-40LIST.IMPROVEDIAGNOSIS.ORG&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=XgKy10ZBkAAspTK9C1C4lvLCgevLg2FPIpnAz8UZ2v4&e=>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__list.improvediagnosis.org_scripts_wa-2DIMPDIAG.exe-3FSUBED1-3DIMPROVEDX-26A-3D1&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=vY_4hwriF6bdGCVu9f3FevTH74HU6G_ySVECi2Py81k&e=> <https://urldefense.proofpoint.com/v2/url?u=http-3A__list.improvediagnosis.org_scripts_wa-2DIMPDIAG.exe-3FSUBED1-3DIMPROVEDX-26A-3D1&d=DwMGaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=UrbAzwfML-iEIuCykgw2Fqn20pE6WrN96IDUMbiOHMs&s=PBCG5hjllVPSANZQoKcMqMD2ZiVXDpIMwesAc65piDE&e=>
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<https://urldefense.proofpoint.com/v2/url?u=http-3A__IMPROVEDX-2DSIGNOFF-2DREQUEST-40LIST.IMPROVEDIAGNOSIS.ORG&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=snJhzpQ9uTloe9Y-g3V6BYo5uX9eznJo78mhRqdd8G4&e=>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX<https://urldefense.proofpoint.com/v2/url?u=http-3A__list.improvediagnosis.org_scripts_wa-2DIMPDIAG.exe-3FINDEX&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=Dcadety29psOr_-8E0LX0qxLmSo1NK4T3ktR6VYc8HI&e=> <https://urldefense.proofpoint.com/v2/url?u=http-3A__list.improvediagnosis.org_scripts_wa-2DIMPDIAG.exe-3FINDEX&d=DwMGaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=UrbAzwfML-iEIuCykgw2Fqn20pE6WrN96IDUMbiOHMs&s=OeKMCzd2zeZiCtm1TGcYnGbJmDibC0OzIARvu7ewz1o&e=>

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.improvediagnosis.org_&d=DwMFaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=xmS5X07s5qt6BK5-a5NnkxubOnqHUjCKYeS_4k0ksOs&s=ptvlHYf8skqQugj2ZK6mmWOTMwsOvOGAH5InnnGbRc8&e=> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.improvediagnosis.org_&d=DwMGaQ&c=_FmMnDvUH5queZcSmOuBzHZMbp7E7EwtGwv5cxxnTj0&r=XZJky8Jx0OuETXcWpBMhx9j_wSYpSZPDVXdInJ5O9gQ&m=UrbAzwfML-iEIuCykgw2Fqn20pE6WrN96IDUMbiOHMs&s=p9DtK4wdXsBYva9iE7z3eA3n1OqmC6On22QUAHNrHbo&e=>

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:

To unsubscribe from the IMPROVEDX:
or click the following link: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG


For additional information and subscription commands, visit:

http://LIST.IMPROVEDIAGNOSIS.ORG/ (with your password)

Visit the searchable archives or adjust your subscription at:

Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br>
<a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1" target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1</a>

HTML Version:
URL: <../attachments/20190211/0d71f6c2/attachment.html>

More information about the Test mailing list