Published results of cognitive bias training or similar efforts

Pat Croskerry croskerry at EASTLINK.CA
Mon Dec 15 13:22:23 UTC 2014

Rob. I enjoyed your paper in Diagnosis – it was a helpful review and I agree that we need a balance in approaches.

However, I’m not comfortable with your assertion that the heuristics and biases approach has been disproportionately represented. Instead, it seems that we have devoted reasonable attention to the ‘mastery and maintenance ‘ of basic diagnostic competency. It is not difficult to find good presentations of the basic Bayesian and other biostatistical tools needed in the ‘analytical’ approach (e.g. Goutham Rao’s book on Rational Medical Decision Making) and historically medical schools have done a reasonable job of teaching this material. In contrast, I don’t know of any medical curriculum before about 5 years ago that coached students on recognizing biases, heuristics and the correction of mental shortcuts. There were certainly papers in the medical literature, and Kassirer and Kopleman’s book on Clinical Reasoning mentioned a handful of cognitive biases, but aside from John Brush’s recent admirable book (The Science of the Art of Medicine) – in which he achieves a nice balance of the two approaches, I haven’t seen any other book that delineates cognitive bias and discusses potential debiasing strategies. I would be happy to think we had spent a disproportionate amount of time on the latter but I don’t believe that is the case. While few would argue the merits of teaching analytical approaches, there are still those who deny that cognitive bias affects diagnostic reasoning! 


Recently,  I have heard it said a few times that there is no merit in teaching the operating characteristics of dual process theory. This surprises me and I find it odd.  It’s a bit like saying we will teach you how to operate a car but you don’t need to know anything about what is under the hood. I know there are some drivers who do get by without knowing anything about the engine that the car depends upon, but if the car malfunctions they have no clue what to do. The Dual Process approach is the engine of decision making and it is important to know how it works – it helps us in the context of diagnostic failure, where cognitive/affective elements make up a significant proportion of what goes wrong. Further, the current work that is going on with dynamic functional neuroimaging (fMRI) is very promising – research is beginning to unravel the spatial and temporal correlates around the neurological substrate of human decision making (see Wan et al, Science 2011) – it is inevitable that we will learn more and more about what hitherto has largely been a black box. This is not the time to back away from the possibility of gaining significant insight into how the engine functions. Science has never progressed by people sticking their heads in the sand


My final point is about the general pessimism around cognitive de-biasing i.e. the notion that it looks difficult and therefore we shouldn’t spend our time trying to undo biases. Certainly, some of the biases in System 1 are difficult to get at. Stanovich has broken down System 1 into 4 sub-groups and two of them look pretty resistant to change – those that are hard-wired from our evolutionary past, and those that have been deeply engrained though multiple repetitions. But there are a variety of strategies out there for dealing with them. Mark Graber’s group found  40 various interventions (BMJ Q & S 2012) and we suggested a few more in our two papers in BMJ Q&S the following year. Medicine has not been oblivious to the problems of cognitive failure – various solutions have been developed over the years: identifying predictable pitfalls and warning against them, developing mnemonics to deal with memory failures; developing a differential diagnosis to avoid anchoring on a single option, constructing lists of tips – usually a series of strategies to avoid various cognitive failures, describing red flags that raise caution in certain situations, developing bundles and forcing functions to avoid missing critical aspects of care, constructing checklists, as well as a variety of caveats and maxims that aim to promote better care, and others strategies. These have evolved because there has been a perceived need and because they work – a common element is that they often involve forcing functions. Arguments are made that there is no evidence that they work, but many of them were developed in the pre-evidence based era, and absence of proof is not proof of their absence. Recently, a large study looked at outcomes following the introduction of a mnemonic-based program for handovers (I-PASS) and found a significant reduction in diagnostic error rate (Starmer et al, NEJM 2014). We might find that some of the other time-honored strategies may also be proven effective. In the meantime, other experimental groups (e.g. Mamede’s group) have shown that specific biases can be mitigated.

Overall, cognitive debiasing strategies are difficult to implement – it is unlikely that one debiasing strategy will work for all, unlikely that one shot will be effective, and very likely that maintenance of debiasing will be a necessary part of clinical practice. Biases by their nature are robust and difficult to change – they wouldn’t be biases if they weren’t. To imagine that one intervention will fix a deeply ingrained bias is naïve.       



Pat Croskerry MD, PhD, FRCP(Edin)

Professor,Department of Emergency Medicine, 

Director, Critical Thinking Program, Division of Medical Education,

Faculty of Medicine,

Dalhousie University,

Halifax, Nova Scotia, CANADA


From: Hamm, Robert M. (HSC) [mailto:Robert-Hamm at OUHSC.EDU] 
Sent: Friday, December 12, 2014 12:24 PM
Subject: Re: [IMPROVEDX] Published results of cognitive bias training or similar efforts


I think that the literature was widely reviewed, in some of the essays in the first issue of Diagnosis.

One of the papers was my own, where are advocated for taking at least as much time teaching the logic of diagnosis explicitly and making sure that clinicians get it, as in teaching them the mistakes and cognitive shortcuts that people take. You can’t understand a fumble without understanding a catch. But there are several other perspectives in the same issue. 


Rob Hamm


From: Brent Smith [mailto:smithb at SMITHNET.US] 
Sent: Friday, December 12, 2014 6:16 AM
Subject: [IMPROVEDX] Published results of cognitive bias training or similar efforts


Hello - 


I sent this out a few days ago but I think I messed up by thread jacking instead of starting my own message.  Here it is again.



Are any members of this list aware of published results on the effectiveness of cognitive de-biasing training among medical trainees?  I'm aware of James Reily's curriculum paper and an effort by Sherbino pasted below.  I've searched for similar papers but haven't found any.  I'm looking specifically for results of educational interventions among medical traininees aimed at raising awareness of and reducing the effect of cognitive biases as they relate to misdiagnosis.


thanks =)


brent smith



Reilly JB, Ogdie AR, Von Feldt JM. Teaching about how doctors think: a longitudinal curriculum in cognitive bias and diagnostic error for residents. BMJ Qual Saf. Aug 2013;22(12):1044-1050.


Sherbino J, Kulasegaram K, Howey E. Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: a controlled trial. CJEM. 2014;16(1):34-40.



To unsubscribe from IMPROVEDX: click the following link: <> &A=1 


Visit the searchable archives or adjust your subscription at:

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:



To unsubscribe from IMPROVEDX: click the following link: <> &A=1 


Visit the searchable archives or adjust your subscription at:

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:

Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine

HTML Version:
URL: <../attachments/20141215/5b4faeb3/attachment.html>

More information about the Test mailing list