Accuracy, Overconfidence and Lack of Time

Jason Maude Jason.Maude at ISABELHEALTHCARE.COM
Mon Sep 2 13:15:12 UTC 2013

I recall from the original Friedman study that these were difficult cases.
However, the issue is that even though the cases were difficult the
physicians were fairly confident that they were right. If they thought
they were really difficult then why didn't their confidence levels fall to
say 10-20% rather than remain at the 60-70% levels?

The Illusion of knowledge is not my idea but was quoted in a talk given by
Barry Marshall who won the Noble prize for the discovery of bacteria as
the cause for ulcers. He was often asked why science was often very slow
to accept new ideas and he quoted Daniel Boorstein, Librarian for the US
Congress who said ³The greatest obstacle to knowledge is not ignorance, it
is the illusion of knowledge.² It 's worth watching his talk and this bit
is at 6 mins

I think time is a really key issue and is an important research topic as
we need to establish whether it is a legitimate issue or just an excuse.
All the time it's accepted as an excuse it means that the real causes are
not tackled.

Jason Maude
Founder and CEO Isabel Healthcare
Tel: +44 1428 644886
Tel: +1 703 879 1890 <>

On 30/08/2013 16:16, "Ashley N. D. Meyer, PhD" <ashley.meyer2 at VA.GOV>

>First off thank you for the kind words about our study. We are glad that
>it has stimulated conversation.
>I wanted to address a few of your comments:
>As far as the accuracy rate goes, we didn¹t want all of the physicians to
>perform perfectly (as that would create ³ceiling effects² and would make
>it difficult to see differences between physicians). As such, we
>purposely chose more difficult cases. Even the ³easier² were difficult
>and the ³more difficult² were very difficult. This may have inadvertently
>created ³floor effects² for the more difficult cases, but the relative
>difference in accuracy between the easier and more difficult cases gave
>us the ability to compare relatively different levels of performance
>within each physician and how they change their confidence, which was our
>main aim. In everyday practice, we do not expect physicians to perform
>this poorly. So, hopefully, that should assuage some concerns.
>In regards to your second comment on how accuracy is not improving over
>time- that is a more difficult question to address. Previous research has
>shown that physicians often make their minds up relatively early in the
>diagnostic process and tend to seek confirming evidence (which would not
>likely change their accuracy). Perhaps we are seeing some of that here.
>In making conclusions about the utility of expensive testing, I think I
>would shy away from that, as this study only looked at 4 separate cases.
>There are likely other diagnoses that would benefit from such testing. It
>might be beneficial for us to dissect the cases a little more to evaluate
>where in the four phases the most informative diagnostic information for
>each case was delivered to better get at this comment. It is definitely a
>good point though.
>³Illusion of knowledge² in place of the term overconfidence is an
>interesting idea. That¹s a great thought. Additional studies could look
>into why physicians rate their confidence as they do. Perhaps some of
>that is an illusion of knowledge. I would assert that there are also
>other factors playing into confidence judgments, including but not
>limited to lack of negative feedback over one¹s career, self-efficacy in
>medicine in general, risk aversion (as many people have mentioned over
>the past few days), experience with similar sounding cases, and various
>other things.
>Jason, you also brought up the issue of time in terms of it not being a
>barrier to ask for additional resources in our study. I think you are
>right- this was not a barrier. However, I do think that physicians are
>likely used to thinking in terms of time and efficiency, so it is
>possible that they are primed to think this way even in situations where
>time is not a factor (i.e., that barrier might exist implicitly or
>unconsciously). I do agree with your point though, that people often say
>they don¹t have time to do something, when really it is just not a
>priority for them. Hopefully this doesn¹t happen when stakes are high,
>but again this is an empirical question.
>Lastly, the use of symptom checkers could be an interesting idea. I think
>in using tools like this, we just need to make sure that the tools
>prevent patients from falling into similar traps or using similar biases
>that physicians do (and frankly those that all decision-makers are
>susceptible to), including only looking for confirming evidence and
>ignoring symptoms that don¹t fit (and then not reporting those to their

Name: default.xml Type: application/xml Size: 3222 bytes Desc: default.xml URL: <../attachments/20130902/3db0d8ae/attachment.xml>

More information about the Test mailing list