FM: Crowd Wisdom .... lack of dx error in the curriculum

Alan Morris Alan.Morris at IMAIL.ORG
Sat Aug 17 18:58:41 UTC 2013

Yes, simulation is reasonable and good - and it take effort to do the
adequately explicit level of knowledge engineering.  In the long run,
however, we would be way ahead because of the consistency and
reproducibility of the eProtocol outputs, once the front-end effort had
been made (this is feasible but scalability has not yet been evaluated).
The current situation is unsustainable and will not, in my view, ever get
us close to desired goals of clinician decisions consistently linked to
credible evidence.  This follows from human decision-making limitations
that do not appear to be able to be overcome by education.  In the long
run, I think evidence indicates the medical community would expend much
less effort with the adequately explicit knowledge engineering approach.


On 8/17/13 9:07 AM, "Robert Bell" <rmsbell at> wrote:

>Would like to think that simulation exercises and the equivalent of
>10,000 tough crossword-like challenges during training to mimic clinical
>situations would start to get us moving in the right direction.
>But think of the effort to get to programs that have that level of
>complexity for both diagnosis and treatment when all is changing so
>often. But maybe not too complex for singularity computers?!
>Rob Bell
>Sent from my iPad
>On Aug 16, 2013, at 5:05 PM, Alan Morris <Alan.Morris at IMAIL.ORG> wrote:
>> There is no question in my mind of the value of educational programs
>> address diagnostic algorithm processes, sensitivity training, etc.
>> However, major differences exist between people regarding sensitivity to
>> others, and regarding intuition.  Trainees with a background in physical
>> science, math, or engineering are better prepared than others to deal
>> a number of approaches to diagnosis and treatment.  Those trained in
>> psychology are frequently better able to address interpersonal issues.
>> Nevertheless, humans are not good at reproducible behavior and error in
>> medicine abounds.  Even when error rates are only 1% - an almost unheard
>> of performance -  the 1% errors led to a major threat to life or limb
>> ICU patients   Ion general, adherence to protocols and guidelines is in
>> most reports poor and infrequently greater than 50%.  We must, in my
>> pay more attention and generate systematic approaches to development,
>> validation, and implementation of decision-support tools that can lead
>> reproducible clinician behavior.  This requires a shift in thinking
>> because this is only achieve with adequately explicit protocols - those
>> with enough detail to be contextually sensitive and that deliver
>> patient-specific (personalized) instructions for diagnosis or treatment.
>> I do not detect much interest in these concepts in this discussion.
>> Have  a nice day.
>> Alan H. Morris, M.D.
>> Professor of Medicine
>> Adjunct Prof. of Medical Informatics
>> University of Utah
>> Director of Research
>> Director Urban Central Region Blood Gas and Pulmonary Laboratories
>> Pulmonary/Critical Care Division
>> Sorenson Heart & Lung Center - 6th Floor
>> Intermountain Medical Center
>> 5121 South Cottonwood Street
>> Murray, Utah  84157-7000, USA
>> Office Phone: 801-507-4603
>> Mobile Phone: 801-718-1283
>> Fax: 801-507-4699
>> e-mail: alan.morris at
>> e-mail: alanhmorris at
>> On 8/16/13 2:22 PM, "SMITH, BRENT W Maj USAF AMC 60 MDOS/SGOF"
>> <brent.smith.1 at US.AF.MIL> wrote:
>>> We are currently running some educational research on a curriculum
>>> designed to teach metacognition and debiasing to family medicine
>>> residents.  It is tricky to teach, trickier to measure but I'm hoping
>>> have some data to help answer the question of whether it can be taught.
>>> The full curriculum will be available for people to use and adapt on
>>> SIDM website in Sept. I'll send a link out when the link goes live.
>>> My intuitive response if something can be learned though experience and
>>> effort, the acquisition of that same skill/knowledge can be accelerated
>>> by coaches and teachers. The 'how to' is the tough part.  Though a
>>> teacher could accelerate the learning process, I think actual clinical
>>> experience is irreplaceable.
>>> Brent Smith MD
>>> Maj, USAF, MC
>>> Family Medicine Faculty
>>> Travis AFB, CA
>>> "FOR OFFICIAL USE ONLY. This electronic transmission may contain
>>> medical information protected by the Privacy Act of 1974 (see AFI
>>> and the Health Insurance Portability and Accountability Act (HIPPA)
>>> DoD
>>> 6025.18-R) not intended for disclosure outside government channels and
>>> exempt from mandatory disclosure under the Freedom of Information Act,
>>> U.S.C., 552. Exemption 6 may apply. Do not release outside of DoD
>>> channels without the consent of the originator's office. If you
>>> this message in error, please notify the sender by reply e-mail and
>>> delete all copies of message."
>>> -----Original Message-----
>>> From: John Brush [mailto:jebrush at MAC.COM]
>>> Sent: Friday, August 16, 2013 11:01 AM
>>> Subject: Re: [IMPROVEDX] FM: Crowd Wisdom .... lack of dx error in the
>>> curriculum
>>> There are others on this listserv, who are much more qualified than me
>>> answer the question: "Can intuition be taught?"
>>> My two cents: I suspect that intuition is a form of intelligence that
>>> innate, but can be shaped, honed, improved, and recalibrated. Intuition
>>> is a talent that can be developed through deliberate practice.
>>> I also think that metacognition can be taught. The goal of education in
>>> any domain should be to encourage students to be more thoughtful - to
>>> actively and critically think about what they are doing. Students need
>>> vocabulary and some background to get them started, though. I think our
>>> goal should be to make this educational process more explicit. What are
>>> the core competencies of good medical reasoning, and how can we
>>> effectively, reliably, and efficiently teach those competencies?
>>> John 
>>> On Aug 16, 2013, at 9:08 AM, Graber, Mark wrote:
>>> Garry's comments raise a very important and fundamental question in our
>>> field that someone out there may know the answer to. If not, add it to
>>> the growing list of research priorities:
>>> In the traditional paradigm, the expert evolves from years of training
>>> and experience, the 'right' education as Garry phrases it.  The expert
>>> becomes so because they've made all the errors there are to make, or
>>> seen them.  The opposing view is that we can shortcut this process if
>>> teach principles of metacognition, present all the cognitive biases and
>>> their antidotes, and teach error prevention strategies.  Robin Hogarth
>>> has a book "Educating Intuition" and Mark Quirk makes many of the same
>>> points in his "Intuition and Metacognition in Medical Education."
>>> The question boils down to whether you believe that you really CAN
>>> educate intuition, or do you have to acquire it the 'old fashioned'
>>> through experience.
>>> ________________________________
>>> From: Garry Nieuwkamp <Garry.Nieuwkamp at HEALTH.NSW.GOV.AU>
>>> Reply-To: Society to Improve Diagnosis in Medicine
>>> <Garry.Nieuwkamp at HEALTH.NSW.GOV.AU>
>>> Date: Thu, 15 Aug 2013 22:13:51 -0400
>>> Subject: Re: [IMPROVEDX] FM: Crowd Wisdom .... lack of dx error in the
>>> curriculum
>>> I have no problems with the lack of error education in medical schools.
>>> Everyone wants a part of medical education because this is important or
>>> that is important. How about just getting the medical education right
>>> let the other stuff work itself out in due course.
>>> Garry Nieuwkamp MBBS DA FACEM MA PhD
>>> ________________________________________
>>> From: Lorri Zipperer [Lorri at ZPM1.COM]
>>> Sent: Friday, 26 July 2013 11:16 AM
>>> To: Garry Nieuwkamp; Society to Improve Diagnosis in Medicine
>>> Subject: [IMPROVEDX] FM: Crowd Wisdom .... lack of dx error in the
>>> curriculum
>>> From: robert bell [mailto:rmsbell at]
>>> Sent: Thursday, July 25, 2013 5:32 PM
>>> To: Society to Improve Diagnosis in Medicine; Dr.Will
>>> Subject: Re: [IMPROVEDX] Crowd Wisdom, Premature psych conclusions, and
>>> harm from misdiagnosis
>>> [Building on the comments of] Will and Amy,
>>> I think the main problem is that medical diagnosis is not well taught
>>> medical/PA/NP schools. There are few simulator programs like the
>>> industry to repeat things time and time again, little instruction on
>>> basics of diagnosis and where to get the best help/support, little
>>> instruction on how to get a good history and elicit meaningful symptom
>>> complexes and changes, etc. And little knowledge is passed on as to
>>> the main errors in diagnosis come from in both the hospital and private
>>> practice offices.
>>> Did I hear that only 50% of teaching schools even address Errors in
>>> Medicine as a whole.
>>> Pretty awful - we are still teaching med students using 100 year old
>>> methods.
>>> Add to this mess to the lack of research in the area and you have one
>>> big, big problem.
>>> This organization could well lobby for more research and medical
>>> training/education in this area.
>>> Rob Bell
>>> On Jul 25, 2013, at 6:51 AM, Dr.Will wrote:
>>> Yes physicians are spending "less time" listening and caring for
>>> because of the additional typing responsibilities due to EMR and LESS
>>> contact (and if someone does the research), you will see even LESS
>>> effective communication coming out of patient due to the "hurried"
>>> visit. Hence more "misdiagnosis".
>>> Second there is A LOT of chronic disease that CAUSES depression like
>>> symptoms.
>>> So most physicians should "slow down you're moving too fast, you've got
>>> to let the office visit last(get to the real issue)" to quote a phrase
>>> from a popular song.
>>> Will Sawyer
>>> Solo Family Medicine
>>> ________________________________
>>> From: Amy Reinert [mailto:amy.reinert at GMAIL.COM]
>>> Sent: Wednesday, July 24, 2013 6:58 PM
>>> Subject: Crowd Wisdom, Premature psych conclusions, and harm from
>>> misdiagnosis
>>> I'm coming in a little bit late to this discussion, but did want to
>>> respond to Dr. Bell's comment about women with history of sexual abuse
>>> seemingly living in the doctor's office.
>>> For my doctoral dissertation research, I conducted a study of women
>>> diagnosed with autoimmune disease whose symptoms had been dismissed by
>>> several physicians over the course of several years. All of these
>>> patients were dismissed as "psych" by male and female physicians alike
>>> (please note that they were screened for pre-existing mental
>>> Some physicians ran tests initially. Others declined to run tests at
>>> Obviously these women truly were ill, however, they experienced
>>> significant psychological harm that can reasonably be attributed to
>>> misdiagnosis or lack of investigation. Of course, there were also
>>> financial implications from all of these unproductive office visits.
>>> much to go into in depth here, but in considering the harm these
>>> experienced in terms of their mental health (being told they was
>>> wrong with them when in fact their bodies were clearly--to them-- ill)
>>> and the worsening of their untreated disease, it seems that there does
>>> need to be further study of patients with unresolved symptoms. The
>>> results of my study also indicated that more study of the influencing
>>> "invisible" personal biases held by physicians is also in order.
>>> A.D. Ruzicka, Ph.D

More information about the Test mailing list