Fwd: [IMPROVEDX] IOM report is released - Diagnosis in actual practice

Art Papier apapier at VISUALDX.COM
Thu Oct 8 16:18:24 UTC 2015

Newtonian physics govern how a plane behaves in the air.  Medicine is
practiced with unknown, gray areas and uncertainty.  With that said the
major difference is that pilots (with the exception of the Wright brothers
and for those piloting for a few decades after) are not trained to memorize
all the parameters of flight and the map.  Pilots now use instruments.  They
are trained in the instruments.  Doctors are trained and rewarded through
board examinations in memorization.  Pilots are trained to RELY on
instruments to guide their flight decisions.  If pilots where trained as
physicians are, they would be memorizing the most common and dangerous
routes and would not know how to use instruments in the cockpit.  As long as
we tolerate physicians only using their unaided brains we will not
significantly change the results.  Thankfully the younger generation in
medicine know that the "flying by the seat of the pants" and memorization to
solve all problems is an impossible task.  Many students, residents and
practicing clinicians do use information technology as they work.  Many of
course do not.  Is there an ideal way to fly a plane? Yes. Is using
information in medicine to guide decisions better than "winging it"?  Yes.
Many medical educators still teach to memorization as the core skill.  We
need to teach core history, physical exam skills and using tools to aide
cognition.  Larry Weed has written extensively on this.  For those that have
not read Medicine in Denial or his paper published in Diagnosis
xml?format=INT it is worth a read.  Also to understand that this is not a
new discussion watch Dr. Weeds 1971 Grand Rounds at Emory.




Art Papier, MD

Chief Executive Officer

585.272.2630 | apapier@ <mailto:apapier at logicalimages.com> visualdx.com


From: Bob Latino [mailto:blatino at RELIABILITY.COM] 
Sent: Thursday, October 08, 2015 10:49 AM
Subject: Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released - Diagnosis
in actual practice


I am not a physician nor a clinician, so I come at this issue basically from
the perspective of a patient.


When the physician becomes the patient, what is the expectation of them
towards their care provider in terms of their diagnosis?  


Physicians themselves would obviously be more critical of their peer's
diagnosis when their lives are involved, because they are 'insiders' and
know the probing questions to ask about how the diagnosis was derived.  What
are those questions?  What should the non-clinical patient be asking of
their doctors when they provide a diagnosis?


I am in the investigation business and work in aviation, nuclear power,
military and other potentially life-threatening businesses.  Many in these
businesses have to make spur of the moment decisions (diagnosing the
problem) and then quickly act on it.  


I will take pilots for instance.  I know healthcare has taken an interest in
Crew Resource Management (CRM) from the training that pilots receive about
effective cockpit communications and teamwork.  They too have to quickly
make a diagnosis and act on it accordingly.


The difference between a pilot and a doctor in these situations is that the
pilot and crew's lives are at stake (along with the passengers) as well,
based on the accuracy of their diagnosis, decisions and actions.  


Given this informative debate about probabilities and looking at
situations/patients singularly versus as a population, how does a pilot make
their quick assessment versus a doctor and their diagnosis?  Does the fact
the pilot's life is at stake differ in their decision as opposed to a
doctor, whose life is not likely at stake based on their decision?  Does it
matter? Should it?



Robert J. Latino, CEO

Reliability Center, Inc.


blatino at reliability.com



From: Jason Maude [mailto:Jason.Maude at ISABELHEALTHCARE.COM] 
Sent: Thursday, October 08, 2015 10:09 AM
Subject: Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released - Diagnosis
in actual practice


Although this is a very stimulating debate, I am struggling to understand
how relevant it actually is to the diagnosis of individual patients as a key
additional variable will always be the personal consequences of a wrong
decision.  The key difference with a probabilistic approach in life
assurance or similar versus diagnosis of a particular patient has to be the
consequences of getting it wrong. This means that nobody is likely to follow
a purely probabilistic approach if they know the patient might die if they
didn't check for something even if it was a lower probability. The odds of
winning the lottery are ludicrously bad but because the prize is so big
(upside consequences) people still try their luck. Personal consequences
will always seriously affect rational calculations of probability.



Jason Maude

Founder and CEO Isabel Healthcare


From: "Jain, Bimal P.,M.D." <BJAIN at PARTNERS.ORG>
Reply-To: Society to Improve Diagnosis in Medicine
Date: Thursday, 8 October 2015 12:13
Subject: Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released - Diagnosis
in actual practice


We can only comment on and critically evaluate material that is published. I
find it simply amazing that a probabilistic approach in which probability is
evidence has not been employed in even a single amidst hundreds of published
CPCs and clinical problem solving exercises. Dr. Brush dismisses CPCs as
artificial, pedagogical exercises employing System 2 thinking over days or
weeks. This is all the more reason to employ a probabilistic approach as the
discussants then have plenty of time to estimate prior probabilities and
calculate posterior probabilities. This is not done simply because this
approach has not been found useful for diagnosis. Some time back, I
carefully examined 50 consecutive CPCs in NEJM from July 2013 to OCTOBER
2014. I found the word probability mentioned only once in these 50 CPCs. If
Dr. Brush thinks this approach is suitable only for System 1 thinking in
diagnosis ,Croskerry has pointed out the danger of such thinking in causing
diagnostic errors. At present, the emperor does not appear to have any
clothes with regard to probabilistic approach to diagnosis in these
exercises. What is  needed ,I think are head to head observational or
experimental studies comparing usual to probabilistic approach in real


The adage 'Common things are common' is useful only in indicating chance of
a disease in a given patient. Certainly, we should look for a common disease
first as it has the greatest chance of being found. The problem arises when
a frequency or probability is taken as evidence for a disease. There is
little doubt in my mind, diagnostic errors due to failure to suspect a
disease in patients with atypical presentation in studies of Hardeep Singh
and John Ely arose from interpreting low prior probability as absence of
evidence for the disease.


In discussion about STEMI, Dr. Brush rightly deals with all patients with
STEMI regardless of  prior probability in the same manner by taking them all
for cardiac cath. His accuracy rate of acute MI of 85 percent in these
patients is close to the rate of 90 percent in my paper. If he were to
analyze his data he would find the majority of patients with acute MI to
have intermediate or high prior probability.


I refer the Central Limit Theorem with regard to distribution of prior
probability which is a continuous variable.


The main problem with a probabilistic approach is that it takes probability
as evidence in a given individual patient while it is true only in groups of
patients. There is no proof that it improves diagnosis in actual practice.
Its use appears to have become a dogma which is hindering efforts to reduce
diagnostic errors. It is only by looking at diagnosis in actual practice
such as in studies of H. Singh and J. Ely and analyzing results without
putting on probabilistic  glasses that we shall make progress.


I mention three examples from history of science of dogmatic beliefs
hindering progress which was made only when phenomena as they occur were

1.      Since the time of Plato, the belief in planetary orbits being
circular due to perfection of a circle as a geometrical figure. All contrary
observations were explained away by drawing circles(epicycles) within
circles. It was only two thousand years later that Kepler determined the
orbit of Mars to be an ellipse when he actually observed and analyzed its

2.      Since the time of Aristotle, every movement was believed to require
a mover. Contrary observations such as flight of an arrow were explained
away in an absurd manner. Again, about two thousand years later, the true
law of motion, that it is change in motion and not motion itself that
requires a force was discovered when Galileo observed and analyzed actual
motion of rolling balls.

3.      And nearer to our age, there was a widespread belief in Absolute
Time since Newton declared it to exist in the 17th century. It was only in
early 20 the century this belief was overthrown by Einstein by his
insightful analysis of actual time in terms of clocks and trains.






Bimal P Jain MD

Pulmonary-Critical Care

North Shore Medical Center

Lynn MA 01904



















From: John Brush [mailto:jebrush at ME.COM] 
Sent: Saturday, October 03, 2015 8:58 AM
Subject: Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released - Diagnosis
in actual practice


I'm afraid that I can't agree with Dr. Jain's argument. I think his argument
is circular, difficulty to follow, and selectively self-serving.

            We have an adage in medicine: "Common things are common."
Otherwise, every diagnostic exercise would become a wild goose chase,
leading us to look into every remote possibility every time. Having said
that, I can also say that if we collect cases over time, uncommon things
become common. Someone somewhere will eventually win the lottery. Uncommon
diagnoses do occur eventually. But the exceptions should not define the

            The STEMI case that Dr. Jain presents proves my point. I am in
interventional cardiologist who frequently takes patients with suspected
STEMI to the cath lab for intervention. I have been getting direct feedback
on these cases for about 25 years. I can tell you that there is a false
positive rate of about 15% among STEMI alerts that are taken to the cath lab
(numerous reports in the literature confirm that estimate). We allow that
false positive rate because we make a subjective calculation of expected
value. Even if a patient has a relatively low initial prior probability of
STEMI, like Dr. Jain's example, we don't want to miss a serious diagnosis
like a STEMI. The EKG findings change the probability estimate and make a
STEMI quite plausible in such a patient. In a patient like Dr. Jain's
example, we know that there is about a 50-50 chance of finding an occluded
artery, which is certainly high enough to activate the cath lab. And sure
enough, over time, 50% is about the frequency that we find in such patients.

            Dr. Jain references central limit theorem. That theorem applies
to probability for a continuous variable, and states that for any
distribution, the sample means of repeated samples will become a normal
distribution. I'm not sure I follow his argument that it applies to a
probability distribution of categorial variables. A diagnostic category is a
countable variable. Kolmogorov's principles, however, do apply. The
probabilities of all of the possibilities do add up to one, if they are all
independent. General knowledge of these probability principles can help us
organize our thinking.

            When we see a patient with chest pain in the ED, we start to
narrow the sample space by asking questions and making observations. For
example, we can eliminate the possibility of a stab wound very quickly by
noticing that there is no knife in the chest. Through early hypothesis
generation, we narrow the range of possibilities to the point were we can
start the process of iterative hypothesis testing. We have at our disposal
many possible tests that we can perform. We can send a troponin, do a CT
scan for dissection, do a stress echo, go directly to the cath lab, etc. We
can't do all of these tests at the same time, and we probably don't want to
do every test on every patient. So how do we decide what test to do first?
We do a little mental calculation of the subjective probabilities, which
gives us an idea of the expected value of each test. We don't want to miss a
diagnosis with serious consequences, like MI or dissection, so an EKG and
CXR are done on virtually everyone, regardless of the prior probability. But
we narrow the sample space as we hone in on the correct diagnosis. We don't
want to narrow the search prematurely, and we use a differential diagnosis
to help us guard against jumping to conclusions. All of this is guided by
some notion of relative probabilities. 

            Dr. Jain talks about the CPC method of diagnosis. This is a
useful pedagogical exercise, where an expert can expound on clinical
medicine, but it is very artificial, as compared to real world practice. An
expert may spend days or weeks preparing a CPC discussion. His/her main
goals are to not miss the diagnosis, and to eloquently discuss all of the
possibilities. It is almost purely System 2 thinking. In the real world,
with time constraints and uncertainty, we have to employ System 1's
intuition. It is helpful, however, if we calibrate our intuition through
knowledge of the relative strength of evidence and the base rates of various
diagnostic possibilities. I think that having an intuitive sense of
probability is the essence of experiential knowledge. Savvy clinicians make
good bets. 

            The fundamental assumption of evidence based medicine is that
the frequencies that we measure in populations of patients can be applied to
an individual patient. The measured frequencies from our aggregated
experience, or from the reports in the literature inform us on how we should
think about an individual. Single event or single patient probability then
becomes a degree of belief, which is then modified by additional information
that we gain through diagnostic testing. In fact, the sensitivity and
specificity of diagnostic tests are defined using a frequency notion of
probability. They are cumulative probabilities, depending on where we draw
the line of demarcation. Some tests, like an x-ray for a broken arm, are so
compelling that they lead to absolute certainty. Other tests, like EKGs,
stress tests, troponins, etc, don't have perfect operating characteristics,
however, and we are left with a probability estimate for each diagnostic
possibility that is somewhere between 0 and 1. Usually we get to a point of
certainty, but sometimes, through adductive reasoning, we are left with the
most plausible diagnosis, but never really know for sure.

            I hate to drag the listserv through this back and forth again,
but to me, Dr. Jain's arguments seem to counter what we have been taught
about evidence-based medicine, but also run counter to principles of
cognitive psychology. Without some intuitive idea of probability and
likelihood, we would be totally adrift in clinical medicine, so I just can't
let this go.



John E. Brush, Jr., M.D., FACC

Professor of Medicine

Eastern Virginia Medical School

Sentara Cardiology Specialists

844 Kempsville Road, Suite 204

Norfolk, VA 23502


Cell: 757-477-1990

jebrush at me.com




On Oct 2, 2015, at 2:45 PM, Mark Graber <mark.graber at IMPROVEDIAGNOSIS.ORG>


Note and manuscript forwarded on behalf of Dr Bimal Jain.



To unsubscribe from IMPROVEDX: click the following link:
1> &A=1




Moderator:David Meyers, Board Member, Society for Improving Diagnosis in

To learn more about SIDM visit:


<Bimal Jain - The Role of Probability in Diagnosis.docx>



From: Jain, Bimal P.,M.D. 
Sent: Thursday, October 01, 2015 1:54 PM
To: 'Mark Graber'
Subject: RE: [IMPROVEDX] IOM report is released - Diagnosis in actual


Hi Mark and all,


It is important to understand how diagnosis is performed in actual practice
as a correct diagnosis is made after all  85 percent of the time in
practice. To reduce diagnostic errors, we need to know if the method in
practice needs to be improved or whether certain deviations from it need to
be eliminated. The most puzzling issue in this regard is the role that
probability plays or does not play in diagnosis. The puzzle arises because a
probabilistic approach has been prescribed for a long time, but it does not
appear to be employed in practice when we look at published CPCs and
clinical problem solving exercises. Does this disparity imply that a
probabilistic approach is not suitable for diagnosis in actual practice?
This is certainly possible as diagnosis is performed in a given, individual
patient with the aim of determining a disease correctly in that particular
patient. And probability, as is well known has been employed most
successfully in practice in areas such as epidemiology and life insurance
business where the focus is on accuracy of prediction in a large group of
persons, not on prediction in a given individual person.

If we look closely, we note that a strict probabilistic approach in which a
probability represents evidence may actually increase diagnostic errors
specially in patients with atypical presentations by encouraging the
cognitive bias of representativeness and inhibiting comprehensive
differential diagnosis (discussed in attached paper).

I have put together my thoughts on this subject in the attached paper 'The
role of probability in diagnosis'. Please review and comment on it. Thanks.





Bimal P Jain MD

Pumonary-Critical Care

North shore Medical CENTER

Lynn MA 01904






To unsubscribe from IMPROVEDX: click the following link:
1> &A=1 


Moderator:David Meyers, Board Member, Society for Improving Diagnosis in

To learn more about SIDM visit:



To unsubscribe from IMPROVEDX: click the following link:
1> &A=1 


Moderator:David Meyers, Board Member, Society for Improving Diagnosis in

To learn more about SIDM visit:

Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br>
<a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1" target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1</a>

HTML Version:
URL: <../attachments/20151008/0e297fed/attachment.html>

More information about the Test mailing list