[IMPROVEDX] IOM report is released - Diagnosis in actual practice

Julianne Nemes Walsh nemeswalsh at GMAIL.COM
Mon Oct 19 02:18:48 UTC 2015


Hello,
I have done multiple searches for a pediatric outpatient trigger tool to
measure diagnostic errors in pediatric primary care.  Has anyone in the
consortium utilized a pediatric outpatient tool?
Thank you in advance,

Julianne Nemes Walsh

On Fri, Oct 16, 2015 at 3:27 PM, Jain, Bimal P.,M.D. <BJAIN at partners.org>
wrote:

> It is difficult in practice to quantify prior probability. It is customary
> to estimate it as being low, intermediate or high as was done in the PIOPED
> study(JAMA 1990) which too can be problematic. As pulmonary embolism is
> known to occur in patients across the whole range of priors ranging from
> the very low to the very high, it is difficult to say how low should the
> prior be before a chest CT angiogram is not done. A prior probability only
> denotes the chance of finding embolism in a given patient and is not prior
> evidence for it. Certainly if we decide not to look for embolism purely on
> basis of low prior, we run the risk of missing it with disastrous
> consequences as presented by Croskerry in a patient in NEJM in 2012 or
> 2013. I think a reasonable approach of most physicians including myself is
> to look for it if we think it may be present. We usually do a D-Dimer first
> if prior is low and if it is negative it effectively rules out embolism as
> it has a likelihood ratio of 0.1. If D-Dimer is positive, I would proceed
> with a chest CT angiogram.
>
> If CT angiogram is positive, I would diagnose embolism definitively in any
> patient. This is validated by our experience of the very high accuracy of
> CT angiogram across patients with all priors. This is legitimate, I
> believe, as likelihood ratio for CT angiogram of 21 is independent of prior.
>
> We do not diagnose embolism or any other disease in practice in a
> probabilistic manner. That is why I believe we need to study diagnosis in
> actual practice more carefully.
>
>
>
> Bimal
>
>
>
> Bimal P Jain MD
>
> Pulmonary-CriticalCare
>
> NorthShore Medical Center
>
> Lynn  MA01904
>
>
>
>
>
> *From:* Amos Cahan [mailto:acahan at US.IBM.COM]
> *Sent:* Thursday, October 15, 2015 4:03 PM
> *To:* IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
> *Subject:* Re: [IMPROVEDX] [IMPROVEDX] IOM report is released - Diagnosis
> in actual practice
>
>
>
> If you base the decision to treat PE on a positive angiogram, then for
> every 100 angiograms you perform, you will end up treating a false positive
> case. So- to justify doing this test in the first place, you need to have
> at least 5 patients with PE for every 100 patients examined (this will give
> you approximately 1 true positive for every false positive result). This
> means you can’t avoid the question of prior probability even if you want
> to, because patients do not present to the ER with their angiogram. A
> physician has to refer them to the test. And such referral may only be
> justified if the prior is at least 5% (and at least in my opinion, this
> number is too low). How do you decide when an angiogram is needed?
>
> Amos
>
> Amos Cahan, MD
> Research scientist, Clinical Informatics
> IBM T. J. Watson Research Center
> 1101 Kitchawan Road, Route 134
> Yorktown Heights, NY 10598
> 1.914.945.2590
> acahan at us.ibm.com
>
>
>
> [image: Inactive hide details for "Jain, Bimal P.,M.D." ---10/15/2015
> 02:36:22 PM---From: "Jain, Bimal P.,M.D." <BJAIN at PARTNERS.ORG> To]"Jain,
> Bimal P.,M.D." ---10/15/2015 02:36:22 PM---From: "Jain, Bimal P.,M.D." <
> BJAIN at PARTNERS.ORG> To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
>
> From: "Jain, Bimal P.,M.D." <BJAIN at PARTNERS.ORG>
> To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
> Date: 10/15/2015 02:36 PM
> Subject: Re: [IMPROVEDX] [IMPROVEDX] IOM report is released - Diagnosis
> in actual practice
> ------------------------------
>
>
>
>
> If we know that out of 100 patients with pulmonary embolism, 21 have
> positive chest CT angiogram, we can say the frequency of positive angiogram
> in this group or the probability of positive angiogram in a patient in this
> group is 21 percent. This is a statement of likelihood. If , in addition,
> we know that out of 100 patients without pulmonary embolism only 1 has
> positive angiogram , we have a likelihood ratio of 21. You are right,
> likelihood or likelihood ratio is not expressed in terms of odds.
>
> Let us now calculate the probability of pulmonary embolism in a patient
> with positive angiogram. For this, we need to know the prior probability of
> pulmonary embolism in this patient which is derived from the distribution
> of embolism in a group of patients like him. Suppose it is 5 percent as the
> presentation is highly atypical. We calculate the posterior probability of
> embolism to be 52 percent.
>
> In the probabilistic approach, we are asked to consider this posterior
> probability as evidence for embolism in this patient and diagnose it on its
> basis. From it , we should diagnose embolism to be indeterminate in this
> patient. In practice, we diagnose embolism in this patient with near
> certainty in this patient from the strong evidence furnished by likelihood
> ratio of 21 alone. In fact, we diagnose embolism with near certainty from
> positive angiogram in any patient regardless of prior probability as the
> likelihood ratio remains 21 in all patients.
>
> On the other hand, if we are epidemiologists and wish to study the
> distribution of pulmonary embolism in the defined group of patients similar
> to this patient, the posterior probability of 52 percent would be evidence
> which we would correctly use to determine this distribution.
>
> For us clinicians, diagnosis is local, so to speak, as it is performed in
> a given individual patient as opposed to epidemiology which is global being
> done in a large group or population. The point I have raised is that
> probability is evidence only at a global not a local level.
>
> The notion of likelihood is simple but subtle which escaped the notice of
> mathematicians of the rank of Gauss and Laplace. It took the genius of
> Ronald Fisher to isolate it. It is employed in all forms of statistical
> inference including the probabilistic approach. The only problem with this
> approach is it is combined with a distribution in the form of a prior
> probability making this approach unsuitable locally in a given individual
> patient in diagnosis.
>
> Bimal
>
>
>
>
>
>
>
> *From:* Ely, John [mailto:john-ely at uiowa.edu <john-ely at uiowa.edu>]
> * Sent:* Wednesday, October 14, 2015 10:59 AM
> * To:* Society to Improve Diagnosis in Medicine; Jain, Bimal P.,M.D.
> * Subject:* RE: [IMPROVEDX] [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> If I have a room with 100 people complaining of sore throat in it and 20
> of them have strep throat and I select one person randomly from that room,
> the probability that that person has strep throat is 20%. So in that sense,
> the probability in the population is the same as the probability in the
> individual.
>
> I think we use the word “likelihood” to mean the same thing as
> probability. For example, a likelihood ratio is the ratio of two
> probabilities (not two odds, not two frequencies, but two probabilities).
> Also, The American Heritage Dictionary of the English Language defines
> likelihood as “The state of being probable; probability.”
>
> John Ely, MD
> University of Iowa
>
> *From:* Jain, Bimal P.,M.D. [mailto:BJAIN at PARTNERS.ORG
> <BJAIN at PARTNERS.ORG>]
> * Sent:* Tuesday, October 13, 2015 3:14 PM
> * To:* IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
> * Subject:* Re: [IMPROVEDX] [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> In response to Dr. Brush, when a CPC discussant says ‘ the most likely
> diagnosis is ‘ he does not mean ‘ the most probable diagnosis is’ in the
> sense of a disease having the highest frequency in a series or group of
> similar patients. What he means is that a disease has the greatest
> likelihood in that the given presentation or a test result has been
> observed to have the highest frequency in this disease compared to other
> diseases. He makes a statement of likelihood, not one of probability. The
> two concepts are completely different as a disease is fixed in a likelihood
> but is a random variable in probability. Despite this difference they are
> often mistaken for each other. Even the great Ronald Fisher thought
> likelihood was a probability when he first introduced it in 1913. However,
> he soon realized his mistake and described its properties clearly as we
> them today in his landmark paper Mathematical Foundations of Statistics in
> 1922.
>
> If we look carefully at the process of diagnosis in CPCs as I have done in
> 50 CPCs, we note that it proceeds by employing likelihood as evidence. An
> extensive differential diagnosis is initially created and the likelihood of
> each disease is estimated from given data. The disease with the highest
> likelihood is then put forth as the most likely diagnosis. One would think
> that if ‘most likely’ is the same as ‘most probable’ as Dr. Brush suggest,
> then the phrase ‘ most probable’ would be used, if he is correct in at
> least a few among the hundreds of CPCs that have been conducted. But this
> phrase has never been used, to the best of my knowledge in any CPC.
>
> Furthermore, a prior probability is not estimated and used as prior
> evidence, a hallmark of the probabilistic approach in any7 CPC. This is not
> done, I suggest, because the prior probability only indicates the
> distribution of a disease in the population from which the given patient
> has been drawn. As this distribution has no bearing as evidence for or
> against a disease in the given patient, a prior probability is not
> estimated.
>
> There is universal agreement that probability is highly accurate in
> representing evidence in terms of frequencies or averages in populations or
> large groups of similar objects or persons.. Therefore it has been
> extensively employed in areas such as epidemiology, life insurance
> business, stock portfolio management, statistical mechanics. Incidentally,
> some excellent comments on application of probability are to be found in
> writings of some great figures in statistical mechanics which include
> Maxwell, Boltzmann, Gibbs, Ehrenfests, Einstein. It is in employing
> probability to represent evidence in a single, given instance that problems
> arise. Here is Charles Sanders Peirce weighing in on this issue. ‘Any
> individual inference must be either true or false.....in reference to a
> single case, considered in itself, probability can have no meaning.’
>
> There are at least four other methods of statistical inference besides the
> probabilistic approach in which evidence is represented by a measure other
> than a probability. These are:
>
> 1. Fisher’s likelihood method, briefly discussed above and championed in
> recent years by AWF Edwards, Richard Royall.
> 2. Fisher’s test of significance method.
> 3. Neyman’s error probabilities (Types 1 and 2) method championed recently
> by Deborah Mayo.
> 4. Neyman’s confidence interval method.
>
>
> One wonders why the founders of EBM (Sackett et al) chose probability as
> evidence and not some other measure for evidence in evidence based
> diagnosis. I believe, we have a clue to their thinking in this regard in
> the title of Sackett’s well known book Clinical Epidemiology. It is
> possible they were so impressed by the accuracy of probability in
> representing evidence as a stable frequency in a population that they chose
> to represent evidence by a probability in the entirely different domain of
> a given individual patient in which diagnosis is performed. It is amazing
> how rapidly and widely this was accepted without any proof of its validity.
> For example, I am still waiting for the first CPC or clinical problem
> solving exercise in which a disease is diagnosed correctly from a
> probability obtained by estimating a prior probability and calculating a
> posterior probability. Furthermore, a probabilistic approach, in my view
> promotes diagnostic error in some patients. Thus what is cognitive bias of
> representativeness in which only diseases with prototypical features (high
> prior probabilities) are suspected and diseases with atypical variants (low
> prior probabilities) ignored but a form of probabilistic reasoning in
> diagnosis.
>
> What needed to be done and was not done by the founders, in my view, was
> to develop evidence based diagnosis in a constructive manner. This means
> starting with a close study and analysis of diagnosis in actual practice. A
> close examination of even a few CPCs would have revealed to them that
> evidence is not assessed by a probability in actual practice. They would
> have noted that probability behaves very differently in a given patient
> compared to in a population. In the latter, a higher probability always
> corresponds to a greater frequency compared to a lower probability which
> always corresponds to a lower frequency. In a given, individual patient, on
> the other hand, a higher as well as a lower probability may correspond to
> presence of a disease. If this initial analysis were done, it is possible,
> evidence in diagnosis may not have been represented by a probability.
>
> In any case, there is such strong dogmatic belief in correctness of
> probability is evidence that even a suggestion to the contrary evokes the
> same incredulity and astonishment that were displayed by the Aristotelians
> when they learnt that uniform motion does not require a mover which is
> described by Galileo in his great book Discorsi or Discourses on two new
> sciences.
>
> It is well to remember that many longstanding incorrect beliefs such as
> planetary orbits being circular, every motion requiring a mover, time being
> absolute were overthrown and corrections made only by careful study and
> analysis of actual processes related to them. Similarly, I believe,
> progress in improving diagnosis will be made only by studying and analyzing
> the process of diagnosis in actual practice. My own view, based on careful
> study of CPCs is that evidence in diagnosis in actual practice is
> represented by likelihood and not by probability. This needs to be
> confirmed or disproved by further study and analysis of actual diagnosis.
>
> In the end, regarding this discussion, I can do no better than leave the
> last word for Einstein in the following long quote:
> Concepts that have proven useful in ordering things easily achieve such an
> authority over us that we forget their earthly origins and accept them as
> unalterable givens. Thus they come to be stamped as ‘necessities of
> thought’, ‘a priori givens’ etc. The path of scientific advance is often
> made impassable for a long time through such errors. For that reason, it is
> by no means an idle game if we become practiced in analyzing the long
> commonplace concepts and exhibiting those circumstances upon which their
> justification and usefulness depend, how they have grown up individually
> out of the givens of experience. By these means, their all too great
> authority will be broken. They will be removed if they cannot be
> legitimized, corrected if their correlation with given things be far too
> superfluous, replaced by others if a new system can be established that we
> prefer for whatever reason.
>
> I make no apologies for this very long email as the discussion of
> diagnosis, which is a complicated topic with many strands, requires many
> well thought out arguments that can only be expressed by detailed
> statements.
>
> I would like to thank Mark Graber for providing a forum in ListServ for
> this important discussion.
>
>
> Bimal
>
>
>
> Bimal P Jain MD
> Pulmonary-CriticalCare
> NorthShore Medical Center
> Lynn MA 01904
>
>
>
>
>
>
>
>
>
>
>
>
>
> *From:* John Brush [mailto:jebrush at ME.COM <jebrush at ME.COM>]
> * Sent:* Friday, October 09, 2015 1:16 PM
> * To:* IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
> * Subject:* Re: [IMPROVEDX] [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> Mark Ebell’s insightful email points to the absurdity of not thinking
> probabilistically. The probabilities that skilled clinicians use in an
> intuitive fashion in practice are derived from scientific evidence and
> experience (experiential knowledge). I am a big believer in evidence-based
> medicine. Without the scientific underpinnings for our clinical activities,
> and explicit acknowledgement of where the science is lacking, we would be
> adrift.
> In David Sackett’s original BMJ article on evidence-based medicine, he
> promoted the use of the best available scientific evidence "especially
> from patient centred clinical research into the accuracy and precision of
> diagnostic tests.” He included with his book a nomogram that a clinician
> could use to incorporate likelihood ratios for diagnostic tests with prior
> probabilities to yield a posterior probability for a single patient
> undergoing a single test. To him and other EBM founders, probability was
> the very foundation of statistical inference and evidence-based medicine.
> In my book, I stated "We can make highly accurate actuarial predictions
> for populations, but we have trouble even comprehending what probability
> means for a single patient or event.” I think this email trail points to
> the difficulty of thinking about probability in medicine. I am concerned
> that Dr. Jain’s paper only adds to the confusion. I would submit that in
> every one of the CPCs that Dr. Jain refers to, the discussant makes a
> statement like, “I think the most likely diagnosis is….” The statement
> "most likely" is another way of saying "most probable." Dr. Jain’s
> statement that probability is only a theoretical consideration and is not
> used in the practice of medicine is, I think, absurd. Not acknowledging the
> uncertainty in medicine through some statement of probability (which is
> simply a way to quantify uncertainty) leads to an illusion of certainty,
> arrogance on the part of the practitioner, and unrealistic patient and
> family expectations.
> In my book, it took me 5 chapters to fully develop the idea of probability
> and how it should be used to think about the diagnosis and treatment
> individual patients. This is tough to think about and comprehend and I
> think it can be misrepresented in an email listserv.
> For excellent reading on probability, I would suggest Ian Hacking’s “An
> Introduction to Probability and Inductive Logic” or Gerd Gigerenzer’s “Risk
> Savvy: How to Make Good Decisions” or “Calculated Risks: How to Know When
> Numbers Deceive You.”
> Ian Hacking and Gerd Gigerenzer participated in a year long sabbatical
> with other philosophers, scientists, and cognitive psychologists where they
> explored the meaning of probability. Hacking has also written “The Taming
> of Chance” and “The Emergence of Probability.” Girgerenzer has also written
> “The Empire of Chance: How Probability Changed Science and Everyday Life”
> for more in-depth reading.
> In clinical medicine, we use conditional probability every day, but
> because the exact numbers are only estimates, we actually use a heuristic
> called anchoring and adjusting. We use reason every day, but because the
> ultimate truth may be unknown, we use a type of reasoning called abductive
> reasoning. (My spell-checker incorrectly changed the spelling to adductive
> reasoning in my prior email.) Abductive reasoning, described by Charles
> Saunders Peirce, is "reasoning toward the most plausible hypothesis." When
> we start the diagnostic process, we are dealing with multiple hypotheses
> (plausible conjectures). We work through the process toward the most
> plausible hypothesis and, again, the term “most plausible" implies some
> concept of relative probability. With abductive reasoning, we blend both
> inductive reasoning and causal reasoning to make an argument (meaning a
> logical statement) that combines both probability and pathophysiologic
> rationale.
> As W. Edwards Deming said, "if you don’t understand the process of what
> you are doing, you don’t know what your are doing.” It is important for
> clinicians to have a better understanding of the process of making a
> diagnosis. By developing and using good habits based on a deep
> understanding of process, the clinician will have the best chance of making
> the correct diagnosis, as reliably as humanly possible.
> My apologies about the long email, but I am very serious and passionate
> about improving the quality of medical decisions.
> John
>
> John E. Brush, Jr., M.D., FACC
> Professor of Medicine
> Eastern Virginia Medical School
> Sentara Cardiology Specialists
> 844 Kempsville Road, Suite 204
> Norfolk, VA 23502
> 757-261-0700
> Cell: 757-477-1990
> jebrush at me.com
>
>
>
> On Oct 8, 2015, at 2:54 PM, Mark H Ebell <ebell at UGA.EDU> wrote:
>
> So, I should order a chest CT for every patient with cough, to rule out
> lung cancer.
>
> And I should order a stress thallium for every single 25 year old with
> chest pain that appears to be musculoskeletal, so I don’t miss the rare MI.
>
> And of course I should get a CT or MRI for every patient with a headache,
> to not miss the rare (and generally untreatable) CNS cancer.
>
> Do you realize the cost and harm of this approach? The complications of
> invasive tests and biopsies and follow-up that go nowhere? The false
> alarms? Radiation?
>
> But at least you won’t successfully sue me. I guess that’s all that
> matters.
>
> Mark
>
>> Mark H. Ebell MD, MS
> Professor of Epidemiology
> University of Georgia
> Editor, *Essential Evidence*
> Deputy Editor, *American Family Physician*
> ebell at uga.edu
>
>
> *From: *Phillip Benton
> * Reply-To: *Society to Improve Diagnosis in Medicine, "pgbentonmd at AOL.COM
> "
> * Date: *Thursday, October 8, 2015 at 1:17 PM
> * To: *"IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG"
> * Subject: *Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> I am an experienced physician-attorney (Medicine 54 Years, Law 44 years)
> teaching Medical Malpractice at a top tier law school for almost 20 years.
> Relevant to this discussion is the fact that *missed or delayed diagnosis**
> tops the list of causes for awards to injured plaintiffs at mediation or in
> jury trials.* Accounts of serious medical error by the *Institute of
> Medicine *(1999) and the *Journal of Public Safety* (Sept, 2013) document
> missed or delayed diagnosis as a leading cause of preventable harm. The IOM
> estimate of up to 98,000 preventable deaths per year means one every 5
> minutes 22 seconds; Dr. John James' evidence-based 2013 update of up to
> 440,000 preventable deaths per year equals one every 72 seconds.
> One standard approach by the plaintiff''s attorney is to have the
> defendant physician or defense expert witnesses agree that it is *important
> to make a differential diagnosis, and then to first rule out the most
> serious and dangerous diagnoses on that list.* In other words *it is not
> the odds but it is the stakes* that matter most. Greater experience of a
> physician may move this process from System II (rational) toward System I
> (intuitive) thinking, but the point is that juries (i.e., patients)
> routinely agree that you should always deal with the most important things
> first.
> A common expression heard when an uncommon disease is misdiagnosed is that *"When
> you hear hoofbeats you think of horses, not zebras."* The savvy attorney
> will then then ask *"And how do you tell the difference? (pause) You
> look!" *Adequate testing to first rule out life-threatening conditions,
> treatable if caught early, may often allow a successful defense.
>
> Phillip G Benton, MD, JD
> Atlanta Georgia
>
> -----Original Message-----
> From: Bob Latino <blatino at RELIABILITY.COM>
> To: IMPROVEDX <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
> Sent: Thu, Oct 8, 2015 10:55 am
> Subject: Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
> I am not a physician nor a clinician, so I come at this issue basically
> from the perspective of a patient.
>
> When the physician becomes the patient, what is the expectation of them
> towards their care provider in terms of their diagnosis?
>
> Physicians themselves would obviously be more critical of their peer's
> diagnosis when their lives are involved, because they are 'insiders' and
> know the probing questions to ask about how the diagnosis was derived. What
> are those questions? What should the non-clinical patient be asking of
> their doctors when they provide a diagnosis?
>
> I am in the investigation business and work in aviation, nuclear power,
> military and other potentially life-threatening businesses. Many in these
> businesses have to make spur of the moment decisions (diagnosing the
> problem) and then quickly act on it.
>
> I will take pilots for instance. I know healthcare has taken an interest
> in Crew Resource Management (CRM) from the training that pilots receive
> about effective cockpit communications and teamwork. They too have to
> quickly make a diagnosis and act on it accordingly.
>
> The difference between a pilot and a doctor in these situations is that
> the pilot and crew's lives are at stake (along with the passengers) as
> well, based on the accuracy of their diagnosis, decisions and actions.
>
> Given this informative debate about probabilities and looking at
> situations/patients singularly versus as a population, how does a pilot
> make their quick assessment versus a doctor and their diagnosis? Does the
> fact the pilot's life is at stake differ in their decision as opposed to a
> doctor, whose life is not likely at stake based on their decision? Does it
> matter? Should it?
>
>
> *Robert J. Latino, CEO*
> Reliability Center, Inc.
> 1.800.457.0645
> blatino at reliability.com
> www.reliability.com
>
> *From:* Jason Maude [mailto:Jason.Maude at ISABELHEALTHCARE.COM
> <Jason.Maude at ISABELHEALTHCARE.COM?>]
> * Sent:* Thursday, October 08, 2015 10:09 AM
> * To:* IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
> * Subject:* Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> Although this is a very stimulating debate, I am struggling to understand
> how relevant it actually is to the diagnosis of individual patients as a
> key additional variable will always be the personal consequences of a wrong
> decision. The key difference with a probabilistic approach in life
> assurance or similar versus diagnosis of a particular patient has to be the
> consequences of getting it wrong. This means that nobody is likely to
> follow a purely probabilistic approach if they know the patient might die
> if they didn’t check for something even if it was a lower probability. The
> odds of winning the lottery are ludicrously bad but because the prize is so
> big (upside consequences) people still try their luck. Personal
> consequences will always seriously affect rational calculations of
> probability.
>
>
> Jason Maude
> Founder and CEO Isabel Healthcare
>
> *From: *"Jain, Bimal P.,M.D." <BJAIN at PARTNERS.ORG>
> * Reply-To: *Society to Improve Diagnosis in Medicine <
> IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, "Jain, Bimal P.,M.D." <
> BJAIN at PARTNERS.ORG>
> * Date: *Thursday, 8 October 2015 12:13
> * To: *"IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG" <
> IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
> * Subject: *Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> We can only comment on and critically evaluate material that is published.
> I find it simply amazing that a probabilistic approach in which probability
> is evidence has not been employed in even a single amidst hundreds of
> published CPCs and clinical problem solving exercises. Dr. Brush dismisses
> CPCs as artificial, pedagogical exercises employing System 2 thinking over
> days or weeks. This is all the more reason to employ a probabilistic
> approach as the discussants then have plenty of time to estimate prior
> probabilities and calculate posterior probabilities. This is not done
> simply because this approach has not been found useful for diagnosis. Some
> time back, I carefully examined 50 consecutive CPCs in NEJM from July 2013
> to OCTOBER 2014. I found the word probability mentioned only once in these
> 50 CPCs. If Dr. Brush thinks this approach is suitable only for System 1
> thinking in diagnosis ,Croskerry has pointed out the danger of such
> thinking in causing diagnostic errors. At present, the emperor does not
> appear to have any clothes with regard to probabilistic approach to
> diagnosis in these exercises. What is needed ,I think are head to head
> observational or experimental studies comparing usual to probabilistic
> approach in real patients.
>
> The adage ‘Common things are common’ is useful only in indicating chance
> of a disease in a given patient. Certainly, we should look for a common
> disease first as it has the greatest chance of being found. The problem
> arises when a frequency or probability is taken as evidence for a disease.
> There is little doubt in my mind, diagnostic errors due to failure to
> suspect a disease in patients with atypical presentation in studies of
> Hardeep Singh and John Ely arose from interpreting low prior probability as
> absence of evidence for the disease.
>
> In discussion about STEMI, Dr. Brush rightly deals with all patients with
> STEMI regardless of prior probability in the same manner by taking them all
> for cardiac cath. His accuracy rate of acute MI of 85 percent in these
> patients is close to the rate of 90 percent in my paper. If he were to
> analyze his data he would find the majority of patients with acute MI to
> have intermediate or high prior probability.
>
> I refer the Central Limit Theorem with regard to distribution of prior
> probability which is a continuous variable.
>
> The main problem with a probabilistic approach is that it takes
> probability as evidence in a given individual patient while it is true only
> in groups of patients. There is no proof that it improves diagnosis in
> actual practice. Its use appears to have become a dogma which is hindering
> efforts to reduce diagnostic errors. It is only by looking at diagnosis in
> actual practice such as in studies of H. Singh and J. Ely and analyzing
> results without putting on probabilistic glasses that we shall make
> progress.
>
> I mention three examples from history of science of dogmatic beliefs
> hindering progress which was made only when phenomena as they occur were
> analyzed.
> 1. Since the time of Plato, the belief in planetary orbits being circular
> due to perfection of a circle as a geometrical figure. All contrary
> observations were explained away by drawing circles(epicycles) within
> circles. It was only two thousand years later that Kepler determined the
> orbit of Mars to be an ellipse when he actually observed and analyzed its
> movement.
> 2. Since the time of Aristotle, every movement was believed to require a
> mover. Contrary observations such as flight of an arrow were explained away
> in an absurd manner. Again, about two thousand years later, the true law of
> motion, that it is change in motion and not motion itself that requires a
> force was discovered when Galileo observed and analyzed actual motion of
> rolling balls.
> 3. And nearer to our age, there was a widespread belief in Absolute Time
> since Newton declared it to exist in the 17th century. It was only in
> early 20 the century this belief was overthrown by Einstein by his
> insightful analysis of actual time in terms of clocks and trains.
>
>
> Bimal
>
>
> Bimal P Jain MD
> Pulmonary-Critical Care
> North Shore Medical Center
> Lynn MA 01904
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *From:* John Brush [mailto:jebrush at ME.COM <jebrush at ME.COM>]
> * Sent:* Saturday, October 03, 2015 8:58 AM
> * To:* IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
> * Subject:* Re: [IMPROVEDX] Fwd: [IMPROVEDX] IOM report is released -
> Diagnosis in actual practice
>
> I’m afraid that I can’t agree with Dr. Jain’s argument. I think his
> argument is circular, difficulty to follow, and selectively self-serving.
> We have an adage in medicine: “Common things are common.” Otherwise, every
> diagnostic exercise would become a wild goose chase, leading us to look
> into every remote possibility every time. Having said that, I can also say
> that if we collect cases over time, uncommon things become common. Someone
> somewhere will eventually win the lottery. Uncommon diagnoses do occur
> eventually. But the exceptions should not define the rules.
> The STEMI case that Dr. Jain presents proves my point. I am in
> interventional cardiologist who frequently takes patients with suspected
> STEMI to the cath lab for intervention. I have been getting direct feedback
> on these cases for about 25 years. I can tell you that there is a false
> positive rate of about 15% among STEMI alerts that are taken to the cath
> lab (numerous reports in the literature confirm that estimate). We allow
> that false positive rate because we make a subjective calculation of
> expected value. Even if a patient has a relatively low initial prior
> probability of STEMI, like Dr. Jain’s example, we don’t want to miss a
> serious diagnosis like a STEMI. The EKG findings change the probability
> estimate and make a STEMI quite plausible in such a patient. In a patient
> like Dr. Jain’s example, we know that there is about a 50-50 chance of
> finding an occluded artery, which is certainly high enough to activate the
> cath lab. And sure enough, over time, 50% is about the frequency that we
> find in such patients.
> Dr. Jain references central limit theorem. That theorem applies to
> probability for a continuous variable, and states that for any
> distribution, the sample means of repeated samples will become a normal
> distribution. I’m not sure I follow his argument that it applies to a
> probability distribution of categorial variables. A diagnostic category is
> a countable variable. Kolmogorov’s principles, however, do apply. The
> probabilities of all of the possibilities do add up to one, if they are all
> independent. General knowledge of these probability principles can help us
> organize our thinking.
> When we see a patient with chest pain in the ED, we start to narrow the
> sample space by asking questions and making observations. For example, we
> can eliminate the possibility of a stab wound very quickly by noticing that
> there is no knife in the chest. Through early hypothesis generation, we
> narrow the range of possibilities to the point were we can start the
> process of iterative hypothesis testing. We have at our disposal many
> possible tests that we can perform. We can send a troponin, do a CT scan
> for dissection, do a stress echo, go directly to the cath lab, etc. We
> can’t do all of these tests at the same time, and we probably don’t want to
> do every test on every patient. So how do we decide what test to do first?
> We do a little mental calculation of the subjective probabilities, which
> gives us an idea of the expected value of each test. We don’t want to miss
> a diagnosis with serious consequences, like MI or dissection, so an EKG and
> CXR are done on virtually everyone, regardless of the prior probability.
> But we narrow the sample space as we hone in on the correct diagnosis. We
> don’t want to narrow the search prematurely, and we use a differential
> diagnosis to help us guard against jumping to conclusions. All of this is
> guided by some notion of relative probabilities.
> Dr. Jain talks about the CPC method of diagnosis. This is a useful
> pedagogical exercise, where an expert can expound on clinical medicine, but
> it is very artificial, as compared to real world practice. An expert may
> spend days or weeks preparing a CPC discussion. His/her main goals are to
> not miss the diagnosis, and to eloquently discuss all of the possibilities.
> It is almost purely System 2 thinking. In the real world, with time
> constraints and uncertainty, we have to employ System 1’s intuition. It is
> helpful, however, if we calibrate our intuition through knowledge of the
> relative strength of evidence and the base rates of various diagnostic
> possibilities. I think that having an intuitive sense of probability is the
> essence of experiential knowledge. Savvy clinicians make good bets.
> The fundamental assumption of evidence based medicine is that the
> frequencies that we measure in populations of patients can be applied to an
> individual patient. The measured frequencies from our aggregated
> experience, or from the reports in the literature inform us on how we
> should think about an individual. Single event or single patient
> probability then becomes a degree of belief, which is then modified by
> additional information that we gain through diagnostic testing. In fact,
> the sensitivity and specificity of diagnostic tests are defined using a
> frequency notion of probability. They are cumulative probabilities,
> depending on where we draw the line of demarcation. Some tests, like an
> x-ray for a broken arm, are so compelling that they lead to absolute
> certainty. Other tests, like EKGs, stress tests, troponins, etc, don’t have
> perfect operating characteristics, however, and we are left with a
> probability estimate for each diagnostic possibility that is somewhere
> between 0 and 1. Usually we get to a point of certainty, but sometimes,
> through adductive reasoning, we are left with the most plausible diagnosis,
> but never really know for sure.
> I hate to drag the listserv through this back and forth again, but to me,
> Dr. Jain’s arguments seem to counter what we have been taught about
> evidence-based medicine, but also run counter to principles of cognitive
> psychology. Without some intuitive idea of probability and likelihood, we
> would be totally adrift in clinical medicine, so I just can’t let this go.
> John
>
> John E. Brush, Jr., M.D., FACC
> Professor of Medicine
> Eastern Virginia Medical School
> Sentara Cardiology Specialists
> 844 Kempsville Road, Suite 204
> Norfolk, VA 23502
> 757-261-0700
> Cell: 757-477-1990
> jebrush at me.com
>
>
>
> On Oct 2, 2015, at 2:45 PM, Mark Graber <mark.graber at IMPROVEDIAGNOSIS.ORG>
> wrote:
>
> Note and manuscript forwarded on behalf of Dr Bimal Jain.
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
>
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
>
> <Bimal Jain - The Role of Probability in Diagnosis.docx>
>
>
> *From:* Jain, Bimal P.,M.D.
> * Sent:* Thursday, October 01, 2015 1:54 PM
> * To:* 'Mark Graber'
> * Subject:* RE: [IMPROVEDX] IOM report is released - Diagnosis in actual
> practice
>
> Hi Mark and all,
>
> It is important to understand how diagnosis is performed in actual
> practice as a correct diagnosis is made after all 85 percent of the time in
> practice. To reduce diagnostic errors, we need to know if the method in
> practice needs to be improved or whether certain deviations from it need to
> be eliminated. The most puzzling issue in this regard is the role that
> probability plays or does not play in diagnosis. The puzzle arises because
> a probabilistic approach has been prescribed for a long time, but it does
> not appear to be employed in practice when we look at published CPCs and
> clinical problem solving exercises. Does this disparity imply that a
> probabilistic approach is not suitable for diagnosis in actual practice?
> This is certainly possible as diagnosis is performed in a given, individual
> patient with the aim of determining a disease correctly in that particular
> patient. And probability, as is well known has been employed most
> successfully in practice in areas such as epidemiology and life insurance
> business where the focus is on accuracy of prediction in a large group of
> persons, not on prediction in a given individual person.
> If we look closely, we note that a strict probabilistic approach in which
> a probability represents evidence may actually increase diagnostic errors
> specially in patients with atypical presentations by encouraging the
> cognitive bias of representativeness and inhibiting comprehensive
> differential diagnosis (discussed in attached paper).
> I have put together my thoughts on this subject in the attached paper ‘The
> role of probability in diagnosis’. Please review and comment on it. Thanks.
>
> Bimal
>
>
> Bimal P Jain MD
> Pumonary-Critical Care
> North shore Medical CENTER
> Lynn MA 01904
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
>
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
>
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
>
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
> ------------------------------
>
> Notice: This UI Health Care e-mail (including attachments) is covered by
> the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is
> confidential and may be legally privileged. If you are not the intended
> recipient, you are hereby notified that any retention, dissemination,
> distribution, or copying of this communication is strictly prohibited.
> Please reply to the sender that you have received the message in error,
> then delete it. Thank you.
> ------------------------------
>
>
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
>
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
>
>
>
>
> ------------------------------
>
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
>
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
>
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/
>
> ------------------------------
>
>
> To unsubscribe from IMPROVEDX: click the following link:
>
> http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
> or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
>
> Moderator:David Meyers, Board Member, Society for Improving Diagnosis in
> Medicine
>
> To learn more about SIDM visit:
> http://www.improvediagnosis.org/






Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine


HTML Version:
URL: <../attachments/20151018/38120924/attachment.html> ATTACHMENT:
Name: image001.gif Type: image/gif Size: 105 bytes Desc: not available URL: <../attachments/20151018/38120924/attachment.gif>


More information about the Test mailing list