stroke misdiagnosis disproportionate in the young says Washington Post

Ely, John john-ely at UIOWA.EDU
Thu Jul 10 13:21:39 UTC 2014


This is a wonderful outline of how to organize our efforts to reduce diagnostic errors.  I didn’t see anything missing, but if there is, we should add it to the outline and then nail it to the wall.  I think all of our individual efforts to reduce diagnostic errors can be placed into one of the categories of this outline.  I think Gordy Schiff has done similar organizational work in the past and may have some thoughts about this.  Anyway, thanks for a great summary that is so well communicated.

John Ely

From: David Newman-Toker [mailto:toker at jhu.edu]
Sent: Thursday, July 10, 2014 5:27 AM
To: John Brush; Society to Improve Diagnosis in Medicine; Ely, John; Swerlick, Robert A
Subject: RE: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Thanks, guys. Interesting stuff.

I completely agree with John Ely that the typical physician on a typical day neither uses quantitative Bayesian approaches, nor relies solely on strictly medical factors in making decisions (whether the ‘non-medical’ part is defensive medicine, being told by your boss to see patients faster, etc.).

I think what John Brush and I are saying is that when people make ‘gut’ diagnostic judgments (based on whatever factors), they are fundamentally probabilistic in nature, even if the math behind it is completely out of mind and difficult (or impossible) to articulate if prompted.

But the more important question for me is not how DO we make decisions but how SHOULD we make decisions. John B. says “… we would do this better if we thought about it more quantitatively.” I’m inclined to believe him (because we’re not quantitative enough, in general), although Geoff Norman and Kevin Eva might disagree, based on some of their experimental work suggesting that purely analytical reasoning doesn’t improve performance (note, however, that actively toggling back and forth between intuitive reasoning and analytical reasoning does improve performance).

This all then gets to Bob’s point. Perhaps we need more effective tools that help us do the math better (disease risk calculators; checklists or algorithms that use heuristics of known probability [e.g., decision rules], etc.)? I suppose it is still an open question whether it is possible to develop diagnostic decision support tools that improve real-time accuracy over current practice, improving patient outcomes without slowing us down too much… but I would bet it is possible, and it is certainly a testable hypothesis (we’ve obviously been working on this approach for dizziness and stroke).

However, if it is not possible to support better diagnostic decision-making by frontline clinicians at the bedside (which, by my read of the literature, is the location where most errors occur), then it seems to me that there are a limited number of options to reduce (harms from) diagnostic error…


1)      engineer out ‘communication’ failures (specimen labeling, test results reporting, referrals, follow-up appointments, etc.)

2)      develop more accurate diagnostic tests (i.e., technology)

3)      improve frontline physician ‘gut’ accuracy

a.       more effective diagnostic training methods (see Eva, PMID: 15612906)

b.      more effective diagnostic performance feedback methods (systematic follow-up; peer review feedback, etc.)

4)      improve access to physicians with better individual (or collective) ‘gut’ accuracy

a.       rapid referrals/triage or tele-consultation to specialists (ideally symptom-specific, rather than disease-specific specialists)

b.      ‘multidisciplinary team diagnosis’ or ‘independent diagnosis second review’ (including crowdsourcing for diagnosis)

5)      create additional safety nets to catch diagnostic errors sooner rather than later (i.e., before irreversible morbidity) after they occur

a.       more effective secondary diagnostic monitoring by allied health professionals (nurses, etc.)

b.      more patient/family engagement around self-monitoring for symptoms/signs of being ‘off plan’ for the diagnosis rendered

c.       more remote monitoring (i.e., technology) to capture deviations from ‘the expected’ disease course

Thoughts?

David


David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: John Brush [mailto:jebrush at me.com]
Sent: Tuesday, July 08, 2014 2:57 PM
To: Society to Improve Diagnosis in Medicine; David Newman-Toker
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

There seems to be a resistance to think about making a diagnosis using probability. We seem to want to think in absolutes and we know that a single patient ultimately only has one diagnosis, so we don't want to think about prior probabilities. We want to arrive at that single diagnosis quickly and with certainty. But that can lead to premature closure and the illusion of certainty.

Granted, there are several scenarios where a prior probability is unnecessary. First is when the diagnosis is obvious - a poison ivy rash, a broken arm, a UTI. These diagnoses don't require any  Bayesian reasoning of probabilities.

The second scenario is when you stumble on a diagnosis by serendipity. Left atrial myxomas are almost always diagnosed because someone ordered an echo for some other reason. No one would calculate the prior probability of a myxoma because it wouldn't be thought of prior to the test.

But in almost every other situation where we start with diagnostic uncertainty and multiple possibilities, we are tacitly thinking about probabilities. If we make a list of possible diagnoses a rank order, we are thinking probabilistically. If we expand the problem space by formulating a differential diagnosis, we are thinking probabilistically. We are thinking probabilistically as we work systematically to eliminate possibilities, decreasing the problem space and converging toward the one remaining diagnosis, that will ultimately be assigned a 100% probability.
My point is that we would do this better if we thought about it more quantitatively.

A couple of other points in response to Bimal's comments. There are almost no tests in cardiology with likelihood ratios of greater than 10 or less than 0.1. Troponin is 4.7. Imaging stress tests are 6.0. ST elevation has a PLR of 11 in appropriately selected patients. If used on unselected patients, there would be a spectrum bias that would result in different likelihood ratios. For definitive "gold standard" tests like cardiac catheterization, likelihood ratios are meaningless and can't be calculated.

And the specificity of ST elevation is not 100%. I know this well because I'm an interventional cardiologist who gets direct feedback on this. The false positive rate is about 15%, due to LVH, early repolarization, pericarditis, conduction defects, etc. Not infrequently, I am called for a STEMI, where the clinical situation is so atypical that we will not take the patient to the cath lab. There is a downside to ignoring the prior probability and only reacting to the ST elevation. The patient may have a PE or aortic dissection, and it could be detrimental to jump to the wrong conclusion. Usually the EKG is performed in the appropriate clinical setting and the ST elevation leads to the correct diagnosis, but sometimes, it requires some reasoning, even though the PLR of ST elevation is very high.
John

Sent from my iPad


From: Swerlick, Robert A [mailto:rswerli at EMORY.EDU]
Sent: Tuesday, July 08, 2014 11:37 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Here lies the crux of the problem. Our tools provide probabilities but we operate in a world where our jobs are defined by yes and no, embedded in a workflow with increasing production pressures. If one attempts to use tests without an appreciation of their probabilistic nature, it increases the likelihood of diagnostic error. If one attempts to operate using probabilistic tools, it becomes increasingly difficult to function in a world which demands yes or no answers.

Bob Swerlick

From: Ely, John [mailto:john-ely at UIOWA.EDU]
Sent: Tuesday, July 08, 2014 8:35 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Thanks David and Bimal.  Lots of great information in your emails.

From the primary care perspective, the question is not whether to cath the 65 year-old man, but whether to admit him.  He would definitely be admitted, even if his ECG was normal.  We don’t use Bayesian reasoning at the bedside because

1.  We don’t know the pretest probability for the individual patient, and even if we did, the post test probability is just a probability, not a yes or no.  Our job is yes or no.
2.  Other factors are not captured by Bayesian calculations (consequences of missing the diagnosis, malpractice worries, patient expectations, recency of most recent disaster, rationalizations based on fatigue, closeness to quitting time, pressure from office staff; i.e., things we never talk about and therefore never understand.)  Good article on this in JAMA (Kowey PR. A piece of my mind:  the silent majority. JAMA. 2011;306(1):18-9.)

John Ely


On Jul 7, 2014, at 4:23 PM, David Newman-Toker <toker at JHU.EDU<mailto:toker at JHU.EDU>> wrote:
Well, Bimal, there are certainly test results (if not tests) that are ‘diagnostic’, largely independent of pre-test probability. For instance, if you get an MRI brain on a patient and it shows a gigantic enhancing lesion with mass effect, you know a few things, independent of whether the patient is asymptomatic or symptomatic, and whether symptoms cohere (headache) or not (toe pain) (i.e., independent of factors that affect pre-test probability). Merely looking at the scan (and assuming it belongs to the patient in question), knowing nothing else about the patient, you know…


a)      the patient has a brain [LR roughly infinite]

b)      the brain is not normal [LR roughly infinite]

c)       there is a lesion that indicates a disease [LR roughly infinite]

d)      that disease is probably a brain tumor (but might be something else – e.g., stroke, demyelination, infection) [LR probably 10-30 for brain tumor, depending on the precise radiographic appearance]

e)      the patient probably has symptoms related to the lesion location in the brain [LR probably 2-5]

Let’s take your low prior probability STEMI case and assume, for the sake of argument, that you won’t go to the cath lab if your expectation is <10% the patient has an MI, but will >=10%. I don’t know enough about the ECG findings you describe, but if they never occur in patients who lack MI (i.e., 100% specificity), then their PLR (=sensitivity / [1-specificity]) is infinite. For findings with infinite LRs, pre-test probability is irrelevant --- your post-test probability with a positive test will always be greater than 10% (and always ~100%). You say, however, that the PLR is only 13. If the LR+ is only 13 (i.e., specificity is <100%), then the pre-test probability matters if it is below ~0.9%. Below that, the ECG findings you describe (if they really have a PLR of 13), only raise your post-test probability to a max of <10%. It may be that you would never order an ECG in someone with a pre-test probability <1%, but you still have to make some judgment about whether to do the ECG at all (based on a tacit prior probability estimate).

This is all by way of saying that you have constructed the problem space in such a way that Bayesian calculations are unnecessary --- if you only ever consider test results with PLR>10 and NLR<0.1 (i.e., you ignore completely anything less than a ‘slam dunk’ finding); and, on the PLR side, if you only apply the test in patients with a pre-test probability >1% and have a threshold for action of >10%, then residual judgments about pre-test probability in patients with positive test results will never be relevant (you still have to judge pre-test probability to be >1%... which you do without much mathematical reasoning using simple heuristics). You have chosen examples of test results and clinical problems that are relatively straightforward and result in simple, go/no-go, binary decision rules based on the presence or absence of specific results (slam dunk STEMI by ECG) of a particular test appropriate to a particular type of patient. Within this frame, you are correct --- pre-test probability is not really relevant… but only because the problem is constructed that way, not because there aren’t tacit probabilistic decisions built into the decision-making.

I am less clear on your 65yo with multiple risk factors and typical chest pain (LR ~113 for coronary atherosclerosis) and a pre-test probability of 90% (maybe closer to 99%?). If the pre-test probability is actually 90%, then (a) you shouldn’t need to order an EKG --- you should just take him to the cath lab [if the 90% probability is for the type of MI for which appropriate treatment is the cath lab]; and (b) you shouldn’t let the non-specific T wave changes dissuade you (both because of the high pre-test probability in the patient and, by your own admonition/logic not to bother with results that have NLR > 0.1, which I assume is the case for nonspecific T wave changes). If you really wouldn’t treat this patient as acute MI, then there is something you’re not telling us in your 90% estimate (e.g., 90% for any MI, but only 20% for a STEMI or cath lab-appropriate MI).

As for the issue of how doctors think… in my opinion, you are correct --- many physicians do not understand probability or test results interpretation --- they often think in absolutes, rather than in uncertainties. So they assume, for many binary tests, that a negative test means the disease is absent and a positive test means it is present. NPV and PPV, as we all know, however, are both usually <100%, because tests are generally imperfect.

Best,

David


David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: Jain, Bimal P.,M.D. [mailto:BJAIN at PARTNERS.ORG]
Sent: Monday, July 07, 2014 12:23 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Dear John,David,

Thank you both for your insightful comments.


1.       I am neither a frequentist nor a Bayesian exclusively. I consider them both as valid interpretations of probability as they both fulfill the rules of probability calculus.

2.       My main interest is in studying clinical diagnosis as it is done in actual practice as this will make us appreciate real life constraints which modify the method we use.

3.       I agree , a Bayesian approach, on face value, appears reasonable and is elegant, but the fact remains, it is only an application of a mathematical theorem which does not take account of some real lfe constraints on diagnosis as I discuss below.

4.       It is not surprising, therefore, it is not employed in actual practice as seen from following examples. Even though I am not a cardiologist, all my examples are from cardiology as they best serve my purpose. Please excuse me , John, if I have made errors in my interpretation of how diagnosis is actually performed in these patients.

5.       The first example is of an actual patient, who was discussed in a clinical problem solving exercise (Pauker, NEJM 1992).  A 40 year old, healthy woman, without any any cardiac risk factor presents with highly atypical chest pain and is found to have acute Q wave and ST elevation EKG changes (acute EKG changes). The pretest probability of acute myocardial infarction (MI) is estimated to be 7 percent. It is combined with the known likelihood ratio of acute EKG changes of 13 by Bayes’ theorem to generate a post test probability of 50 percent. It seems to me the Bayesian diagnosis from this post test probability should be of acute MI being indeterminate in this patient. But the discussing physician diagnosed acute MI with near certainty from acute EKG changes alone which he considered to be strong evidence for it. Clearly, he did not diagnose in a Bayesian manner.

6.       Let us consider another patient, a 65 year old man with multiple cardiac risk factors who presents with highly typical chest pain. His EKG reveals non specific T wave changes. The pretest probability of acute MI is very high, let us say , 90 percent while the LR of nonspecific T wave changes is about 1. By combining them we get a post test probability of 90 percent, from which the Bayesian diagnosis would be of near certain presence of acute MI in this patient. I doubt however if this diagnosis would be made in actual practice.

7.       EKG reading physicians routinely diagnose acute MI definitively from acute EKG changes alone without knowledge of clinical presentations in these patients.

8.       All patients with acute EKG changes seen in ER are diagnosed to have STEMI and sent to cardiac cath. Lab regardless of pretest probabilities.

9.       In all these examples,diagnosis is performed in a non-Bayesian manner because, I suggest it accords with our experience. Thus in the 40 year old woman, we would need to have encountered a number of patients like her with acute EKG changes to experience a frequency of acute MI of 50 percent tin these patients which would validate our indterminate diagnosis of acute MI in the given patient. It is well known from experience we are not likely to encounter one, let alone a number of such patients. On the other hand, we would have encountered a number of patients with varying pretest probabilities in whom acute EKG changes diagnosed acute MI correctly (Rude, Am J Card 1983). It is this experience , I suggest, the dicussing physician calls upon to diagnose acute MI with near certainty.

10.   In the 65 year old man, our experience of non specific T wave changes in other patients leads us not to diagnose acute MI definitively in the given patient even though the psot test probability is very high.

11.   I believe it is the wide variation in pretest probability of a given disease in different patients which prevents us from having experience which would justify a Bayesian diagnosis in a given , individual patient.

12.   I think, we need to observe, analyse and experiment with clinical diagnosis in actual practice to learn how it is actually performed which may be quite different from our theoretical preconceptios, however reasonable they may appear.

13.   An example from history of science is quite instructive in this regard. The entirely reasonable appearing beliefs that heavy bdies fall faster than lighter ones and that every motion requires a mover or force were held for a long time.. Both these beliefs were shown to be erroneous when Galileo observed, analysed and experimented on actual motion of bodies.



Thank you all for amost enjoyable and important discussion on clinical diagnosis.



Bimal



Pulmonary-Critical Care

NorthShore Medical Center

Lynn, MA 01904



From: John Brush [mailto:jebrush at ME.COM]
Sent: Friday, July 04, 2014 10:10 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

I’ll wander back into this discussion because I can’t help myself. I think we established in a previous email exchange that Bimal is a frequentist and I am a Bayesian.
                The essence of our dilemma is: “How do we apply statistics regarding the operating characteristics of tests and regarding experience from populations of patients to the care of an individual patient?” In the emergency room, we generally get one shot at making a correct diagnosis. How do we apply our tools and experience to give that encounter the best chance of success?
                As we all know, the frequentist argues that the notion of probability can only be applied to populations of patients. The frequency notion of probability describes the rate at which something occurs over the long run. The personal notion of probability is different in that it gives us a way to think about a single patient or event. The personal notion of probability is our degree of belief in some proposition (such as a diagnostic category). It is a way to quantify whether we feel hot or cold about some idea. For conditional probability, the Bayesian approach does require that we assign an estimation of prior probability.
                As Bimal points out, the frequentist argues that assigning a prior probability is too subjective, and therefore, when used to calculate a posterior probability, the whole thing falls apart. But the frequentist asks you to make other assumptions that are just as conceptually difficult. The frequentist asks you to imagine having your situation repeat itself and it asks you to imagine, if the situation is repeated, how something would turn out over the long run. For a diagnosis, which is a categorical variable, you would have to imagine taking repeated samples from an imaginary population to create an imaginary binary distribution that would describe how things would turn out over the long run. To me, that is more of a stretch in thinking than the notion of a subjective prior probability.
                I think it is more reasonable to try, based on available information and intuition, to assign a point estimate of prior probability. Granted, there is ambiguity around that estimate, so you could imagine a distribution of prior probability estimates. If you know very little about the patient, the distribution would be broad. If you have detailed information, the distribution would be narrow around a specific point estimate of prior probability. To be clear, I am describing a conceptual distribution of probability estimates, not a distribution of rates from imaginary repeated samples.
                The test, itself, is not perfect. We have sensitivity and specificity to estimate how imperfect a test is, but there is ambiguity around those estimates as well, so that the predictive value of a test result should be viewed as a distribution that can have a variable effect on our prior probability.
                The Bayesians describe a “triplot,” which gives a visual demonstration of how a prior probability (which is a distribution) is changed by a test result (which is also a distribution) and the combination of the two distributions yield a third distribution that visually represents the posterior probability. Generally, the additional new information will reduce ambiguity, so the shape of the posterior probability distribution curve is typically more narrow than the shape of the prior probability distribution curve, and the point estimate of probability is moved in either direction, depending on whether the test result is positive or negative. I’m not a statistician, but to me, the concept of a triplot gives me a way to visualize the ambiguity around the probability estimates, and how the prior probability and new information distributions combine to yield a posterior probability distribution.
                Given the difficulty in precisely calculating this real time in the real world, we “satisfice” to use Herbert Simon’s word. We use the anchoring and adjusting heuristic to estimate the prior probability and “calculate”  the posterior probability. We generally have some idea in our heads of the point estimates and the ambiguity around those point estimates. The thoughtful clinician is constantly thinking “What do I think, and am I sure?"  The reason that this works in practice as well as it does is that the estimates don’t have to be exact. We just need to know whether the posterior probability estimate is on one side or the other of some threshold.
                The challenge, in my mind, is to teach trainees these concepts, and to assure that we all do this consistently, reliably, and most effectively. How can we make our subjective estimates as objective as possible, and how can we do this consistently? In some cases, the situation may be simple enough or the test may be effective enough that you can automate this with a computer to get maximum reliability. In most situations, however, there is too much uncertainty, and we will have to rely on the human mind and on intuition to make the necessary associations and estimations, using heuristics.
John

John E. Brush, Jr., M.D., FACC
Professor of Medicine
Eastern Virginia Medical School
Sentara Cardiology Specialists
844 Kempsville Road, Suite 204
Norfolk, VA 23502
757-261-0700
Cell: 757-477-1990
jebrush at me.com<mailto:jebrush at me.com>



On Jul 3, 2014, at 1:50 PM, David Newman-Toker <toker at JHU.EDU<mailto:toker at JHU.EDU>> wrote:

I’ve re-engineered the trail below to try to restore it to ‘togetherness’, because I messed it up before (trying to keep it together)! ☺

Dear Bimal,

Thanks for sharing your interesting take on diagnosis. If I understand correctly, you have squarely placed us into the ‘frequentist’ vs. ‘Bayesian’ statistical inference space… a space that may be unfamiliar to many of our ListServ readers. I commend to those unfamiliar two classic papers on inference related to this issue written by Steve Goodman, who is one of the most rigorous thinkers in the world on these issues:


Goodman SN. Toward evidence-based medical statistics. 1: The P value fallacy.

Ann Intern Med. 1999 Jun 15;130(12):995-1004. PubMed PMID: 10383371.
http://www.ncbi.nlm.nih.gov/pubmed/10383371
http://www.google.ae/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja&uact=8&ved=0CDAQFjAC&url=http%3A%2F%2Fwww.perfendo.org%2Fdocs%2FBayesProbability%2F5.3_GoodmanAnnIntMed99all.pdf&ei=h4y1U-ycIqef7AaO34HIDg&usg=AFQjCNEGg_9Ue0WCuA9Lnoft9RwjIxDNdA&bvm=bv.70138588,d.bGE

Goodman SN. Toward evidence-based medical statistics. 2: The Bayes factor. Ann
Intern Med. 1999 Jun 15;130(12):1005-13. PubMed PMID: 10383350.
http://www.ncbi.nlm.nih.gov/pubmed/10383350

The first of these two articles outlines the problems inherent in trying to make individual-case decisions about probability (short-run inference) based purely on observed long-run data (long-run inference). The paper does so in regard to assessing the strength of evidence from research studies, but the argument is conceptually the same as that which would be applied to short-run vs. long-run inferences in diagnosis.

As I understand the Neyman Pearson hypothesis test and your argument below, your long-run data about EKGs for diagnosis of MI in chest pain patients would support rejecting the (long-run) null hypothesis that EKG findings do not predict MI, independent of any specific patient characteristics… i.e., EKGs probably do predict MI, on average, most of the time, if we do them repeatedly in a lot of patients over time.

But, despite the appeal of using this approach to avoid the messy business of assigning pre-test probabilities, one cannot meaningfully convert that long-run inference into a specific short-run inference about an individual patient whom you are trying to diagnose, without taking into account pre-test probability for that specific patient (i.e., prevalence, judgments about the extent of match between that single patient and the previous patients from your long-run EKG/MI experience). You do that tacitly already in your argument below by restricting discussion to patients with ‘chest pain’… but, by that I assume you mean ‘chief complaint chest pain presenting for medical care’ rather than ‘incidental minor chest pain’ in a patient who suffered trauma and has a broken leg or in a patient who just ate a spicy meal and doesn’t seek medical care. In other words, the EKG PPV is 95% (or whatever the numbers are) precisely because your patients have certain characteristics that put their pre-test probability into the relevant operating range for a test of sufficient LR (your >10 rule of thumb) to make the correct diagnosis, rather than yield a false positive --- i.e., you already know SOMETHING about pre-test probability. Same, presumably, in the reverse, for ruling out a disease (with an NLR <0.1), rather than ruling it in.

So you can use Neyman Pearson to have a high probability of being right, on average, across patients, but not to have a high probability of being right for the patient in front of you… for that, you need Bayes theorem (i.e., pre-test probability plus likelihood ratios to derive post-test probability). Neyman Pearson can help us favor one approach over another (e.g., EKG >> reading sheep entrails for MI diagnosis), but it can’t tell us whether a particular patient likely has a particular diagnosis (e.g., MI) when only considered in a vacuum (i.e., without pre-test probability estimates).

A more nuanced framing of your argument might be that we need not obsess about assigning a highly specific pre-test probability (e.g., 10% vs. 20%) if we already have sufficient prior knowledge of pre-test probability to know that we are in the ‘sweet spot’ operating range where our test will help us diagnostically. This is the ‘thresholds’ argument in my prior email in this trail below, so I won’t reiterate here.

At least that’s my understanding.

Best,

David


David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.


From: Jain, Bimal P.,M.D. [mailto:BJAIN at PARTNERS.ORG]
Sent: Wednesday, July 02, 2014 8:53 AM
To: 'Society to Improve Diagnosis in Medicine'; David Newman-Toker
Subject: RE: [IMPROVEDX] stroke misdiagnosis ... Washington Post [CB]

Dear all,

I present below a realistic model of clinical diagnosis.
There are three features that any realistic model, that is, a model which explains diagnosis as it is performed in real life needs to take into account.. These three features are:

(a)    The clinical aim of diagnosing a disease correctly in each individual patient.

(b)   The wide range of clinical presentations and therefore of pretest probabilities of a given disease in different patients.

(c)    The validation of a diagnosis in a given patient by our experience.


1.       In developing this model, let us consider one particular disease, acute myocardial infarction(MI) It is well known its presentations and therefore its pretest probabilities , vary widely from characteristic chest pain in a middlle aged man with multiple cardiac risk factors ( high PTP, say around 80-90 percent) to uncharacteristic chest pain in a healthy 40 year woman with no cardiac risk factor (low PTP,7 percent) (Pauker, NEJM 1992).

2.       Let us now take the group of all patients with varying clinical presentations and therefore varying PTPs in whom acute MI could be possibly present. Each patient in this group can be looked upon, I suggest, as being drawn from a hypothetical, infinite population in which the distribution or frequency of acute MI corresponds to the pretest probability in the patient.

3.       The group of patients in whom we would suspect acute MI constitutes therefore a heterogenous group with varying PTPs. I would like to point out our experience of diagnosing acute MI would be gained solely from this group.

4.       Our aim clinically is to diagnose acute MI if it is present correctly in each individual patient. We find knowing the PTP in a given patient does not help much as the given patient has been drawn from the corresponding infinite population purely by chance. This is true whether PTP is high or low. Therefore all we can do is to suspect acute MI in the given patient from the presentation.

5.       The next step is to determine if acute MI is present or not, in our given patient.

6.       For this purpose, we perform a test an EKG in our patient. Let us suppose we observe acute Q wave and ST elevation changes (acute EKG changes).

7.       We diagnose acute MI definitively from acute EKG changes which have a LR of 13

8.       If we diagnose acute MI definitively from acute EKG changes repeatedly in patients in our heterogenous group, we shall diagnose correctly in 90 percent patients (Rude, Am J Card. 1983)

9.       The definitive diagnosis of acute MI from acute EKG changes in our particular patient is validated therefore by our experience.

10.   The argument employed here is not probabilistic(Bayesian) as PTP does not play any role in diagnosis of acute MI from acute EKG changes.

11.   Instead, the argument employed, I suggest, is Neyman’s confidence argument.

12.   In the confidence argument, as is well known, a test result (usually with values in an interval called confidence interval) is repeatedly sampled from a heterogeneous population of patients with a certain disease with varying PTPs. it diagnoses the disease correctly in about 95 percent patients.

13.   An example of a confidence argument is employment of PSA level in interval 0-4 which correctly diagnoses absence of prostate CA in 95 percent persons in a heterogenous population.

14.   In clinical diagnosis, a confidence argument is modified slightly, in that a test result like acute EKG changes may be a point(present or absent) instead of an interval and the accuracy rate may be other than 95 percent.

15.   In clinical diagnosis it is customary to employ a test result with LR of 10 or higher(Jaeschke 2002)  for definitive diagnosis of a disease as it leads to about 90 percent accuracy in a heterogenous population. Thus LR of acute EKG CHANGES IS 13, LR of positive chest CT angiogram for pulmonary embolism is 21, LR of positive venous ultrasound for DVT is 19.

16.   An essential feature of the confidence argument is that PTP does not play any role in it.

17.   That it is emplyed in actual practice is clearly seen from the following examples:

18.   (a) EKG reading physicians  diagnose acute MI with near certainty from acute EKG changes alone. Similarly, radiologists diagnose pulmonary embolism from positive chest CT angiogram alone.

19.   (b)Acute MI was diagnosed with near certainty from acute EKG changes alone in the 40 year old woman with uncharacteristic chest pain mentioned above(Pauker 1992)

20.   In conclusion, I suggest, clinical diagnosis is performed by suspecting a disease from a presentation in a given patient and the suspected disease is diagnosed definitively from a test result with LR of 10 or higher by employing a confidence argument.

Please review and comment. Thanks

Bimal P Jain MD
Northshore medical center
Pulmonary-Critical Care
Lynn MA 01904




From: David Newman-Toker [mailto:toker at JHU.EDU]
Sent: Thursday, June 26, 2014 4:11 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis ... Washington Post [CB]

I think there are some critically important concepts in this trail. I’ll offer my take on a few without trying to address every point.

Note that I wrote this email before Frank’s email came through, but I pasted here after his to keep the trail in one piece. I think Frank and I probably agree on most of this… since it looks like we independently came up with several of the same arguments.


1)      HISTORY AND EXAM AS TESTS: we tend to talk about ‘pre-test’ probability before a laboratory-based or imaging diagnostic test, but all elements of history and exam are also ‘tests’ (just often with poorly studied sensitivity and specificity); this means that every piece of information about vascular risk factors, age, symptom particulars, etc. has an impact in shifting pre- to post-test probability… it is not enough just to stop at the population prevalence of a disease and call that pre-test probability… it is highly case specific --- which is why correct case representation/problem formulation can so powerfully influence disease probability estimates (Dr. Zamir’s point)



2)      THRESHOLDS FOR ACTION: it is important to separate discussions about disease probability from discussions about what probabilities are low enough to assume (i.e., act as if) the patient does not have the target disorder (sometimes called threshold decision-making); diagnostic tests are only useful for patients whose pretest probability is between one of two boundaries --- at the low end, the ‘testing threshold’ (below which the disease probability is so low that the test, of a fixed ‘rule in power’ [i.e., positive likelihood ratio] could not hope to increase the post-test probability above a the level where treatment would be indicated) and, at the high end, the ‘test-treatment threshold’ (above which the disease probability is so high that the test, of a fixed ‘rule out power’ [i.e., negative likelihood ratio] could not hope to decrease the post-test probability below a level where treatment would no longer be indicated) (Pauker & Kassirer NEJM 1980) --- the importance of powerful tests (Dr. Jain’s point about PLR >10 and NLR<0.1) is that they more often help us to cross these thresholds… but there is no specific level at which tests are uniformly helpful or unhelpful, because it depends on pre-test probability and thresholds for a specific symptom-disease-diagnostic test combination; if you have a good enough test, the operating range may be quite wide --- the NLR on HINTS is about 0.01 and PLR is about 25 in acute, continuous dizziness, so it might be able to meaningfully influence your decision making about stroke with a pre-test probability for stroke anywhere between about 0.1% and 50%, depending on the patient’s personal preferences



3)      VALUE JUDGMENTS AND SHARED DECISION MAKING: threshold decisions are ‘value laden’ and should be based upon some assessment of the risks and benefits of subsequent action (further testing or treatment) with an emphasis on the patient’s personal valuation of various options and outcomes (i.e., shared decision-making); there is no absolute link between disease probability and action at some arbitrary, fixed threshold --- one patient might be unhappy risking a 0.1% chance of missed stroke, while another might be comfortable with a 10% risk of a missed stroke, based solely on personal preference; the risks of treatment or harm from the disease and its long term consequences might differ dramatically across patients --- e.g., a 30 year old in previously good health vs. a 99 year old with end stage cancer, sepsis, and multi-organ failure; if society deemed that diagnostic tests likely to produce ‘cost-effective’ benefits of resulting downstream treatment (e.g., <$50-100K/QALY) should routinely be implemented, it might mean that for some disease scenarios the testing threshold was at a pre-test probability of 0.1% while for another it was at 1% and yet another it was at 10%; and so on…



4)      CASE ATYPIA: as Pat Croskerry pointed out, atypical cases are disproportionately associated with diagnostic error; in some cases, this represents a low pre-test probability; in other cases, it represents a failure of our existing mental models (symptom/illness scripts, heuristics) to more accurately represent the problem and the spectrum of cases/causes, including our awareness of findings that should (but don’t necessarily for all clinicians --- Dr. Zamir’s point about superior diagnosticians with superior mental models) properly influence our pretest probability (Dr. Jain’s point about decision rules); sometimes, it is an amalgam of these things --- for instance, the pre-test probability of stroke in an ED patient with dizziness or vertigo as a chief complaint (given no other information) is 3-5%; many people would consider this a ‘low’ pre-test probability, but (unknown to most clinicians) if the patient has acute, continuous dizziness or vertigo, the pre-test probability is ~25%... which most people would consider a relatively high pre-test probability for a dangerous disorder… so the structure of the problem formulation (and the diagnostic expertise and experience of the physician) can have a profound effect on a clinician’s estimate of pre-test probability; it is also atypical for strokes to present with dizziness or vertigo as a chief complaint (no more than 10-20% of strokes present this way, and it is probably closer to 5-10% that do) --- a typical mental model of stroke is that of a patient with a ‘hemi’-deficit (weakness or numbness) or aphasia (i.e., a typical middle cerebral artery territory syndrome), but most patients who present with dizziness or vertigo as the lead symptom of stroke (i.e., a typical posterior circulation syndrome) do not have such deficits (no more than 20% have any lateralizing signs typically thought of as ‘focal’ neurologic deficits); another typical heuristic for stroke is that patients are older and/or have vascular risk factors… but a large fraction of those presenting dizziness or vertigo have vertebral artery dissections as the cause, and these patients are typically younger and lack vascular risk factors; so dizzy strokes are atypical on most counts, but not always low pre-test probability



5)      ROLES OF META-COGNITION, EDUCATION, AND DECISION SUPPORT TOOLS: meta-cognition alone will probably not help us when our mental models fail us and we underestimate disease probability based on how atypical a case appears to be (even though someone more expert or experienced would more accurately identify a higher disease probability); education in diagnostic skills could certainly help, but it will require a commitment to educate based on repetitive case examples that are progressively more “atypical” --- i.e., we can’t just teach from “classic” cases and expect learners to be able to identify atypical forms (experts become experts through extensive experience with the boundaries around each condition, rather than just the ‘sweet spot’ of the ‘obvious’ case; decision support tools can help us… but probably only if we commit to use them routinely under pre-specified circumstances --- required checklists, pathways, (stroke) risk calculators, etc. for all patients with a particular symptom or problem (e.g., dizziness or vertigo in the ED)… rather than choosing to use them only if the case seems representative enough for us to think of stroke in the first place



David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: Papa, Frank [mailto:Frank.Papa at unthsc.edu]
Sent: Thursday, June 26, 2014 4:03 PM
To: Society to Improve Diagnosis in Medicine; David Newman-Toker
Subject: RE: [IMPROVEDX] stroke misdiagnosis ... Washington Post [CB]

A couple of comments regarding recent considerations offered by Drs. Jain, Zamir and Newman-Toker ….

Regarding Dr Jain’s suggestion that the rates of diagnostic error in the PIOPED study, the Newman-Toker study, and the broader error estimates produced by Dr Graber may be a reflection of a normalized distribution of difficult to easy to diagnose cases – with erroneously diagnosed cases representing those with  low,  objectively determined probability estimates …

I believe that his suggestion that such objective probabilistic estimates that a given case (with its particular set of signs and symptoms) is indeed a representation of disease ‘x’, might be correlated with a clinician’s subjective, pretest estimate (of the same case) as representing a low, intermediate, high ‘match’ with disease ‘x’, is both insightful and informative. While I am not aware of any research and evidence in support of his hypothesis, awareness of prior, and the initiation of new research involving these issues, would be very useful for those interested in further understanding the factors underlying DDX error and accuracy.

Regarding Dr Zamir’s comments that physicians can vary widely in their respective subjective, pretest estimate of the probability that a given case is disease ’x’, I believe that he is probably right when one physician’s area of specialization is different from another physician’s. However, I would suspect that physicians sharing the same area of specialization are more likely to provide convergent rather than divergent pretest estimates that a given patient with a given set of signs and symptoms is a representation of disease ’x’.

I’d like to add to this discussion the fact that cognitive models of the factors contributing to diagnostic accuracy/error have been used to explore the relationship between a given case’s ‘typicality’ and the probability that a given case will be correctly diagnosed. The findings in this area of research have demonstrated that diagnostic performance (accuracy) is a function of a case’s typicality. That is, the more closely a given case both approximates the prototypical portrayal of the disease for which it is a representation, and, the degree to which the findings in the case at hand make it distinguishable from the closest competing disease’s prototype, the more likely that case will be correctly diagnosed.

Research directed at exploring possible relationships/correlations between: 1) objective, probabilistic estimates that a given case is a representation of a given disease class, 2) estimates of the degree to which that same case is a ‘typical’ representation of a given disease class, 3) subjective physician estimates (low, intermediate, high) that the same case is a representation of a given disease class, and 4) the diagnostic performance (accuracy) of a cadre of physicians against that same case (or set of cases), would be a very rich and potentially useful area for DDX research.

Regarding comments form Dr Newman-Toker’s and others interested in problem-specific workups and tools. If indeed diagnostic accuracy is a function of a case’s typicality (as suggested via cognitive sciences models), and, if typicality is expressible in terms of the degree to which a case with its particular constellation of signs and symptoms both approximates the prototypical portrayal of the disease for which it is a representation, and, the degree to which the findings in the case at hand make it distinguishable from the closest competing disease’s prototype, then it makes lots of sense to adopt decision support tools that ensure the pursuit and collection of those signs and symptoms associated with each of the common and important differentials for the problem at hand (i.e., problem-specific data gathering protocols). Such findings, in conjunction with objective and subjective information processing mechanisms (such as via artificial intelligence tools, neural nets, Bayesian and other cognitive science-based inferencing tools) would offer the clinician a variety of perspectives with which a leading, and list of rank-ordered alternative diagnosis could be offered at the bedside.

Attached is an article for those interested in a cognitive sciences based exploration of the relationship between case typicality and diagnostic performance.

Frank


Frank J Papa, DO, PhD
Professor, Medical Education and Emergency Medicine
Director, TCOM Academy of Medical Educators
Associate Dean, Curricular Design and Faculty Development
University of North Texas Health Science Center

From: Benbassat Jochanan [mailto:benbasat at JDC.ORG]
Sent: Thursday, June 26, 2014 1:05 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Dear Ehud,

How then do you propose identifying a competent diagnostician for whom every fever is of known origin?

Jochanan Benbassat MD
________________________________
From: Ehud Zamir [ezamir at UNIMELB.EDU.AU<mailto:ezamir at UNIMELB.EDU.AU>]
Sent: Thursday, June 26, 2014 10:43
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post
Dear Dr Jain
Pre test probability is determined subjectively by the doctor. What constitutes high pretest probability for one doctor with a high index of suspicion for a condition will be judged as low pretest probability by another. Therefore I would suggest that clinical competence and diagnostic skill are the solution, rather than over investigation of patients with low pretest probability. We should bear in mind that in the face of truly low pretest probability, even positive results do not push the post test probability very far, unless the test is diagnostic by itself. So I am not sure I agree with your statement that "An incresed awareness thatn a substantial proportion of patients with a given disease, about 10-15 percent, encountered by us  is likely to have low pretest probability." Perhaps the fact that these 15% are CONSIDERED low test probability is simply the root cause of the diagnostic error. It could be argued that a more competent diagnostician would not have regarded these as low PTP, and that the more competent the diagnostician, the more likely their "low pretest probability" judgement is to be a true negative.
It reminds me of the "fever of unknown origin" issue, to which my Professor of Medicine in medical school used to refer to by asking "unknown to whom?"...
Regards
Ehud Zamir


________________________________
From: Jain, Bimal P.,M.D. [BJAIN at PARTNERS.ORG<mailto:BJAIN at PARTNERS.ORG>]
Sent: Wednesday, 25 June 2014 9:58 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post
Dear Drs. Newman-Toker,Kohn,Gordon,

I have followed your discussions about early diagnosis of stroke in ER with great interest.  I would like to make the following comments about diagnosis in general which may have relevance to diagnosis of stroke.


1.       Since introduction of am probabilistic  model of diagnosis by Lesley and Lusted in 1959 (Science ’59), it has become customary to represent pretest certainty about a disease by its pretest probability.

        2.A pretest probability depends upon a number of independent factors such as symptoms, risk factors, patient’s age, sex etc. which together constitute a clinical presentation. Therefore, a prêt
            est. probability, like any other measure such as height or intelligence quotient, which depends upon a number of independent factors, will tend to be distributed normally in patients with a given
            ease encountered by us (Tao, Best Writing  in Mathematics 2013)

 3.This means most patients with a disease (68 percent) will have intermediate pretest probability (20-79 percent), a few (16 percent) will have low pretest probability (0-19 percent), and other few
             (16 percent) will have high pretest probability(80-100 percent).

         4. This trend towards normal distribution has been observed, for example, in the PIOPED study about diagnosis of pulmonary embolism(JAMA 1990), 67 percent of 252 patients with pulmonary
              Embolism had intermediate pretest probability.

           5. Diagnostic error in general has been found to occur in 10-15 percent patients (Graber 2013).
                In Newman- Toker’s fine study in Diagnosis too, missed diagnosis of stroke in ER was found to be about 13 percent.

          6. The closeness of these diagnostic error rates to the expected percentage of patients with low pretest probability seems to suggest that most if not all diagnostic errors occur in these  patients.

          7. A major cause of diagnostic error in these patients, I suggest, is erroneus interpretation of a low pretest probability which is considered to be minimal evidence for a disease which is ruled out wit
               hout testing in a given, individual patient.

           8. Its correct interpretation, I suggest, is as a distribution, which only indicates a few patients with a disease in a series of similar patients. It does not tell us anything at all about presence or absence
                Of a disease in a given patient.

           9. The presence or absence of a disease in any patient regardless of pretest probability can only be determined by a test result with a high likelihood ratio ( 10 or higher) or a low likelihood ratio
               (0.1 or lower) respectively (Jaeschke 2002).

           10. For widespread use, a test capable of generating such a result needs to be simple and inexpensive. An example of such a test is EKG, which is performed in practically every patient with chest
                 pain seen in ER for evaluation of acute myocardial infarction.

           11. HINTS appears to be such a test for evaluating for stroke in patients with dizziness seen in ER, as suggested by Newman-Toker.

           12.In conclusion, I believe, the following measures could help minimise diagnostic errors.


(a)    An incresed awareness thatn a substantial proportion of patients with a given disease, about 10-15 percent, encountered by us  is likeky to have low pretest probability.

(b)   A disease cannot be ruled out purely from its low pretest probability.

(c)    In a given patient with low pretest probability, a disease can only be ruled out if a test result with low likelihood ratio (0.1 or lower) is observed.







Bimal P Jain MD

Pulmonary-Critical Care

Northshore Medical Center

Lynn, MA 01904





From: David Newman-Toker [mailto:toker at JHU.EDU]
Sent: Friday, June 20, 2014 10:50 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Thanks David. I’ve copied the ListServ because I think this sort of discussion might be interesting to others --- it is the messy real-world business of doing diagnosis in clinical practice! For those interested, see David G.’s excellent points in the trail below.

In response to each of your four points:

1a) OTTAWA SAH RULE IN PRINCIPLE --- Personally, I wouldn’t tap every patient over 20 with a new headache peaking in less than an hour – if they had a very compelling migraine story (e.g., classic visual aura), and it peaked progressively over 55 minutes (or anything over 30, probably), and they were in the correct age group for migraine onset (e.g., 15-40), and didn’t have a personal/family history of aneurysm/SAH, and had none of the dangerous Ottawa SAH rule features, I wouldn’t even CT them.  Nevertheless… I totally understand your perspective, and doing CT-LP in all of the patients in their series (using their entry criteria rather than their final rule) might be slightly simpler than following their rule; it would, however, increase the fraction of headache patients who got (presumably unnecessary) CT/LP by ~15%, which, back of the napkin, is probably at least 30,000 excess CTs a year in the US at a cost of about $10M/year… it may be a drop in the healthcare bucket, but, for that amount, we could do some really nice diagnostic research to refine decision rules to increase performance, usability, and buy in. ☺

1b) OTTAWA SAH RULE IN PRACTICE --- I think there is a wider evidence-practice gap in average community ED practice than might be imagined… only 2% of US headache patients undergo an LP (Goldstein, Cephalalgia, 2006) --- that includes all the suspected meningitis and SAH cases; I think most people believe that at least 2% of the total (probably more) have one or the other (meningitis or SAH) as a cause… so 2% is probably a lot fewer LPs than we should be doing, if we consider the asymmetric risk associated with LP vs. missed meningitis/SAH. So I would venture a guess that the average community ED physician is not being as thorough about looking for missed SAH as you are being in your practice… unfortunately, probably none of them are reading this ListServ to benefit from your thoughtful perspective.

2) HINTS --- Agree it is operator dependent, and some of this may go away when devices become more ubiquitous… BUT… how should we respond --- knowingly miss 35% of all the strokes using a bad decision-making approach that is standard practice… or seek out training to learn to do HINTS properly? Maybe ‘ok’ HINTS is still better than ‘great’ ABCD2/vascular risk stratification?

3) WHEN TO APPLY DECISION RULES --- This is an under-discussed but critical problem; one I spoke about in my commentary about JJ Perry’s SAH decision rule; but these rules are developed with fairly strict entry criteria, so I think that applying them in practice is mostly about pattern matching to the study methods in the paper (which, unfortunately, is probably rarely done in practice). I agree it can be tough, though, even if you try hard. I was giving grand rounds at Cornell earlier this week, and they took me to see an acute patient in the ED with dizziness --- it took some skill just to know whether HINTS should be applied or not… and you probably couldn’t have acquired that skill simply by reading the article. I think the co-symptoms issue is less problematic --- whatever the allowable co-symptoms were in the study are what’s relevant, and the determination is made based on the patient’s chief symptom/complaint --- I realize that this is not a perfectly reliable measure (and we have done studies that prove that inter-observer variation is more common than you’d like), but it is certainly a familiar enough one to all physicians… and, until computers are taking all of our histories from patients for us, it will likely remain part of our ‘art.’

4) ISCHEMIC STROKE MECHANISMS IN THE YOUNG --- I believe that we are getting fatter and more diabetic at a younger age, but, honestly, I’m not worried about strokes being missed in patients who are 35 that have HTN, DM, high cholesterol, chronic renal insufficiency, peripheral vascular disease, and a history of 2 prior MIs. No one will ignore all that just because of age. I remember a normal weight guy with no PMH who was 35 and presented with episodic blurred vision and confusion… he was sent home as suspected migraine… came back with turned out to be the proband for a family with undiagnosed familial hypercholesterolemia. So there will be some cases where stroke is the index event that discloses a patient’s (previously unknown) vascular risk and are tricky (esp. those who are non-obese)… but most of the ones we miss will likely be dissections, cardiac emboli, and ‘cryptogenic’ --- I think we should focus our attention on solving the diagnostic problems for those patients.

David


David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: David Gordon, M.D. [mailto:davidc.gordon at duke.edu]
Sent: Thursday, June 19, 2014 5:03 PM
To: David Newman-Toker
Subject: RE: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Hi David,

Thanks for highlighting these articles. So some specific and then general comments:


1)      Ottawa SAH rule--- This is interesting because I have to say, while this rule is a good teaching tool for highlighting the red flags of headache, it would not personally impact my practice pattern.  If I have anyone 20 or older ( the rule uses 15) with “new severe nontraumatic headache reaching maximum intensity within 1 hour” that alone is enough for me to cross over the diagnostic threshold  for ruling out SAH.  The other variables in the rule don’t mean much at that point for testing purposes. I would venture to say most emergency physicians would acknowledge this as the culture of their training. So in the patients with missed SAH in your recent study, it would be very interesting to know the specifics of their presentation. Did they have an atypical presentation for SAH that not even the rule would capture or would this rule if employed by the physician as a diagnostic aid have appropriately steered them towards work-up they neglected to pursue. Need to see this rule studied prospectively.

2)      HINTS- compelling CDR but I would venture to say limited by operator dependence—at least until the devices are widely available and employed.  I personally would not feel confident in my current proficiency in assessing eye movements. You alluded to this in the article, but it seems this has been employed mainly by specialists to date. Would the sensitivity change in the hands of less experienced practitioners? As you also alluded to, used indiscriminately, this CDR runs the risk of MRI overuse given that a normal physiologic response is a bad sign in this tool.  So I still think there is more studies to be done before recommending for general use.

3)      This has been echoed before, but applying such rules prospectively in an undifferentiated population could be challenging owing to overlapping spectrum of symptoms. Patients come in with headache alone, headache + dizziness, headache and speech disturbance, no headache and speech disturbance?  Which rule is one to apply?

4)      This discussion of missing strokes in the young covers a broad array of potential etiologies. What exactly is the pathophysiology being missed here? Are we seeing accelerated atheroembolic disease in young patients due to HTN and DM? Or do young people represent a separate physiology from older patients with stroke owing to vertebral dissections and venothrombotic events?  I think the more we understand the pathophysiology at play here, the better we can advise clinicians to either adjust their age threshold for the development of atheroembolic disease or to make sure to consider these alternate disease processes for stroke-like symptoms in the young.

Thanks,
David G


David Gordon, MD
Associate Professor
Undergraduate Education Director
Division of Emergency Medicine
Duke University

From: David Newman-Toker [mailto:toker at jhu.edu]
Sent: Thursday, June 19, 2014 7:37 AM
To: David Gordon, M.D.; Society to Improve Diagnosis in Medicine
Subject: RE: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

Thanks David. So I gather you think that these two CDRs below addressing stroke diagnosis in patients with headache and dizziness, respectively, are lacking some combination of good performance, ease of use, or buy in? (“rule has good performance, easy to use, and is bought into by both emergency physicians and neurologists”)

Perry JJ, Stiell IG, Sivilotti ML, et al. Clinical decision rules to rule out subarachnoid hemorrhage for acute headache. JAMA : the journal of the American Medical Association 2013;310:1248-55.

Newman-Toker DE, Kerber KA, Hsieh YH, et al. HINTS Outperforms ABCD2 to Screen for Stroke in Acute Continuous Vertigo and Dizziness. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2013;20:986-96.



David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: David Gordon, M.D. [mailto:davidc.gordon at duke.edu]
Sent: Wednesday, June 18, 2014 9:40 AM
To: Society to Improve Diagnosis in Medicine; David Newman-Toker
Subject: RE: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post

David,

A real challenge here is trying to separate the signal from the noise, and when it comes to neurologic complaints, there is unfortunately a lot of noise in emergency departments. Overcrowding and financial pressures further compound the difficulty of who requires the full work-up.

I think risk stratification is key to this issue. We have imperfect but overall good processes and tools in place for the risk stratification of ACS and pulmonary embolism. As an emergency physician, I don't feel I have the same cognitive tools available for independently risk stratifying TIA/stroke. I am fortunate to work in a clinical environment where I have ready access to neurology consultation to assist in the process and an observation protocol for equivocal/intermediate cases, but I gather to say this is far from the norm.

As far as the treatment of neurologic complaints in the emergency setting, we need more evidence. It is going to take prospective analysis of all-comers to the ED with stroke-like symptoms to better understand who needs immediate work-up and who can be safely discharged. Perhaps we will end up with 2 different stratification tools- one for the young and one for the old.

As far as whether diagnostic aids will be utilized or ignored due to CDRs, I think it depends. If the rule has good performance, easy to use, and is bought into by both emergency physicians and neurologists, I do think it would be readily employed - especially if the evidence becomes increasingly convincing that the epidemiology of stroke is changing (or becoming better understood) and young patient's are being misdiagnosed.

-David

David Gordon, MD
Associate Professor
Undergraduate Education Director
Division of Emergency Medicine
Duke University

The information in this electronic mail is sensitive, protected information intended only for the addressee(s). Any other person, including anyone who believes he/she might have received it due to an addressing error, is requested to notify the sender immediately by return electronic mail, and to delete it without further reading or retention. The information is not to be forwarded to or shared unless in compliance with Duke Medicine policies on confidentiality and/or with the approval of the sender.
________________________________
From: David Newman-Toker [toker at JHU.EDU<mailto:toker at JHU.EDU>]
Sent: Tuesday, June 17, 2014 2:56 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: [IMPROVEDX] stroke misdiagnosis disproportionate in the young says Washington Post
Stroke is a major public health problem, and recent work suggests young patients are having more strokes, with rates rising alarmingly in recent years, according to an article in today’s Washington Post…

http://www.washingtonpost.com/national/health-science/strokes-long-on-the-decline-among-the-elderly-are-rising-among-younger-adults/2014/06/16/f1f54538-e5d9-11e3-a86b-362fd5443d19_story.html

They are also much more likely to be misdiagnosed (7-fold greater risk in those 18-45 relative to those >75)…

http://www.degruyter.com/view/j/dx.2014.1.issue-2/dx-2013-0038/dx-2013-0038.xml

Thoughts?

David


David E. Newman-Toker, MD, PhD
Associate Professor, Department of Neurology
Johns Hopkins Hospital, Meyer 8-154; 600 North Wolfe Street, Baltimore, MD 21287
Email: toker at jhu.edu<mailto:toker at jhu.edu>; 410-502-6270 (phone); 410-502-6265 (fax)
Web address: Johns Hopkins Neurology (David Newman-Toker)<http://www.hopkinsmedicine.org/neurology_neurosurgery/specialty_areas/vestibular/profiles/team_member_profile/516F40C024FCA3D4B4B633D0E080FE1B/David_Newman-Toker>


Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.


________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1

or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX


Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/



________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com<mailto:Lorri at ZPM1.com>, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/


________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged.  If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited.  Please reply to the sender that you have received the message in error, then delete it.  Thank you.
________________________________

To unsubscribe from the IMPROVEDX:
mail to:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
or click the following link: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG

For additional information and subscription commands, visit:
http://www.lsoft.com/resources/faq.asp#4A

http://LIST.IMPROVEDIAGNOSIS.ORG/ (with your password)

Visit the searchable archives or adjust your subscription at:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br>
<a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1" target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1</a>
</p>

HTML Version:
URL: <../attachments/20140710/9296a70b/attachment.html>


More information about the Test mailing list