Sens/spec, +/-LRs, PPV/NPVs

Mark Graber Mark.Graber at IMPROVEDIAGNOSIS.ORG
Tue Sep 11 20:58:11 UTC 2018


Having encountered the same eye-rolling when trying to explain Bayesian predictions, I’ve sometimes resorted to a simple visual illustration to make the point that you can’t just look at the test result, you have to understand the test characteristics, the power of the test itself, and how this applies to your patient’s problem.

This was a patient who had a right nephrectomy performed about a year earlier.  He came to clinic saying he was doing well and felt fine, but noticed a new, enlarging mass on the right side of his abdomen.  You can see the bulge in the picture.
[cid:image001.png at 01D449F0.9E9E7330]

We were worried he had a cancer recurrence, and someone else raised the possibility that this was a benign post-op hygroma related to the surgery.  So we ordered a CT scan, which came back……… NORMAL, except for the absent right kidney.  So that is the test result - is the patient truly normal???  Scan down for the diagnosis and explanation……..









The right sided mass is bowel, herniated through the surgical scar from his nephrectomy.  It is plainly visible when he stands up but self-reduces when he lies down, and of course the CT was done in the recumbent position.  For accurate diagnosis, you really need to know the characteristics of the test.

Mark

Mark L Graber MD FACP
President, SIDM
Senior Fellow, RTI International
Professor Emeritus, Stony Brook University
[cid:image002.jpg at 01D449F0.9E9E7330]


From: John Brush <0000001122b4be5c-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG>
Reply-To: Listserv ImproveDx <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, John Brush <jebrush at ME.COM>
Date: Tuesday, September 11, 2018 at 9:05 AM
To: Listserv ImproveDx <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] Sens/spec, +/-LRs, PPV/NPVs

Dr. Yuen,
I would teach your learners that you just have to be a little mathematical in practice. There’s no getting around it! Science is a quantitative discipline and if you want to practice medicine based on science, you have to apply just a little math. In my book, I tried to make it a simple as possible so the translation of quantitative science to semi-quantitative practice would become intuitive.
The fundamental question that I tried to answer in my book is this: How do you apply the science of medicine (which is derived from populations of patients) to an individual patient? You can do this, but it takes some mathematical thinking and takes some practice. When we learned mathematics as school kids, we learned by practicing problems until the concepts finally sunk in. You need to do this with likelihood ratios as well. If you play around with them a bit, you start to see how the strength of a test is well represented by the likelihood ratio number. You start to know intuitively that a likelihood ratio of 5 is pretty amazing. 10 is really compelling. ST elevation in the proper clinical setting is so compelling that we activate a cath lab based on that test alone. It has a LR(+) of about 13. The point of statistical inference is that numbers can have meaning. For likelihood ratios, learners need to understand what the numbers mean.
LRs are calculated directly from the sensitivity and specificity. Any test with a known sensitivity or specificity has a LR that can be easily calculated. LR(+) is interesting because it rises dramatically when specificity rises above 80%, as shown below. If you play around with probability, odds, and likelihood ratios, you start to get a sense of this, but you have to play around with the numbers a bit to become facile in converting probability to odds, multiply by the LR, then convert posttest odds back to probability. The point of the exercise is ultimately to develop an intuitive sense of how much emphasis you should put on a test result, depending on the test's likelihood ratio.
When your learners start to think this way, they start to appreciate that you have to think about baseline risk (based on an estimate of prevalence of disease), the strength of test results, and thresholds of posttest probability that will become your cutoff for decision-making. You start to think in ways that avoid base-rate neglect, anchoring, and other ways of jumping to conclusions. Your diagnostic decision making becomes more organized and precise. But it takes practice, just like anything else.
[cid:A4E6E11F-C7D6-41BD-A6CB-60E42FD3A632]
John E. Brush, Jr., M.D., FACC
Professor of Medicine
Eastern Virginia Medical School
Sentara Cardiology Specialists
844 Kempsville Road, Suite 204
Norfolk, VA 23502
757-261-0700
Cell: 757-477-1990
jebrush at me.com<mailto:jebrush at me.com>



On Sep 10, 2018, at 7:26 AM, David Meyers <dm0015 at COMCAST.NET<mailto:dm0015 at COMCAST.NET>> wrote:

Dr Yuen writes again:


“Not sure if this was posted-repost:


Thanks for the suggestion-including Dr. Brush's direct reply,


However, I had a copy of the book from a few years ago and read it several times before posting.  In particular I combed through chapter 4 ("Decision Making: Making Choices) - extremely well written and helpful - just not for the specific questions I (and my residents) had. In the book John makes the excellent point of how pre-test probably affects our interpretation of test results and the role of sensitivity and specificity and introduces likelihood ratios and Bayesian reasoning-all concepts I taught during my lecture (citing his book of course!)

However, he doesn't specifically address the following questions and I can't seem to find an answer to [them].  Again I understand the mathematical differences, but I am just questioning the practical differences.  For example, specifically:


1. Are there any tests for which specificity is NOT directly related to a positive likelihood ratio (+LR) and positive predictive value (PPV)?  If not, then a very specific test ROUGHLY means a test with a high PPV, ROUGHLY means high +LR.  In my residents mind (and my own actually) then, there really isn't any difference in knowing the subtle nuances between those numbers-a positive test would mean that the patient very likely has the condition tested for regardless of which statistic you looked up. And yes, if you use exact numbers there certainly are numeric differences in the actual probabilities, but for a busy clinician it all "means about the same thing".


2. Are there instances when using Sensitivity/specificity vs LR vs PPV/NPV results in an incorrect conclusion?  In other words, are there specific cases when interpreting a test result using a test's published sensitivity would lead you to a different conclusion than if you looked up the -LR? Again if not, for the typical learner- I understand why it seems like a "waste of time" trying to figure out the differences between all these terms if they roughly lead one to the same conclusion.


3. When would you preferentially use specificity vs. LR vs PPV?  If the resident has a patient with a moderate pretest clinical probability of acute myocardial infarction (AMI) and the high sensitivity troponin is negative- does it really matter if they refer to the test's LR vs NPV vs sensitivity?  They all seem to lead to the same clinical conclusion?


I apologize if I am misunderstanding an elementary concept and these seem like silly questions - but as I am asked these questions I really didn't have any good answer and I don't feel any closer to a satisfactory answer than 2 days ago. Again I can point out the statistical and mathematical differences in calculating these values- but to me they all seem to "tell the same story"

Your thoughts appreciated
Tom”


________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1

or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX


Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/



________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

To unsubscribe from the IMPROVEDX:
mail to:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG
or click the following link: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG

For additional information and subscription commands, visit:
http://www.lsoft.com/resources/faq.asp#4A

http://LIST.IMPROVEDIAGNOSIS.ORG/ (with your password)

Visit the searchable archives or adjust your subscription at:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br>
<a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1" target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1</a>
</p>

HTML Version:
URL: <../attachments/20180911/d5851ee7/attachment.html> ATTACHMENT:
Name: image001.png Type: image/png Size: 613419 bytes Desc: image001.png URL: <../attachments/20180911/d5851ee7/attachment.png> ATTACHMENT:
Name: image002.jpg Type: image/jpeg Size: 48106 bytes Desc: image002.jpg URL: <../attachments/20180911/d5851ee7/attachment.jpg> ATTACHMENT:
Name: image003.png Type: image/png Size: 107698 bytes Desc: image003.png URL: <../attachments/20180911/d5851ee7/attachment-0001.png> ATTACHMENT:
Name: Right sided mass.pdf Type: application/pdf Size: 83075 bytes Desc: Right sided mass.pdf URL: <../attachments/20180911/d5851ee7/attachment.pdf>


More information about the Test mailing list