Sens/spec, +/-LRs, PPV/NPVs

John Brush jebrush at ME.COM
Wed Sep 12 14:38:30 UTC 2018


	I also agree with the point about thresholds. Thinking about likelihood ratios can lead you to start to think more quantitatively about thresholds. For example, what is the threshold for which you would feel comfortable discharging a patient from the ED with chest pain? Let’s say it is 0.05 or 5%. Using a test with 80% sensitivity and 80% specificity (NLR=0.25), you would have to have a pretest probability of less than 18% to get to a posttest probability that low. It makes no sense to do an EKG stress test on that patient if you think the pretest probability is higher. You would need a test with greater sensitivity than 80%, which is justification for doing imaging stress tests in that setting.
	In your editorial, the positive LR for the Ottawa SAH test is only 1.18, which is so low that you almost can’t believe a positive test. Someone with a pretest probability of 50% would only have a posttest probability of 54% with a positive test. The LR allows you to see how weak a positive test result is. Also, I am skeptical about a test with a reported sensitivity of 100%. It makes you question whether the gold standard test found everyone with disease.
	Your editorial points to what I think is a frequent problem with ED testing. Using tests to screen patients as a “rule out” decimates the specificity and positive likelihood ratio, creating a lot of false positive tests and misleading results. Again, thinking about this more quantitatively brings all of these issues to the forefront. 
	
John Brush




On Sep 11, 2018, at 9:59 PM, David Newman-Toker <toker at JHU.EDU> wrote:

Dear Tom,
 
I agree with Harold’s point about thresholds.
 
I would be careful about equating a low NLR with a high sensitivity (or a high PLR with a high specificity). The NLR and PLR are mathematical transformations of sensitivity AND specificity, taken together. You cannot equate a likelihood ratio with either sensitivity or specificity alone. You can find the formulas relating them on the internet (https://en.wikipedia.org/wiki/Likelihood_ratios_in_diagnostic_testing <https://en.wikipedia.org/wiki/Likelihood_ratios_in_diagnostic_testing>, https://en.wikipedia.org/wiki/Sensitivity_and_specificity <https://en.wikipedia.org/wiki/Sensitivity_and_specificity>). I have pasted a screen shot below.
 
At the end of the day, the NLR is the “rule out power” of the test and the PLR is the “rule in power” of the test. The sensitivity is only a good proxy for NLR when it is truly 100% --- which, in practice, is never. The specificity is only a good proxy for PLR when it is truly 100% --- which is also never.
 
This has very real implications for the average practicing clinician. The Ottawa SAH rule is a perfect example of a very high-sensitivity test (estimated ~100% with lower 95% confidence bound 97.2%) with a relatively modest NLR (upper confidence bound 0.39), because of a very low specificity (~15%). See my editorial with Jonathan Edlow on this subject (attached, PMID: 24065009).
 
Best,
David
 
<image003.jpg>
 
 
David E. Newman-Toker, MD PhD
Professor of Neurology, Ophthalmology, & Otolaryngology <http://www.hopkinsmedicine.org/profiles/results/directory/profile/0015937/david-newman-toker>
Director, Division of Neuro-Visual & Vestibular Disorders <http://www.hopkinsmedicine.org/neurology_neurosurgery/centers_clinics/vestibular/team/>
Director, Armstrong Institute Center for Diagnostic Excellence <http://www.hopkinsmedicine.org/armstrong_institute/center_for_diagnostic_excellence/>
Core Faculty, Brain Injury OutcomeS (BIOS) Clinical Trials Unit <http://braininjuryoutcomes.com/>
President-Elect, Society to Improve Diagnosis in Medicine <http://www.improvediagnosis.org/?page=BoardMembers>
 
Johns Hopkins University School of Medicine
Johns Hopkins Hospital; Pathology Building 2-221
600 North Wolfe Street; Baltimore, MD 21287-6921
 
Administrator: Myriha Montgomery (410) 955-2536; mmontg20 at jhmi.edu <mailto:mmontg20 at jhmi.edu>

Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.
 
From: Harold Lehmann <lehmann at JHMI.EDU <mailto:lehmann at JHMI.EDU>> 
Sent: Tuesday, September 11, 2018 9:00 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] Sens/spec, +/-LRs, PPV/NPVs
 
Please don't forget to discuss thresholds. Prevalence, along with LR+ or LR-, gives positive/negative PV (=posterior)), don't mean anything without an honest discussion of thresholds (whether it's the threshold for "pay attention" or for "test" or for "treat" (without testing). For instance, I asked residents for 20 years, what's their lowest absolute probability of bacterial meningitis for getting an LP on a one year home with a fever (or the highest probability to send the infant home), and the mode was 1/1,000, with a range of 1/1,000,000 (this resident was a lawyer before she was a doctor) and 1/10 (we educated this resident that she was wrong, using the other residents' responses as data).
 
I know there has been research done on the failure of thresholds for explaining variability in performance, but it's got to be useful in communication. (Needs to be tested. Still. Or have I missed the paper?)
 
Harold


On Sep 11, 2018, at 6:52 PM, Tom Yuen <0000001243181998-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG <mailto:0000001243181998-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG>> wrote:
 
Many thanks to everyone's reply, especially from Dr Brush and John Ely for his thoughtful response.
 
I think the bottom line that my residents needed to answer was basically if there was any difference and/or advantage between sensitivity/specificity and +LR/-LR.  PPV/NPV, as John (Ely) pointed out is dependent on disease prevalence. 
 
But it seems to me, based on everyone's reply is that there doesn't seem to be much PRACTICAL difference between sens/spec and LR.  Just different ways of expressing essentially the same thing. (if I have this horribly wrong please correct me as I am giving a followup lecture Thursday and will be addressing the questions they have last week!)
 
Essentially a test with a high +LR and a high specificity- if positive in a patient with a moderate pretest prob of a disease, dramatically increases your posttest prob of them truly having the disease.  I think my residents were struggling (as was I) with whether it mattered which statistic (LR vs. spec/sens) we looked at- and it doesn't seem to matter.

Thank you again everyone, while I am a first-time poster I have been following this listserv for years and am humbled by the collective wisdom assembled here.

Tom Yuen, MD
 

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1>
 
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX>
 

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/ <http://www.improvediagnosis.org/>
 
 
 

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1>
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX>

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/ <http://www.improvediagnosis.org/>

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1>

or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG <mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX>


Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/ <http://www.improvediagnosis.org/>

<Ed_Newman-Toker_High-Stakes-Diagnostic-Decision-Rules_JAMA-2013.pdf>







Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine


HTML Version:
URL: <../attachments/20180912/8d703691/attachment.html>


More information about the Test mailing list