Sens/spec, +/-LRs, PPV/NPVs

David Newman-Toker toker at JHU.EDU
Wed Sep 12 01:59:35 UTC 2018


Dear Tom,

I agree with Harold's point about thresholds.

I would be careful about equating a low NLR with a high sensitivity (or a high PLR with a high specificity). The NLR and PLR are mathematical transformations of sensitivity AND specificity, taken together. You cannot equate a likelihood ratio with either sensitivity or specificity alone. You can find the formulas relating them on the internet (https://en.wikipedia.org/wiki/Likelihood_ratios_in_diagnostic_testing, https://en.wikipedia.org/wiki/Sensitivity_and_specificity). I have pasted a screen shot below.

At the end of the day, the NLR is the "rule out power" of the test and the PLR is the "rule in power" of the test. The sensitivity is only a good proxy for NLR when it is truly 100% --- which, in practice, is never. The specificity is only a good proxy for PLR when it is truly 100% --- which is also never.

This has very real implications for the average practicing clinician. The Ottawa SAH rule is a perfect example of a very high-sensitivity test (estimated ~100% with lower 95% confidence bound 97.2%) with a relatively modest NLR (upper confidence bound 0.39), because of a very low specificity (~15%). See my editorial with Jonathan Edlow on this subject (attached, PMID: 24065009).

Best,
David

[cid:image003.jpg at 01D44A1A.B78FB280]


David E. Newman-Toker, MD PhD
Professor of Neurology, Ophthalmology, & Otolaryngology<http://www.hopkinsmedicine.org/profiles/results/directory/profile/0015937/david-newman-toker>
Director, Division of Neuro-Visual & Vestibular Disorders<http://www.hopkinsmedicine.org/neurology_neurosurgery/centers_clinics/vestibular/team/>
Director, Armstrong Institute Center for Diagnostic Excellence<http://www.hopkinsmedicine.org/armstrong_institute/center_for_diagnostic_excellence/>
Core Faculty, Brain Injury OutcomeS (BIOS) Clinical Trials Unit<http://braininjuryoutcomes.com/>
President-Elect, Society to Improve Diagnosis in Medicine<http://www.improvediagnosis.org/?page=BoardMembers>

Johns Hopkins University School of Medicine
Johns Hopkins Hospital; Pathology Building 2-221
600 North Wolfe Street; Baltimore, MD 21287-6921

Administrator: Myriha Montgomery (410) 955-2536; mmontg20 at jhmi.edu<mailto:mmontg20 at jhmi.edu>

Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: Harold Lehmann <lehmann at JHMI.EDU>
Sent: Tuesday, September 11, 2018 9:00 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Subject: Re: [IMPROVEDX] Sens/spec, +/-LRs, PPV/NPVs

Please don't forget to discuss thresholds. Prevalence, along with LR+ or LR-, gives positive/negative PV (=posterior)), don't mean anything without an honest discussion of thresholds (whether it's the threshold for "pay attention" or for "test" or for "treat" (without testing). For instance, I asked residents for 20 years, what's their lowest absolute probability of bacterial meningitis for getting an LP on a one year home with a fever (or the highest probability to send the infant home), and the mode was 1/1,000, with a range of 1/1,000,000 (this resident was a lawyer before she was a doctor) and 1/10 (we educated this resident that she was wrong, using the other residents' responses as data).

I know there has been research done on the failure of thresholds for explaining variability in performance, but it's got to be useful in communication. (Needs to be tested. Still. Or have I missed the paper?)

Harold


On Sep 11, 2018, at 6:52 PM, Tom Yuen <0000001243181998-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG<mailto:0000001243181998-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG>> wrote:

Many thanks to everyone's reply, especially from Dr Brush and John Ely for his thoughtful response.

I think the bottom line that my residents needed to answer was basically if there was any difference and/or advantage between sensitivity/specificity and +LR/-LR.  PPV/NPV, as John (Ely) pointed out is dependent on disease prevalence.

But it seems to me, based on everyone's reply is that there doesn't seem to be much PRACTICAL difference between sens/spec and LR.  Just different ways of expressing essentially the same thing. (if I have this horribly wrong please correct me as I am giving a followup lecture Thursday and will be addressing the questions they have last week!)

Essentially a test with a high +LR and a high specificity- if positive in a patient with a moderate pretest prob of a disease, dramatically increases your posttest prob of them truly having the disease.  I think my residents were struggling (as was I) with whether it mattered which statistic (LR vs. spec/sens) we looked at- and it doesn't seem to matter.

Thank you again everyone, while I am a first-time poster I have been following this listserv for years and am humbled by the collective wisdom assembled here.

Tom Yuen, MD

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1

or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX


Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/



________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/






Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine


HTML Version:
URL: <../attachments/20180912/23aa0879/attachment.html> ATTACHMENT:
Name: image003.jpg Type: image/jpeg Size: 122274 bytes Desc: image003.jpg URL: <../attachments/20180912/23aa0879/attachment.jpg> ATTACHMENT:
Name: Ed_Newman-Toker_High-Stakes-Diagnostic-Decision-Rules_JAMA-2013.pdf Type: application/pdf Size: 117769 bytes Desc: Ed_Newman-Toker_High-Stakes-Diagnostic-Decision-Rules_JAMA-2013.pdf URL: <../attachments/20180912/23aa0879/attachment.pdf>


More information about the Test mailing list