Sens/spec, +/-LRs, PPV/NPVs

Jain, Bimal P.,M.D. BJAIN at PARTNERS.ORG
Wed Sep 12 12:07:19 UTC 2018


Hi David,

Your point about rule in power of positive Likelihood Ratio ( PLR )and rule out power of negative Likelihood Ratio ( NLR) is excellent. This power arises, as you mention in your JAMA editorial from the fact that LR represents an increase or decrease of odds of a disease in a given patient, as we see from the odds form of Bayes' theorem, Likelihood ratio = Posterior odds/Prior odds. In general, a PLR greater than 10 of a test result is employed to rule in a disease in practice. For example, acute MI is ruled in with ST elevation EKG changes with PLR of 13 in a patient.
The other point of interest is that the change in odds represented by a LR is independent of prior odds or prior probability, so that it represents the same rule in power in all patients regardless of prior probability. Is this the reason that ST elevation EKG changes, for example, are employed to rule in acute MI in patients with a broad range of prior probabilities?

Regards

Bimal

Bimal P Jain MD
Northshore Medical Center
Lynn MA 01907.


From: David Newman-Toker <toker at JHU.EDU>
Sent: Tuesday, September 11, 2018 10:00 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Subject: Re: [IMPROVEDX] Sens/spec, +/-LRs, PPV/NPVs


        External Email - Use Caution
Dear Tom,

I agree with Harold's point about thresholds.

I would be careful about equating a low NLR with a high sensitivity (or a high PLR with a high specificity). The NLR and PLR are mathematical transformations of sensitivity AND specificity, taken together. You cannot equate a likelihood ratio with either sensitivity or specificity alone. You can find the formulas relating them on the internet (https://en.wikipedia.org/wiki/Likelihood_ratios_in_diagnostic_testing, https://en.wikipedia.org/wiki/Sensitivity_and_specificity). I have pasted a screen shot below.

At the end of the day, the NLR is the "rule out power" of the test and the PLR is the "rule in power" of the test. The sensitivity is only a good proxy for NLR when it is truly 100% --- which, in practice, is never. The specificity is only a good proxy for PLR when it is truly 100% --- which is also never.

This has very real implications for the average practicing clinician. The Ottawa SAH rule is a perfect example of a very high-sensitivity test (estimated ~100% with lower 95% confidence bound 97.2%) with a relatively modest NLR (upper confidence bound 0.39), because of a very low specificity (~15%). See my editorial with Jonathan Edlow on this subject (attached, PMID: 24065009).

Best,
David

[cid:image002.jpg at 01D44A6C.ABB27630]


David E. Newman-Toker, MD PhD
Professor of Neurology, Ophthalmology, & Otolaryngology<http://www.hopkinsmedicine.org/profiles/results/directory/profile/0015937/david-newman-toker>
Director, Division of Neuro-Visual & Vestibular Disorders<http://www.hopkinsmedicine.org/neurology_neurosurgery/centers_clinics/vestibular/team/>
Director, Armstrong Institute Center for Diagnostic Excellence<http://www.hopkinsmedicine.org/armstrong_institute/center_for_diagnostic_excellence/>
Core Faculty, Brain Injury OutcomeS (BIOS) Clinical Trials Unit<http://braininjuryoutcomes.com/>
President-Elect, Society to Improve Diagnosis in Medicine<http://www.improvediagnosis.org/?page=BoardMembers>

Johns Hopkins University School of Medicine
Johns Hopkins Hospital; Pathology Building 2-221
600 North Wolfe Street; Baltimore, MD 21287-6921

Administrator: Myriha Montgomery (410) 955-2536; mmontg20 at jhmi.edu<mailto:mmontg20 at jhmi.edu>

Confidentiality Notice: The information contained in this email is intended for the confidential use of the above named recipient. If the reader of this message is not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this in error, please notify the sender immediately by telephone at the number set forth above and destroy this email message. Thank you.

From: Harold Lehmann <lehmann at JHMI.EDU<mailto:lehmann at JHMI.EDU>>
Sent: Tuesday, September 11, 2018 9:00 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] Sens/spec, +/-LRs, PPV/NPVs

Please don't forget to discuss thresholds. Prevalence, along with LR+ or LR-, gives positive/negative PV (=posterior)), don't mean anything without an honest discussion of thresholds (whether it's the threshold for "pay attention" or for "test" or for "treat" (without testing). For instance, I asked residents for 20 years, what's their lowest absolute probability of bacterial meningitis for getting an LP on a one year home with a fever (or the highest probability to send the infant home), and the mode was 1/1,000, with a range of 1/1,000,000 (this resident was a lawyer before she was a doctor) and 1/10 (we educated this resident that she was wrong, using the other residents' responses as data).

I know there has been research done on the failure of thresholds for explaining variability in performance, but it's got to be useful in communication. (Needs to be tested. Still. Or have I missed the paper?)

Harold

On Sep 11, 2018, at 6:52 PM, Tom Yuen <0000001243181998-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG<mailto:0000001243181998-dmarc-request at LIST.IMPROVEDIAGNOSIS.ORG>> wrote:

Many thanks to everyone's reply, especially from Dr Brush and John Ely for his thoughtful response.

I think the bottom line that my residents needed to answer was basically if there was any difference and/or advantage between sensitivity/specificity and +LR/-LR.  PPV/NPV, as John (Ely) pointed out is dependent on disease prevalence.

But it seems to me, based on everyone's reply is that there doesn't seem to be much PRACTICAL difference between sens/spec and LR.  Just different ways of expressing essentially the same thing. (if I have this horribly wrong please correct me as I am giving a followup lecture Thursday and will be addressing the questions they have last week!)

Essentially a test with a high +LR and a high specificity- if positive in a patient with a moderate pretest prob of a disease, dramatically increases your posttest prob of them truly having the disease.  I think my residents were struggling (as was I) with whether it mattered which statistic (LR vs. spec/sens) we looked at- and it doesn't seem to matter.

Thank you again everyone, while I am a first-time poster I have been following this listserv for years and am humbled by the collective wisdom assembled here.

Tom Yuen, MD

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1

or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX


Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/



________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/

________________________________

Address messages to: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>

To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG<mailto:IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG>

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.






Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine


HTML Version:
URL: <../attachments/20180912/bbfeb2ad/attachment.html> ATTACHMENT:
Name: image002.jpg Type: image/jpeg Size: 27032 bytes Desc: image002.jpg URL: <../attachments/20180912/bbfeb2ad/attachment.jpg>


More information about the Test mailing list