[EXTERNAL] [IMPROVEDX] Bayesian diagnosis & free text vs coded data

Eta S Berner eberner at UAB.EDU
Mon Jul 21 16:07:42 UTC 2014


A main purpose of the EHR is to provide documentation and communication of key findings and the clinician's assessment of the meaning of the findings.  I would assume that the story as it unfolds, as Mark says, can assist the physician in reaching a diagnosis, but how much of that "story" is really needed for documentation and communication?

*********************************************
Eta S. Berner, Ed.D.
Professor, Health Informatics
Director, Center for Health Informatics for Patient Safety/Quality
Department of Health Services Administration
School of Health Professions
Professor, Department of Medical Education
School of Medicine
University of Alabama at Birmingham
1705 University Blvd. #590J
Birmingham, AL 35294
Phone: (205)975-8219
Fax:       (205)975-6608
Email:   eberner at uab.edu


-----Original Message-----
From: Graber, Mark [mailto:Mark.Graber at VA.GOV] 
Sent: Monday, July 21, 2014 9:23 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Subject: Re: [IMPROVEDX] [EXTERNAL] [IMPROVEDX] Bayesian diagnosis & free text vs coded data

Harold raises an issue that we ultimately will need to address:  Is diagnosis enhanced or degraded by a process like AHLTA's where data is captured from pull down lists, as opposed to using free-text entries.

We had this debate in the VA (where we use free text notes) when we were considering merging with AHLTA (all pull down lists).  If the pull down lists were somehow ever perfected and allowed more detailed or more accurate selections, I guess these could ultimately approximate free text.  Just seems to me that there is so much information in HOW a patient tells his\jher story, and in what order, and what gets associated with what.  All this rich data would be lost in pull down lists, where everything has to fit a prescribed bucket.  I'll also acknowledge that many of the free-text notes created these days are just horrible or frankly misleading, so there's a downside to both approaches.

A related question is whether using pull-down lists will facilitate research into diagnostic quality, or whether you can approximate this using text mining software to extract comparable data.

________________________________
From: Harold Lehmann <lehmann at jhmi.edu>
Reply-To: Society to Improve Diagnosis in Medicine <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, Harold Lehmann <lehmann at jhmi.edu>
Date: Sat, 19 Jul 2014 22:31:39 -0400
To: <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] [EXTERNAL] [IMPROVEDX] Bayesian diagnosis

If we ever get to have patient findings stored in an EHR in a retrievable way (like the military's AHLTA system does, but in a manner that physicians embrace), then those specific findings that the physician should be using to arrive at a prior would in fact turn that assessment into a true posterior probability. "Medicine in Denial," by Larry and Lincoln Weed makes this point by advocating the "structured" collection of patient data. (I don't think they advocate Bayesian reasoning, but the result is the same.)

Harold

From: "Michael.H.Kanter at KP.ORG" <Michael.H.Kanter at KP.ORG>
Reply-To: Society to Improve Diagnosis in Medicine <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, "Michael.H.Kanter at KP.ORG" <Michael.H.Kanter at KP.ORG>
Date: Saturday, July 19, 2014 at 8:23 PM
To: "IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG" <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: Re: [IMPROVEDX] [EXTERNAL] [IMPROVEDX] Bayesian diagnosis

In my view, part of the problem with Bayesian approaches is that of obtaining the prior probability.  Most examples attempt to use the prevalence of the disease in the population giving the appearance of it sounding scientific.  This is probably okay when screening asymptomatic people for certain diseases where the physician actually never sees individual patients but is instead looking at the population.   Once the patient is seen by the physician, the prior probability changes as not all patients are alike.
.  In true Bayesian statistics, the prior probability is a subjective probability. (this is also one reason why many statisticians do not feel Bayesian statistics are "scientific").  So, if a patient presents with certain symptoms, the prior probability should ideally be determined by the physician who uses all information about the patient other than the various tests/symptoms that are used to modify the prior probability to create a final conditional probability that the patient has a disease given certain symptoms or test results.  This then becomes an exercise in subjectivity where the physicians' intuition, prior experience, and "gut feel" may and should be used in determining prior probabilities.   Most advocates of a Bayesian approach simply select a prior probability based on a few variables and require that all physicians use that prior probability.  This is a fundamental flaw, in my view.
 In the example below (item 6) where Bayesian approaches may be problematic due to failure to consider a rare disease, this  failure to consider a rare disease can be viewed as simply using the wrong prior probability for the more common disease.  Physicians who consider the rare disease would have a much lower prior probability of the non rare disease than those who do not consider the rare disease.
Thus, the concept that a Bayesian approach is more objective may not be correct.   One can force it to be objective by requiring use of a certain prior probability but this then creates other potential errors and in theory is less accurate because it may not consider variables that are available to the physician seeing the patient but not in the model.   Advocates of a Bayesian approach seem to forget that the prior probability is generally not known but is merely a subjective "guess".   So, comparing a Bayesian approach to the usual approaches is something that would be real interesting to do yet, one must be careful in defining exactly what a Bayesian approach means.

Michael Kanter, M.D.
Regional Medical Director of Quality & Clinical Analysis Southern California Permanente Medical Group
(626) 405-5722 (tie line 8+335)
THRIVE By Getting Regular Exercise

NOTICE TO RECIPIENT:  If you are not the intended recipient of this e-mail, you are prohibited from sharing, copying, or otherwise using or disclosing its contents.  If you have received this e-mail in error, please notify the sender immediately by reply e-mail and permanently delete this e-mail and any attachments without reading, forwarding or saving them.  Thank you.




From:        "Graber, Mark" <Mark.Graber at VA.GOV>
To:        IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Date:        07/19/2014 10:03 AM
Subject:        Re: [IMPROVEDX] [EXTERNAL] [IMPROVEDX] Bayesian diagnosis
________________________________



I'd love to see the kind of comparative study Bimal is advocating.  One of the suggestions on how to improve diagnosis that seems like it should work is exactly what Bimal is advocating: expanding the differential diagnosis and weighing the options appropriately.  Hardeep's work found that in cases of dx error there was no differential diagnosis listed 80% of the time.  Would this be an antidote to dx error, as Jason Maude has suggested (Differential diagnosis: The key to reducing diagnostic error?   DIAGNOSIS 2014; 1:107-109).

Just to make a point though:  The Ledley\Lusted approach was to use the full range of conditional probability analysis in diagnosis, which includes first deriving all the possible candidate diagnoses, then weighing the evidence pro and con, and using tests to adjust probabilities to narrow the list down..  It seems that when people refer to Bayesian analysis these days they are referring to the narrower usage of Bayes theorem to calculate the probability of a SINGLE disease probability.  This focus on just one entity is exactly the opposite of what Ledley\Lusted were advocating, and to the extent that it discourages thinking about all the other diagnostic options, could work against diagnostic accuracy.


________________________________
From: Lorri Zipperer <Lorri at zpm1.com>
Reply-To: Society to Improve Diagnosis in Medicine <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, Lorri Zipperer <Lorri at zpm1.com>
Date: Thu, 17 Jul 2014 17:05:08 -0400
To: <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: [EXTERNAL] [IMPROVEDX] Bayesian diagnosis

**** moderator forwarded ****

-----Original Message-----
From: Jain, Bimal P.,M.D. [mailto:BJAIN at PARTNERS.ORG <mailto:BJAIN at PARTNERS.ORG> ]
Sent: Wednesday, July 16, 2014 5:35 AM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG; 'Graber, Mark'
Subject: RE: [IMPROVEDX] [EXTERNAL] [IMPROVEDX] Patient-Provider Communication and Diagnostic Error

Great references. Here are some thoughts on a somewhat unrelated topic-- Bayesian diagnosis.

1. Since Ledley and Lusted introduced a probabilistic (Bayesian) approach to clinical diagnosis in 1959 (SCIENCE 1959), innumerable books and papers have been written advocating its use in actual practice.

2. However, this does not seem to have happened to any great extent in last
55 years. For example, CPCs in NEJM continue to be discussed in a manner that is not Bayesian. Also discussions about diagnosis in clinical problem solving exercises on actual patients published in journals do not employ a Bayesian approach. In the occasional patient in whom it is employed, it does not come out looking too good as it seems to suggest an erroneous diagnosis (Pauker, NEJM 1992).

3. There is no published evidence it is superior to the usual method of diagnosis in actual practice which is about 85 percent accurate (Graber BMJ 2013). There are no head to head trials which demonstrate superiority of Bayesian over usual method of diagnosis.

4. It is ironic, in this day and age of evidence based medicine, Bayesian diagnosis continues to be advocated without any evidence for its superiority.
If it were a treatment, this issue would have been settled a long time back by a controlled trial.

5. In all studies on diagnostic errors (Gordon Schiff, Hardeep Singh, Mark Graber), the method of diagnosis has not been identified as a source of diagnostic error. How would switching to a Bayesian approach reduce error?

6. One of the major causes of errors found in these studies is a failure to suspect a disease in patients with atypical presentations. This error has potential to increase with a Bayesian approach in which a low pretest probability may be interpreted as low plausibility or low pretest evidence leading to ruling out a disease without testing.

7. A Bayesian approach appears attractive, it seems, due to consistency and elegance of underlying Bayes theorem which it employs.But reliance solely on mathematical criteria without observation can be misleading as seen from the well known example of planetary orbits. For over two thousand years, planetary orbits were believed to be circular due to mathematical beauty and elegance of a circle as a geometrical figure. This belief was found to be erroneous when Kepler observed positions of Mars and calculated its orbit to be elliptical.

8. I believe we need to have observational evidence of some sort, probably from a controlled trial, to decide if Bayesian diagnosis is superior to usual diagnosis or not. Only if it is found to be superior can it be recommended for widespread use.

Bimal

Bimal Jain MD
NorthShore Medical Center
Lynn MA 01904

-----Original Message-----
From: Graber, Mark [mailto:Mark.Graber at VA.GOV <mailto:Mark.Graber at VA.GOV> ]
Sent: Tuesday, July 15, 2014 5:54 PM
To: IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG
Subject: Re: [IMPROVEDX] [EXTERNAL] [IMPROVEDX] Patient-Provider Communication and Diagnostic Error

Barbara - Thanks for asking a GREAT question.  I think we all know the
answer.   The bad news is that I don't think you'll find a paper that
clearly links effective communication with improved diagnostic accuracy.
The reason is that so far we don't have a good way to actually measure diagnostic accuracy.  We really need that tool to start making progress in studying diagnostic accuracy and error.  David Sofen (Palo Alto Foundation Medical Group) and I have discussed this exact issue a couple of times, and the possibility of looking to see if there is correlation between communication style and diagnostic accuracy just amongst all the physicians in a large practice.  Hopefully he will weigh in on his own as well.


The good news is that there is PLENTY of INDIRECT evidence to support the relationship between effective communication and diagnostic quality:

1)   Communication breakdowns are a common cause of diagnostic error.
(Graber et al.  Diagnostic Error in Internal Medicine.  Arch Int Med (2005).
165:1493-99).  Hardeep Singh also has a series of publications on diagnostic errors arising from breakdowns in lab communications, which is a related issue.

2)  As Charlene Weir points out, there is evidence that the diagnosis
emerges from the history alone in the majority of cases.    (Peterson MC,
Holbrook JH, Von Hales D, et al. Contributions of the history, physical examination, and laboratory investigation in making medical diagnoses. The Western Journal of Medicine. BMJ 1992;156:163-5.  Wahner-Roedler DL, Chaliki SS, Bauer BA, et al. Who makes the diagnosis? The role of clinical skills and diagnostic test results. J Eval Clin Pract 2007;13:321-5.)  There is also a famous Osler quote on this:  "Listen to your patient, he is telling you the diagnosis".

3  There is substantial evidence that high levels of patient satisfaction (typically reflecting communication) correlate with improved health outcomes in general. Articles that David sent me address this point:

§  Doyle, C., Lennox, L., & Bell, D. (2013). "A systematic review of evidence on the links between patient experience and clinical safety and effectiveness." BMJ Open, 3. Available at http://bmjopen.bmj.com/content/3/1/e001570.full <http://bmjopen.bmj.com/content/3/1/e001570.full> . This study finds consistent positive associations among patient experience, patient safety, and clinical effectiveness for a wide range of disease areas, settings, outcome measures, and study designs. It finds that patient experience is positively associated with clinical effectiveness and patient safety, and the results support the case for the inclusion of patient experience as one of the central pillars of quality in health care.

§  Boulding, W., Glickman, S. W., Manary, M. P., Schulman, K. A., & Staelin, R. (2011). "Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days", American Journal of Managed Care, 17(1), 41-48. Focusing on three common ailments-heart attack, heart failure and pneumonia-the authors measured 30-day readmission rates at roughly 2,500 hospitals and found that patient satisfaction scores were more closely linked with high-quality hospital care than clinical performance measures.
Hospitals that scored highly on patient satisfaction with discharge planning also tended to have the lowest number of patients return within a month for all three specified ailments. Overall, high patient satisfaction scores were more closely linked to a hospital's low readmission rates than a solid showing on clinical performance measures.

§  Glickman, S. W., Boulding, W., Manary, M., Staelin, R., Roe, M. T., Wolosin, R. J., et al. (2010). "Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction", Cardiovascular Quality and Outcomes, 3(2), 188-195. Hospitals use patient satisfaction surveys to assess their quality of care. The objective of this study was to determine whether patient satisfaction is associated with adherence to practice guidelines and outcomes for acute myocardial infarction and to identify the key drivers of patient satisfaction. The authors found that higher patient satisfaction is associated with improved guideline adherence and lower inpatient mortality rates, suggesting that patients are good discriminators of the type of care they receive. Thus, patients' satisfaction with their care provides important incremental information on the quality of acute myocardial infarction care.

§  Isaac, T., Zaslavsky, A. M., Cleary, P. D., & Landon, B. E. (2010). "The relationship between patients' perception of care and measures of hospital quality and safety", Health Services Research, 45(4), 1024-1040. The overall rating of the hospital and willingness to recommend the hospital had strong relationships with technical performance in all medical conditions and surgical care. The authors found that better patient experiences for each measure domain were associated with lower decubitus ulcer rates, and for at least some domains with each of the other assessed complications, such as infections due to medical care.

§  Jha, A. K., Orav, E. J., Zheng, J., & Epstein, A. M. (2008). "Patients'
perception of hospital care in the United States", New England Journal of Medicine, 359, 1921-1931. This study assessed the performance of hospitals across multiple domains of patients' experiences and found a positive relationship between patients' experiences and the quality of clinical care.
The authors found that patients who received care in hospitals with a high ratio of nurses to patient-days reported somewhat better experiences than those who received care in hospitals with a lower ratio, and hospitals that performed well on the HCAHPS survey provided a higher quality of care across all measures of clinical quality than did those that did not perform well on the survey.


Mark L Graber MD FACP
President, SIDM
Sr Fellow, RTI International



________________________________
From: Barbara Balik <Barbara at THECOMMONFIRE.COM>
Reply-To: Society to Improve Diagnosis in Medicine <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>, Barbara Balik <Barbara at THECOMMONFIRE.COM>
Date: Tue, 15 Jul 2014 10:30:03 -0400
To: <IMPROVEDX at LIST.IMPROVEDIAGNOSIS.ORG>
Subject: [EXTERNAL] [IMPROVEDX] Patient-Provider Communication and Diagnostic Error

I am looking for citations that link effective patient-provider communication and improvement in diagnosis accuracy.

Effective communication from the patient's view is often described as an effective listener, understanding the patient's story, and respecting values and choices

I appreciate any help



Barbara Balik, RN, EdD

Balik.Barbara at gmail.com
www.thecommonfire.com <www.thecommonfire.com> <http://www.thecommonfire.com <http://www.thecommonfire.com/> >

Cell: 651.249.9237
Office: 505.797.8933

Common Fire Healthcare Consulting:
Partnering with healthcare leaders to forge quality and safety outcomes



For additional information and subscription commands, visit:
http://www.lsoft.com/resources/faq.asp#4A <http://www.lsoft.com/resources/faq.asp#4A>

http://LIST.IMPROVEDIAGNOSIS.ORG/ <http://list.improvediagnosis.org/> (with your password)

Visit the searchable archives or adjust your subscription at:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX>

Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br> <a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1> " target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1> </a> </p>



For additional information and subscription commands, visit:
http://www.lsoft.com/resources/faq.asp#4A <http://www.lsoft.com/resources/faq.asp#4A>

http://LIST.IMPROVEDIAGNOSIS.ORG/ <http://list.improvediagnosis.org/> (with your password)

Visit the searchable archives or adjust your subscription at:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX>

Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br> <a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1> " target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1 <http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1> </a> </p>


________________________________


To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1

or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX


Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/


________________________________


To unsubscribe from IMPROVEDX: click the following link:
http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1
or send email to: IMPROVEDX-SIGNOFF-REQUEST at LIST.IMPROVEDIAGNOSIS.ORG

Visit the searchable archives or adjust your subscription at: http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?INDEX

Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:
http://www.improvediagnosis.org/






Moderator: Lorri Zipperer Lorri at ZPM1.com, Communication co-chair, Society for Improving Diagnosis in Medicine

To unsubscribe from the IMPROVEDX list, click the following link:<br> <a href="http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1" target="_blank">http://list.improvediagnosis.org/scripts/wa-IMPDIAG.exe?SUBED1=IMPROVEDX&A=1</a>
</p>










More information about the Test mailing list