Unintended consequences of machine learning

Bruno, Michael mbruno at PENNSTATEHEALTH.PSU.EDU
Tue Aug 8 18:53:43 UTC 2017

Hi everybody,

There is an interesting article in the current issue of JAMA from an Italian group, a perspective piece on the downside of Machine Learning and decision-support systems in medicine.  Here is the link:


A major concern is the potential negative impact on the skills of physicians.  Here is a brief excerpt:

Reducing the Skills of Physicians

A major issue related to incorporation of ML-DSS in medicine could be overreliance on the capabilities of automation. Although the phenomenon of overreliance on technology could be tempting to users in the short term for the convenience and efficiency of automated aids, in the long term these tools can lead to the related phenomenon of deskilling3<http://jamanetwork.com/journals/jama/fullarticle/2645762?utm_medium=alert&utm_source=JAMALatestIssue&utm_campaign=08-08-2017#jvp170094r3> (i.e., the reduction of the level of skill required to complete a task when some or all components of the task are partly automated, and which may cause serious disruptions of performance or inefficiencies whenever technology fails or breaks down). This process can affect physicians' ability to derive informed opinions on the basis of detectable signs, symptoms, and available data.

For example, in a study of 50 mammogram readers, there was a 14% decrease in diagnostic sensitivity when more discriminating readers were presented with challenging images marked by computer-aided detection.4<http://jamanetwork.com/journals/jama/fullarticle/2645762?utm_medium=alert&utm_source=JAMALatestIssue&utm_campaign=08-08-2017#jvp170094r4> Another study of 30 internal medicine residents showed that the residents exhibited a decrease in diagnostic accuracy (from 57% to 48%) when electrocardiograms were annotated with inaccurate computer-aided diagnoses.5<http://jamanetwork.com/journals/jama/fullarticle/2645762?utm_medium=alert&utm_source=JAMALatestIssue&utm_campaign=08-08-2017#jvp170094r5> Further research is needed to better understand whether the overreliance on ML-DSS that could outperform or perform as well as human observers could also cause a subtle loss of self-confidence and affect the willingness of a physician to provide a definitive interpretation or diagnosis.


3Hoff  T.  Deskilling and adaptation among primary care physicians using two work innovations.  Health Care Manage Rev. 2011;36(4):338-348.
4Povyakalo  AA, Alberdi  E, Strigini  L, Ayton  P.  How to discriminate between computer-aided and computer-hindered decisions.  Med Decis Making. 2013;33(1):98-107

5Tsai  TL, Fridsma  DB, Gatti  G.  Computer decision support as a source of interpretation error.  J Am Med Inform Assoc. 2003;10(5):478-483.

All the best,

[cid:image004.png at 01D112FF.F77F98B0]
Michael A. Bruno, M.S., M.D., F.A.C.R.
Professor of Radiology & Medicine
Vice Chair for Quality & Patient Safety
Chief, Division of Emergency Radiology
Penn State Milton S. Hershey Medical Center
* (717) 531-8703  |  6 (717) 531-5737
* mbruno at pennstatehealth.psu.edu<mailto:mbruno at pennstatehealth.psu.edu>  |
[inspired to keep patient safe]

*****E-Mail Confidentiality Notice*****
This message (including any attachments) contains information intended for a specific individual(s) and purpose that may be privileged, confidential or otherwise protected from disclosure pursuant to applicable law.  Any inappropriate use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalty.  If you have received this transmission in error, please reply to the sender indicating this error and delete the transmission from your system immediately.

Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine

HTML Version:
URL: <../attachments/20170808/6b264a1b/attachment.html> ATTACHMENT:
Name: image001.png Type: image/png Size: 2281 bytes Desc: image001.png URL: <../attachments/20170808/6b264a1b/attachment.png> ATTACHMENT:
Name: image002.png Type: image/png Size: 6043 bytes Desc: image002.png URL: <../attachments/20170808/6b264a1b/attachment-0001.png>

More information about the Test mailing list