Deep Differential Diagnoses - A new(?) kind of vignette-based assessment item

Mark Gusack gusackm at COMCAST.NET
Wed Jan 2 19:37:12 UTC 2019

Good Afternoon Martin:


What you are doing is actually a modification of the Delphi Method developed after WWII by Rand Corporation to help predict the future using input from ‘experts’ in a particular field.  In this case you are using clinicians as your experts.  This approach was utilized by Dr. Laura Zwann several years ago and published in a journal article I highly recommend you read (I can’t remember the journal.  Sorry).  A more recent article using the Delphi approach was published in Diagnosis recently.


As for plotting a frequency distribution of lists of diagnoses submitted by 100 clinicians.  I’m not sure that is valid from a theoretical point of view.  Each diagnosis is discrete so you are doing a comparison of frequencies of discrete categories and since each clinician is allowed to provide a differential diagnosis the individual data points may not be fully independent from each other for each clinician.  Therefore, calculating a mean, standard deviation, skewdness, and kurtosis may not be correct.  Instead, it may be more appropriate to calculate percentages and using Euler diagrams (what you were taught as Venn diagrams…) to present your data.


It should be noted that the greatest value in your approach would be to determine the ‘correct’ diagnosis first, then send out a set of clinical findings, and compare the results of querying 100 clinicians to see how reliable they are.  I would be quite interested in seeing a plot of that!


As for presenting cases with indeterminate diagnosis, I have rarely seen this done.  I think the NEJM has done this on rare occasion in their Grand Rounds section.




Mark Gusack, M.D.


MANX Enterprises, Ltd.

304 521-1980 <> 


From: Martin Pusic <mpusic at GMAIL.COM> 
Sent: Wednesday, January 2, 2019 1:39 PM
Subject: [IMPROVEDX] Deep Differential Diagnoses - A new(?) kind of vignette-based assessment item


Hi all,

I'm  writing a proposal to develop a new kind of vignette-driven DDx case item for our residents. In it we propose to use crowd-sourcing to identify ALL the plausible diagnoses that could originate from a given case vignette. ie, by having a 100 clinicians list their possible diagnoses, we get a frequency distribution of possibilities that people considered plausible.  We would use as many clinicians as necessary to achieve "saturation" where no further plausible diagnoses are being added.

After being cleaned up for duplicates/terminology etc, the idea would be to present the weighted DDx list as the "answer" to the vignette.  The idea is that some diagnoses fall nicely into a single best answer DDx (e.g. zoster/shingles - left hand pane of attached image) while some require an inductive approach that has a "long tail" which requires more inductive effort  (e.g. lower abdominal pain - right hand pane).   

My question to the group:  is this novel?  ie, are there existing systematized approaches where the answer to the case is NOT a best single answer but rather the weighted DDx?   For pedagogic reasons, would anyone ever present a case and NOT present what it turned out to be, but leave it at what it MIGHT have been? 

Any literature references sincerely appreciated.

--Martin Pusic, NYU School of Medicine




To unsubscribe from IMPROVEDX: click the following link: <> &A=1 


Visit the searchable archives or adjust your subscription at:

Moderator:David Meyers, Board Member, Society for Improving Diagnosis in Medicine

To learn more about SIDM visit:

Moderator: David Meyers, Board Member, Society to Improve Diagnosis in Medicine

HTML Version:
URL: <../attachments/20190102/a0feb594/attachment.html>

More information about the Test mailing list