Evaluation of medical decision support systems (DDX generators) using real medical cases of varying complexity and origin.

BACKGROUND: Medical decision support systems (CDSSs) are increasingly used in medicine, but their utility in daily medical practice is difficult to evaluate. One variant of CDSS is a generator of differential diagnoses (DDx generator). We performed a feasibility study on three different, publicly available data sets of medical cases in order to identify the frequency in which two different DDx generators provide helpful information (either by providing a list of differential diagnosis or recognizing the expert diagnosis if available) for a given case report. METHODS: Used data sets were n = 105 cases from a web-based forum of telemedicine with real life cases from Afghanistan (Afghan data set; AD), n = 124 cases discussed in a web-based medical forum (Coliquio data set; CD). Both websites are restricted for medical professionals only. The third data set consisted 50 special case reports published in the New England Journal of Medicine (NEJM). After keyword extraction, data were entered into two different DDx generators (IsabelHealth (IH), Memem7 (M7)) to examine differences in target diagnosis recognition and physician-rated usefulness between DDx generators. RESULTS: Both DDx generators detected the target diagnosis equally successfully (all cases: M7, 83/170 (49%); IH 90/170 (53%), NEJM: M7, 28/50 (56%); IH, 34/50 (68%); differences n.s.). Differences occurred in AD, where detection of an expert diagnosis was less successful with IH than with M7 (29.7% vs. 54.1%, p = 0.003). In contrast, in CD IH performed significantly better than M7 (73.9% vs. 32.6%, p = 0.021). Congruent identification of target diagnosis occurred in only 46/170 (27.1%) of cases. However, a qualitative analysis of the DDx results revealed useful complements from using the two systems in parallel. CONCLUSION: Both DDx systems IsabelHealth and Memem7 provided substantial help in finding a helpful list of differential diagnoses or identifying the target diagnosis either in standard cases or complicated and rare cases. Our pilot study highlights the need for different levels of complexity and types of real-world medical test cases, as there are significant differences between DDx generators away from traditional case reports. Combining different results from DDx generators seems to be a possible approach for future review and use of the systems.

Effect of incomplete milking on milk production rate and composition with 2 daily milkings.

The primary aim of this study was to assess the effects of incomplete milking on milk secretion and milk composition at the quarter level. Twelve cows were enrolled beginning at 5 d in milk and remained on study through 47 d in milk. Half of each contralateral udder was incompletely milked (treatment), detaching the teat cup early to leave approximately 30% of the total milk yield behind. This target milk remaining in the gland was based on weekly calibration milking measurements of quarter total milk yield. Control quarters were milked completely until milk flow had decreased to 0 kg/min based on visual assessment. Harvested milk yield was measured twice daily at each milking, and milk components (fat, protein, lactose, solids nonfat, milk urea nitrogen) and somatic cell count, were measured twice weekly at the quarter level. The experimental unit in this design was the half-udder, and a mixed-model approach was used to assess the main and interactive effects of experiment week and treatment on milk production rate, milk remaining in the gland, and milk composition. The effect of treatment on milk production rate was significant, with the average control half-udder producing 0.97 kg/h and the treatment half-udder 0.73 kg/h. The effect of week on milk production rate and the interaction of week × treatment were also significant. The effect of treatment on milk remaining in the gland was significant, illustrating that an increase in milk remaining in the cisternal compartment had been achieved. We detected a significant decrease in milk lactose percentage in treatment half-udders, and a significant increase in somatic cell count (log10). The increase was relatively small, from a geometric mean of 26,300 cells/mL in control quarters to 48,300 cells/mL in treatment quarters. The decrease in milk production rate in treatment half-udders supports current knowledge about how mammary epithelial cell secretion, proliferation, and apoptosis are modulated by autocrine-paracrine factors.