The Risk of Endometrial Malignancy and Other Endometrial Pathology in Women with Abnormal Uterine Bleeding: An Ultrasound-Based Model Development Study by the IETA Group.

OBJECTIVES: The aim of this study was to develop a model that can discriminate between different etiologies of abnormal uterine bleeding. DESIGN: The International Endometrial Tumor Analysis 1 study is a multicenter observational diagnostic study in 18 bleeding clinics in 9 countries. Consecutive women with abnormal vaginal bleeding presenting for ultrasound examination (n = 2,417) were recruited. The histology was obtained from endometrial sampling, D&C, hysteroscopic resection, hysterectomy, or ultrasound follow-up for >1 year. METHODS: A model was developed using multinomial regression based on age, body mass index, and ultrasound predictors to distinguish between: (1) endometrial atrophy, (2) endometrial polyp or intracavitary myoma, (3) endometrial malignancy or atypical hyperplasia, (4) proliferative/secretory changes, endometritis, or hyperplasia without atypia and validated using leave-center-out cross-validation and bootstrapping. The main outcomes are the model's ability to discriminate between the four outcomes and the calibration of risk estimates. RESULTS: The median age in 2,417 women was 50 (interquartile range 43-57). 414 (17%) women had endometrial atrophy; 996 (41%) had a polyp or myoma; 155 (6%) had an endometrial malignancy or atypical hyperplasia; and 852 (35%) had proliferative/secretory changes, endometritis, or hyperplasia without atypia. The model distinguished well between malignant and benign histology (c-statistic 0.88 95% CI: 0.85-0.91) and between all benign histologies. The probabilities for each of the four outcomes were over- or underestimated depending on the centers. LIMITATIONS: Not all patients had a diagnosis based on histology. The model over- or underestimated the risk for certain outcomes in some centers, indicating local recalibration is advisable. CONCLUSIONS: The proposed model reliably distinguishes between four histological outcomes. This is the first model to discriminate between several outcomes and is the only model applicable when menopausal status is uncertain. The model could be useful for patient management and counseling, and aid in the interpretation of ultrasound findings. Future research is needed to externally validate and locally recalibrate the model.

Risk assessment for endometrial cancer in women with abnormal vaginal bleeding: Results from the prospective IETA-1 cohort study.

OBJECTIVE: To investigate the association between personal history, anthropometric features and lifestyle characteristics and endometrial malignancy in women with abnormal vaginal bleeding. METHODS: Prospective observational cohort assessed by descriptive and multivariable logistic regression analyses. Three features-age, body mass index (BMI; calculated as weight in kilograms divided by the square of height in meters), and nulliparity-were defined a priori for baseline risk assessment of endometrial malignancy. The following variables were tested for added value: intrauterine contraceptive device, bleeding pattern, age at menopause, coexisting diabetes/hypertension, physical exercise, fat distribution, bra size, waist circumference, smoking/drinking habits, family history, use of hormonal/anticoagulant therapy, and sonographic endometrial thickness. We calculated adjusted odds ratio, optimism-corrected area under the receiver operating characteristic curve (AUC), R2 , and Akaike's information criterion. RESULTS: Of 2417 women, 155 (6%) had endometrial malignancy or endometrial intraepithelial neoplasia. In women with endometrial cancer median age was 67 years (interquartile range [IQR] 56-75 years), median parity was 2 (IQR 0-10), and median BMI was 28 (IQR 25-32). Age, BMI, and parity produced an AUC of 0.82. Other variables marginally affected the AUC, adding endometrial thickness substantially increased the AUC in postmenopausal women. CONCLUSION: Age, parity, and BMI help in the assessment of endometrial cancer risk in women with abnormal uterine bleeding. Other patient information adds little, whereas sonographic endometrial thickness substantially improves assessment.

ROC curves for clinical prediction models part 1. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models.

OBJECTIVES: Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations. STUDY DESIGN AND SETTING: We conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data. RESULTS: Of 86 identified prediction modeling studies, 52 (60%) presented ROC curves without thresholds and one (1%) presented an ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information have an unstable shape even for the same area under the curve (AUC) and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk thresholds. CONCLUSION: ROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk thresholds. Therefore, if discriminatory ability must be visualized, classification plots are attractive.

Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study.

When developing risk prediction models on datasets with limited sample size, shrinkage methods are recommended. Earlier studies showed that shrinkage results in better predictive performance on average. This simulation study aimed to investigate the variability of regression shrinkage on predictive performance for a binary outcome. We compared standard maximum likelihood with the following shrinkage methods: uniform shrinkage (likelihood-based and bootstrap-based), penalized maximum likelihood (ridge) methods, LASSO logistic regression, adaptive LASSO, and Firth's correction. In the simulation study, we varied the number of predictors and their strength, the correlation between predictors, the event rate of the outcome, and the events per variable. In terms of results, we focused on the calibration slope. The slope indicates whether risk predictions are too extreme (slope < 1) or not extreme enough (slope > 1). The results can be summarized into three main findings. First, shrinkage improved calibration slopes on average. Second, the between-sample variability of calibration slopes was often increased relative to maximum likelihood. In contrast to other shrinkage approaches, Firth's correction had a small shrinkage effect but showed low variability. Third, the correlation between the estimated shrinkage and the optimal shrinkage to remove overfitting was typically negative, with Firth's correction as the exception. We conclude that, despite improved performance on average, shrinkage often worked poorly in individual datasets, in particular when it was most needed. The results imply that shrinkage methods do not solve problems associated with small sample size or low number of events per variable.

Risk of complications in patients with conservatively managed ovarian tumours (IOTA5): a 2-year interim analysis of a multicentre, prospective, cohort study.

BACKGROUND: Ovarian tumours are usually surgically removed because of the presumed risk of complications. Few large prospective studies on long-term follow-up of adnexal masses exist. We aimed to estimate the cumulative incidence of cyst complications and malignancy during the first 2 years of follow-up after adnexal masses have been classified as benign by use of ultrasonography. METHODS: In the international, prospective, cohort International Ovarian Tumor Analysis Phase 5 (IOTA5) study, patients aged 18 years or older with at least one adnexal mass who had been selected for surgery or conservative management after ultrasound assessment were recruited consecutively from 36 cancer and non-cancer centres in 14 countries. Follow-up of patients managed conservatively is ongoing at present. In this 2-year interim analysis, we analysed patients who were selected for conservative management of an adnexal mass judged to be benign on ultrasound on the basis of subjective assessment of ultrasound images. Conservative management included ultrasound and clinical follow-up at intervals of 3 months and 6 months, and then every 12 months thereafter. The main outcomes of this 2-year interim analysis were cumulative incidence of spontaneous resolution of the mass, torsion or cyst rupture, or borderline or invasive malignancy confirmed surgically in patients with a newly diagnosed adnexal mass. IOTA5 is registered with ClinicalTrials.gov, number NCT01698632, and the central Ethics Committee and the Belgian Federal Agency for Medicines and Health Products, number S51375/B32220095331, and is ongoing. FINDINGS: Between Jan 1, 2012, and March 1, 2015, 8519 patients were recruited to IOTA5. 3144 (37%) patients selected for conservative management were eligible for inclusion in our analysis, of whom 221 (7%) had no follow-up data and 336 (11%) were operated on before a planned follow-up scan was done. Of 2587 (82%) patients with follow-up data, 668 (26%) had a mass that was already in follow-up at recruitment, and 1919 (74%) presented with a new mass at recruitment (ie, not already in follow-up in the centre before recruitment). Median follow-up of patients with new masses was 27 months (IQR 14-38). The cumulative incidence of spontaneous resolution within 2 years of follow-up among those with a new mass at recruitment (n=1919) was 20·2% (95% CI 18·4-22·1), and of finding invasive malignancy at surgery was 0·4% (95% CI 0·1-0·6), 0·3% (<0·1-0·5) for a borderline tumour, 0·4% (0·1-0·7) for torsion, and 0·2% (<0·1-0·4) for cyst rupture. INTERPRETATION: Our results suggest that the risk of malignancy and acute complications is low if adnexal masses with benign ultrasound morphology are managed conservatively, which could be of value when counselling patients, and supports conservative management of adnexal masses classified as benign by use of ultrasound. FUNDING: Research Foundation Flanders, KU Leuven, Swedish Research Council.

The potential value of activin B and fibronectin for the triage of pregnancies of unknown location and prediction of first trimester viability.

AIM: We have assessed the potential predictive ability of the biomarkers activin B and fibronectin (FN1) alone and when added to established markers for triaging patients as being at low or high risk of ectopic pregnancy (EP). We also assessed their use as predictors of viability at 12 weeks gestation. METHODS: Exploratory secondary analysis of a prospective study including all women classified as a pregnancy of known location (PUL) based on transvaginal ultrasonography between January and December 2007 at the early pregnancy unit of St Georges' Hospital (London). We used multinomial logistic regression to assess the diagnostic potential of the biomarkers to triage PUL at high risk of complications (EP or persistent PUL), and standard binary logistic regression to predict first trimester viability at 12 weeks. RESULTS: For discriminating high-risk (n = 16) from low-risk PUL (n = 93), the area under the receiver operating characteristic curve (AUC) was 0.75 (95% confidence interval 0.60-0.85) for activin B and 0.55 (0.41-0.68) for FN1. Adding activin B to a multinomial logistic regression model incorporating ß-hCG ratio and initial progesterone yielded odds ratios of 0.16 (0.05-0.55) for failing vs high-risk PUL and 0.29 (0.07-1.19) for intrauterine vs high-risk PUL and increased the model's AUC from 0.84 to 0.89. At a risk threshold of 5% for high-risk PUL, sensitivity increased from 84% to 87% and specificity from 48% to 64%. For discriminating viable (n = 28) from non-viable (n = 81) pregnancies at 12 weeks, both markers had an AUC of 0.54. CONCLUSIONS: Our results suggested that activin B may be a promising marker to improve PUL triage in addition to established markers.

Doctors' perception of support and the processes involved in complaints investigations and how these relate to welfare and defensive practice: a cross-sectional survey of the UK physicians.

OBJECTIVE: How adverse outcomes and complaints are managed may significantly impact on physician well-being and practice. We aimed to investigate how depression, anxiety and defensive medical practice are associated with doctors actual and perceived support, behaviour of colleagues and process issues regarding how complaints investigations are carried out. DESIGN: A survey study. Respondents were classified into three groups: no complaint, recent/current complaint (within 6 months) or past complaint. Each group completed specific surveys. SETTING: British Medical Association (BMA) members were invited to complete an online survey. PARTICIPANTS: 95 636 members of the BMA were asked to participate. 7926 (8.3%) completed the survey, of whom 1780 (22.5%) had no complaint, 3889 (49.1%) had a past complaint and 2257 (28.5%) had a recent/current complaint. We excluded those with no complaints leaving 6144 in the final sample. PRIMARY OUTCOMES MEASURES: We measured anxiety and depression using the Generalised Anxiety Disorder Scale 7 and Physical Health Questionnaire 9. Defensive practice was assessed using a new measure for avoidance and hedging. RESULTS: Most felt supported by colleagues (61%), only 31% felt supported by management. Not following process (56%), protracted timescales (78%), vexatious complaints (49%), feeling bullied (39%) or victimised for whistleblowing (20%), and using complaints to undermine (31%) were reported. Perceived support by management (relative risk (RR) depression: 0.77, 95% CI 0.71 to 0.83; RR anxiety: 0.80, 95% CI 0.74 to 0.87), speaking to colleagues (RR depression: 0.64, 95% CI 0.48 to 0.84 and RR anxiety: 0.69, 95% CI 0.51 to 0.94, respectively), fair/accurate documentation (RR depression: 0.80, 95% CI 0.75 to 0.86; RR anxiety: 0.81, 95% CI 0.75 to 0.87), and being informed about rights (RR depression 0.96 (0.89 to 1.03) and anxiety 0.94 (0.87 to 1.02), correlated positively with well-being and reduced defensive practice. Doctors worried most about professional humiliation following a complaint investigation (80%). CONCLUSION: Poor process, prolonged timescales and vexatious use of complaints systems are associated with decreased psychological welfare and increased defensive practice. In contrast, perceived support from colleagues and management is associated with a reduction in these effects.

Doctors' experiences and their perception of the most stressful aspects of complaints processes in the UK: an analysis of qualitative survey data.

OBJECTIVES: To examine doctors' experiences of complaints, including which aspects are most stressful. We also investigated how doctors felt complaints processes could be improved. DESIGN AND METHODS: A qualitative study based on a cross-sectional survey of members of the British Medical Association (BMA). We asked the following: (1) Try to summarise as best as you can your experience of the complaints process and how it made you feel. (2) What were the most stressful aspects of the complaint? (3) What would you improve in the complaints system? PARTICIPANTS: We sent the survey to 95 636 doctors, and received 10 930 (11.4%) responses. Of these, 6146 had a previous, recent or current complaint and 3417 (31.3%) of these respondents answered questions 1 and 2. We randomly selected 1000 answers for analysis, and included 100 using the saturation principle. Of this cohort, 93 responses for question 3 were available. MAIN RESULTS: Doctors frequently reported feeling powerless, emotionally distressed, and experiencing negative feelings towards both those managing complaints and the complainants themselves. Many felt unsupported, fearful of the consequences and that the complaint was unfair. The most stressful aspects were the prolonged duration and unpredictability of procedures, managerial incompetence, poor communication and perceiving that processes are biased in favour of complainants. Many reported practising defensively or considering changing career after a complaint, and few found any positive outcomes from complaints investigations. Physicians suggested procedures should be more transparent, competently managed, time limited, and that there should be an open dialogue with complainants and policies for dealing with vexatious complaints. Some felt more support for doctors was needed. CONCLUSIONS: Complaints seriously impact on doctors' psychological wellbeing, and are associated with defensive practise. This is not beneficial to patient care. To improve procedures, doctors propose they are simplified, time limited and more transparent.

Age-related differences in the sonographic characteristics of endometriomas.

STUDY QUESTION: Do sonographic characteristics of ovarian endometriomas vary with age in premenopausal women? SUMMARY ANSWER: With increasing age, multilocular cysts and cysts with papillations and other solid components become more common whereas ground glass echogenicity of cyst fluid becomes less common. WHAT IS KNOWN ALREADY: Expectant or medical management of women with endometriomas is now accepted. Therefore, the accuracy of non-invasive diagnosis of these cysts is pivotal. A clinically relevant question is whether the sonographic characteristics of ovarian endometriomas are the same irrespective of the age of the woman. STUDY DESIGN, SIZE, DURATION: This is a secondary analysis of cross-sectional data in the International Ovarian Tumor Analysis (IOTA) database. The database contains clinical and ultrasound information collected pre-operatively between 1999 and 2012 from 5914 patients with adnexal masses in 24 ultrasound centres in 10 countries. PARTICIPANTS/MATERIALS, SETTING, METHODS: There were 1005 histologically confirmed endometriomas in adult premenopausal patients found in the database and these were used in our analysis. The following ultrasound variables (defined using IOTA terminology) were used to describe the ultrasound appearance of the endometriomas: tender mass at ultrasound, largest diameter of lesion, tumour type (unilocular, unilocular-solid, multilocular, multilocular-solid, solid), echogenicity of cyst content, presence of papillations, number of papillations, height (mm) of largest papillation, presence and proportion of solid tissue and number of cyst locules, as well as vascularity in papillations and colour content of the tumour scan (colour score) on colour or power Doppler ultrasounds. Results are reported as median difference or odds ratio (OR) per 10 years increase in age. MAIN RESULTS AND THE ROLE OF CHANCE: Maximal lesion diameter did not vary substantially with age (+1.3 mm difference per 10 years increase in age, 95% confidence interval (CI) -1.4 to 4.0). Tender mass at scan was less common in the older the woman (OR 0.75, 95% CI 0.63-0.89), as were unilocular cysts relative to multilocular cysts (OR 0.70, 95% CI 0.57-0.85) and to lesions with solid components (OR 0.61, 95% CI 0.48-0.77), and ground glass echogenicity relative to homogeneous low-level echogenicity (OR 0.74, 95% CI 0.58-0.94) and other types of echogenicity of cyst contents (OR 0.64, 95% CI 0.50-0.81). Papillations were more common the older the woman (OR 1.65, 95% CI 1.24-2.21), but their height and vascularization showed no clear relation to age. LIMITATIONS, REASONS FOR CAUTION: It is a limitation that we have little clinical information on the women included, e.g. previous surgery or medical treatment for endometriosis. It is important to emphasize that we do not know the age of the endometrioma itself and that our study is not longitudinal and so does not describe changes in endometriomas over time. The differences in the ultrasound appearance of endometriomas between women of different ages might be explained by previous surgery or medical treatment and might not be an effect of age per se. WIDER IMPLICATIONS OF THE FINDINGS: Awareness of physicians that the ultrasound appearance of endometriomas differs between women of different ages may facilitate a correct diagnosis of endometrioma. STUDY FUNDING/COMPETING INTERESTS: This study was supported in part by the Regione Autonoma della Sardegna (project code CPR-24750). B.V.C., A.C. and D.T. are supported by the Fund for Scientific Research Flanders, Belgium (FWO). The authors declare that there is no conflict of interest.

A calibration hierarchy for risk models was defined: from utopia to empirical data.

OBJECTIVE: Calibrated risk models are vital for valid decision support. We define four levels of calibration and describe implications for model development and external validation of predictions. STUDY DESIGN AND SETTING: We present results based on simulated data sets. RESULTS: A common definition of calibration is "having an event rate of R% among patients with a predicted risk of R%," which we refer to as "moderate calibration." Weaker forms of calibration only require the average predicted risk (mean calibration) or the average prediction effects (weak calibration) to be correct. "Strong calibration" requires that the event rate equals the predicted risk for every covariate pattern. This implies that the model is fully correct for the validation setting. We argue that this is unrealistic: the model type may be incorrect, the linear predictor is only asymptotically unbiased, and all nonlinear and interaction effects should be correctly modeled. In addition, we prove that moderate calibration guarantees nonharmful decision making. Finally, results indicate that a flexible assessment of calibration in small validation data sets is problematic. CONCLUSION: Strong calibration is desirable for individualized decision support but unrealistic and counter productive by stimulating the development of overly complex models. Model development and external validation should focus on moderate calibration.