Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies.

OBJECTIVE: To systematically examine the design, reporting standards, risk of bias, and claims of studies comparing the performance of diagnostic deep learning algorithms for medical imaging with that of expert clinicians. DESIGN: Systematic review. DATA SOURCES: Medline, Embase, Cochrane Central Register of Controlled Trials, and the World Health Organization trial registry from 2010 to June 2019. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Randomised trial registrations and non-randomised studies comparing the performance of a deep learning algorithm in medical imaging with a contemporary group of one or more expert clinicians. Medical imaging has seen a growing interest in deep learning research. The main distinguishing feature of convolutional neural networks (CNNs) in deep learning is that when CNNs are fed with raw data, they develop their own representations needed for pattern recognition. The algorithm learns for itself the features of an image that are important for classification rather than being told by humans which features to use. The selected studies aimed to use medical imaging for predicting absolute risk of existing disease or classification into diagnostic groups (eg, disease or non-disease). For example, raw chest radiographs tagged with a label such as pneumothorax or no pneumothorax and the CNN learning which pixel patterns suggest pneumothorax. REVIEW METHODS: Adherence to reporting standards was assessed by using CONSORT (consolidated standards of reporting trials) for randomised studies and TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) for non-randomised studies. Risk of bias was assessed by using the Cochrane risk of bias tool for randomised studies and PROBAST (prediction model risk of bias assessment tool) for non-randomised studies. RESULTS: Only 10 records were found for deep learning randomised clinical trials, two of which have been published (with low risk of bias, except for lack of blinding, and high adherence to reporting standards) and eight are ongoing. Of 81 non-randomised clinical trials identified, only nine were prospective and just six were tested in a real world clinical setting. The median number of experts in the comparator group was only four (interquartile range 2-9). Full access to all datasets and code was severely limited (unavailable in 95% and 93% of studies, respectively). The overall risk of bias was high in 58 of 81 studies and adherence to reporting standards was suboptimal (<50% adherence for 12 of 29 TRIPOD items). 61 of 81 studies stated in their abstract that performance of artificial intelligence was at least comparable to (or better than) that of clinicians. Only 31 of 81 studies (38%) stated that further prospective studies or trials were required. CONCLUSIONS: Few prospective deep learning studies and randomised trials exist in medical imaging. Most non-randomised trials are not prospective, are at high risk of bias, and deviate from existing reporting standards. Data and code availability are lacking in most studies, and human comparator groups are often small. Future studies should diminish risk of bias, enhance real world clinical relevance, improve reporting and transparency, and appropriately temper conclusions. STUDY REGISTRATION: PROSPERO CRD42019123605.

Financial performance of English NHS trusts and variation in clinical outcomes: a longitudinal observational study.

OBJECTIVES: To examine the association between financial performance as measured by operating margin (surplus/deficit as a proportion of turnover) and clinical outcomes in English National Health Service (NHS) trusts. SETTING: Longitudinal, observational study in 149 acute NHS trusts in England between the financial years 2011 and 2016. PARTICIPANTS: Our analysis focused on outcomes at individual NHS Trust-level (composed of one or more acute hospitals). PRIMARY AND SECONDARY OUTCOMES: Outcome measures included readmissions, inpatient satisfaction score and the following process measures: emergency department (Accident and Emergency (A&E)) waiting time targets, cancer referral and treatment targets and delayed transfers of care (DTOCs). RESULTS: There was a progressive increase in the proportion of trusts in financial deficit: 22% in 2011, 27% in 2012, 28% in 2013, 51% in 2014, 68% in 2015 and 91% in 2016. In linear regression analyses, there was no significant association between operating margin and clinical outcomes (readmission rate or inpatient satisfaction score). There was, however, a significant association between operating margin and process measures (DTOCs, A&E breaches and cancer waiting time targets). Between the best and worst financially performing Trusts, there was an approximately 2-fold increase in A&E breaches and DTOCs overall although this variation decreased over the 6 years. Between the best and worst performing trusts on cancer targets, the magnitude of difference was smaller (1.16 and 1.15-fold), although the variation slowly rose during the 6 years. CONCLUSIONS: Operating margins in English NHS trusts progressively worsened during 2011-2016, and this change was associated with poorer performance on several process measures but not with hospital readmissions or inpatient satisfaction. Significant variation exists between the best and worst financially performing Trusts. Further research is needed to examine the causal nature of relationships between financial performance, process measures and outcomes.

Respiratory disease mortality in the United Kingdom compared with EU15+ countries in 1985-2015: observational study.

OBJECTIVE: To compare age standardised death rates for respiratory disease mortality between the United Kingdom and other countries with similar health system performance. DESIGN: Observational study. SETTING: World Health Organization Mortality Database, 1985-2015. PARTICIPANTS: Residents of the UK, Austria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland, Italy, Luxembourg, the Netherlands, Portugal, Spain, Sweden, Australia, Canada, the United States, and Norway (also known as EU15+ countries). MAIN OUTCOME MEASURES: Mortality from all respiratory disease and infectious, neoplastic, interstitial, obstructive, and other respiratory disease. Differences between countries were tested over time by mixed effect regression models, and trends in subcategories of respiratory related diseases assessed by a locally weighted scatter plot smoother. RESULTS: Between 1985 and 2015, overall mortality from respiratory disease in the UK and EU15+ countries decreased for men and remained static for women. In the UK, the age standardised death rate (deaths per 100 000 people) for respiratory disease mortality in the UK fell from 151 to 89 for men and changed from 67 to 68 for women. In EU15+ countries, the corresponding changes were from 108 to 69 for men and from 35 to 37 in women. The UK had higher mortality than most EU15+ countries for obstructive, interstitial, and infectious subcategories of respiratory disease in both men and women. CONCLUSION: Mortality from overall respiratory disease was higher in the UK than in EU15+ countries between 1985 and 2015. Mortality was reduced in men, but remained the same in women. Mortality from obstructive, interstitial, and infectious respiratory disease was higher in the UK than in EU15+ countries.

Trends in mortality from pneumonia in the Europe union: a temporal analysis of the European detailed mortality database between 2001 and 2014.

BACKGROUND: Pneumonia is responsible for approximately 230,000 deaths in Europe, annually. Comprehensive and comparable reports on pneumonia mortality trends across the European Union (EU) are lacking. METHODS: A temporal analysis of national mortality statistics to compare trends in pneumonia age-standardised death rates (ASDR) of EU countries between 2001 and 2014 was performed. International Classification of Diseases version 10 (ICD-10) codes were used to extract data from the World Health Organisation European Detailed Mortality Database and trends were analysed using Joinpoint regression. RESULTS: Median pneumonia mortality across the EU for the last recorded observation was 19.8 / 100,000 and 6.9 / 100,000 for males and females, respectively. Mortality was higher in males across all EU countries, most notably in Estonia and Lithuania where the ratio of male to female ASDR was 4.0 and 3.7, respectively. Gender mortality differences were lowest in the UK and Demark with ASDR ratios of 1.1 and 1.5, respectively. Pneumonia mortality across all countries decreased by a median of 31.0% over the observation period. Countries that demonstrated an increase in pneumonia mortality were Poland (males + 33.1%, females + 10.2%), and Lithuania (males + 6.0%). CONCLUSIONS: Mortality from pneumonia is improving in most EU countries, however substantial variation in trends remains between countries and between genders.

A systematic analysis of UK cancer research funding by gender of primary investigator.

OBJECTIVES: To categorically describe cancer research funding in the UK by gender of primary investigator (PIs). DESIGN: Systematic analysis of all open-access data. METHODS: Data about public and philanthropic cancer research funding awarded to UK institutions between 2000 and 2013 were obtained from several sources. Fold differences were used to compare total investment, award number, mean and median award value between male and female PIs. Mann-Whitney U tests were performed to determine statistically significant associations between PI gender and median grant value. RESULTS: Of the studies included in our analysis, 2890 (69%) grants with a total value of £1.82 billion (78%) were awarded to male PIs compared with 1296 (31%) grants with a total value of £512 million (22%) awarded to female PIs. Male PIs received 1.3 times the median award value of their female counterparts (P<0.001). These apparent absolute and relative differences largely persisted regardless of subanalyses. CONCLUSIONS: We demonstrate substantial differences in cancer research investment awarded by gender. Female PIs clearly and consistently receive less funding than their male counterparts in terms of total investment, the number of funded awards, mean funding awarded and median funding awarded.

The impact of improving teamwork on patient outcomes in surgery: A systematic review.

BACKGROUND: The aviation industry pioneered formalised crew training in order to improve safety and reduce consequences of non-technical error. This formalised training has been successfully adapted and used to in the field of surgery to improve post-operative patient outcomes. The need to implement teamwork training as an integral part of a surgical programme is increasingly being recognised. We aim to systematically review the impact of surgical teamwork training on post-operative outcomes. METHODS: Two independent researchers systematically searched MEDLINE and Embase in accordance with PRISMA guidelines. Studies were screened and subjected to inclusion/exclusion criteria. Study characteristics and outcomes were reported and analysed. RESULTS: Our initial search identified 2720 articles. Following duplicate removal, title and abstract screening, 107 full text articles were analysed. Eight articles met our inclusion criteria. Overall, three articles supported a positive effect of good teamwork on post-operative patient outcomes. We identified key areas in study methodology that can be improved upon, including small cohort size, lack of unified training programme, and short training duration, should future studies be designed and implemented in this field. CONCLUSION: At present, there is insufficient evidence to support the hypothesis that teamwork training interventions improve patient outcomes. We believe that non-significant and conflicting results can be attributed to flaws in methodology and non-uniform training methods. With increasing amounts of evidence in this field, we predict a positive association between teamwork training and patient outcomes will come to light.

Effects of health and social care spending constraints on mortality in England: a time trend analysis.

OBJECTIVE: Since 2010, England has experienced relative constraints in public expenditure on healthcare (PEH) and social care (PES). We sought to determine whether these constraints have affected mortality rates. METHODS: We collected data on health and social care resources and finances for England from 2001 to 2014. Time trend analyses were conducted to compare the actual mortality rates in 2011-2014 with the counterfactual rates expected based on trends before spending constraints. Fixed-effects regression analyses were conducted using annual data on PES and PEH with mortality as the outcome, with further adjustments for macroeconomic factors and resources. Analyses were stratified by age group, place of death and lower-tier local authority (n=325). Mortality rates to 2020 were projected based on recent trends. RESULTS: Spending constraints between 2010 and 2014 were associated with an estimated 45 368 (95% CI 34 530 to 56 206) higher than expected number of deaths compared with pre-2010 trends. Deaths in those aged ≥60 and in care homes accounted for the majority. PES was more strongly linked with care home and home mortality than PEH, with each £10 per capita decline in real PES associated with an increase of 5.10 (3.65-6.54) (p<0.001) care home deaths per 100 000. These associations persisted in lag analyses and after adjustment for macroeconomic factors. Furthermore, we found that changes in real PES per capita may be linked to mortality mostly via changes in nurse numbers. Projections to 2020 based on 2009-2014 trend was cumulatively linked to an estimated 152 141 (95% CI 134 597 and 169 685) additional deaths. CONCLUSIONS: Spending constraints, especially PES, are associated with a substantial mortality gap. We suggest that spending should be targeted on improving care delivered in care homes and at home; and maintaining or increasing nurse numbers.

Unemployment and HIV mortality in the countries of the Organisation for Economic Co-operation and Development: 1981-2009.

OBJECTIVES: To determine an association between unemployment rates and human immunodeficiency virus (HIV) mortality in the Organisation for Economic Co-operation and Development (OECD). DESIGN: Multivariate regression analysis. PARTICIPANTS: OECD member states. SETTING: OECD. MAIN OUTCOME MEASURES: World Health Organization HIV mortality. RESULTS: Between 1981 and 2009, a 1% increase in unemployment was associated with an increase in HIV mortality in the OECD (coefficient for men 0.711, 0.334-1.089, p = 0.0003; coefficient for women 0.166, 0.071-0.260, p = 0.0007). Time lag analysis showed a significant increase in HIV mortality for up to two years after rises in unemployment: p = 0.0008 for men and p = 0.0030 for women in year 1, p = 0.0067 for men and p = 0.0403 for women in year 2. CONCLUSIONS: Rises in unemployment are associated with increased HIV mortality. Economic fiscal policy may impact upon population health. Policy discussions should take into consideration potential health outcomes.

Effects of economic downturns on child mortality: a global economic analysis, 1981-2010.

OBJECTIVES: To analyse how economic downturns affect child mortality both globally and among subgroups of countries of variable income levels. DESIGN: Retrospective observational study using economic data from the World Bank's Development Indicators and Global Development Finance (2013 edition). Child mortality data were sourced from the Institute for Health Metrics and Evaluation. SETTING: Global. PARTICIPANTS: 204 countries between 1981 and 2010. MAIN OUTCOME MEASURES: Child mortality, controlling for country-specific differences in political, healthcare, cultural, structural, educational and economic factors. RESULTS: 197 countries experienced at least 1 economic downturn between 1981 and 2010, with a mean of 7.97 downturns per country (range 0-21; SD 0.45). At the global level, downturns were associated with significant (p<0.0001) deteriorations in each child mortality measure, in comparison with non-downturn years: neonatal (coefficient: 1.11, 95% CI 0.855 to 1.37), postneonatal (2.00, 95% CI 1.61 to 2.38), child (2.93, 95% CI 2.26 to 3.60) and under 5 years of age (5.44, 95% CI 4.31 to 6.58) mortality rates. Stronger (larger falls in the growth rate of gross domestic product/capita) and longer (lasting 2 years rather than 1) downturns were associated with larger significant deteriorations (p<0.001). During economic downturns, countries in the poorest quartile experienced ∼1½ times greater deterioration in neonatal mortality, compared with their own baseline; a 3-fold deterioration in postneonatal mortality; a 9-fold deterioration in child mortality and a 3-fold deterioration in under-5 mortality, than countries in the wealthiest quartile (p<0.0005). For 1-5 years after downturns ended, each mortality measure continued to display significant deteriorations (p<0.0001). CONCLUSIONS: Economic downturns occur frequently and are associated with significant deteriorations in child mortality, with worse declines in lower income countries.