Statistical significance and its critics: practicing damaging science, or damaging scientific practice?

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves risk damaging science. We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim-if all thresholds are abandoned-then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variability.

Building back better needs better use of statistics.

Paul Allin and David J. Hand call for official statistics to take centre stage.

F*: an interpretable transformation of the F-measure.

The F-measure, also known as the F1-score, is widely used to assess the performance of classification algorithms. However, some researchers find it lacking in intuitive interpretation, questioning the appropriateness of combining two aspects of performance as conceptually distinct as precision and recall, and also questioning whether the harmonic mean is the best way to combine them. To ease this concern, we describe a simple transformation of the F-measure, which we call F ∗ (F-star), which has an immediate practical interpretation.

Validating and Verifying AI Systems.

AI systems will only fulfill their promise for society if they can be relied upon. This means that the role and task of the system must be properly formulated; that the system must be bug free, be based on properly representative data, and can cope with anomalies and data quality issues; and that its output is sufficiently accurate for the task.

Emission analysis of recycled tire rubber modified asphalt in hot and warm mix conditions.

The hazardous emissions of crumb rubber (CR) modified asphalt during construction has been a concern for a long period. This study aims to identify the emission components in the CR modified asphalt in traditional hot mix asphalt (HMA) and with recently developed warm mix asphalt (WMA). The dynamic headspace gas chromatography-mass spectrometry (GCMS) was employed for identifying the emission of asphalt binders at 120°C, 140°C and 160°C. The coupling of gas chromatography and Fourier-transform infrared spectroscopy (GC-FTIR) was used to analyze the emission during the plant mixing for conventional HMA, CR-HMA and CR-WMA. The results showed the emission amount was highly dependent on mixing temperature. The warm mix technology can reduce the emission level significantly and should be encouraged in the asphalt mixture containing CR. Asphalt source and other extra additives in producing CR modified asphalt can also affect the emission significantly. Asphalt mixture containing CR can release toxic emissions such as xylene and toluene significantly higher compared to that without CR. In addition, it was found that the emission amount from the GCMS test for asphalt binder was lower than that in the field test for asphalt mix due to the thin asphalt film of asphalt mix.

Aspects of Data Ethics in a Changing World: Where Are We Now?

Ready data availability, cheap storage capacity, and powerful tools for extracting information from data have the potential to significantly enhance the human condition. However, as with all advanced technologies, this comes with the potential for misuse. Ethical oversight and constraints are needed to ensure that an appropriate balance is reached. Ethical issues involving data may be more challenging than the ethical challenges of some other advanced technologies partly because data and data science are ubiquitous, having the potential to impact all aspects of life, and partly because of their intrinsic complexity. We explore the nature of data, personal data, data ownership, consent and purpose of use, trustworthiness of data as well as of algorithms and of those using the data, and matters of privacy and confidentiality. A checklist is given of topics that need to be considered.

From evidence to understanding: a commentary on Fisher (1922) 'On the mathematical foundations of theoretical statistics'.

The nature of statistics has changed over time. It was originally concerned with descriptive 'matters of state'--with summarizing population numbers, economic strength and social conditions. But during the course of the twentieth century its aim broadened to include inference--how to use data to shed light on underlying mechanisms, about what might happen in the future, about what would happen if certain actions were taken. Central to this development was Ronald Fisher. Over the course of his life he was responsible for many of the major conceptual advances in statistics. This is particularly illustrated by his 1922 paper, in which he introduced many of the concepts which remain fundamental to our understanding of how to extract meaning from data, right to the present day. It is no exaggeration to say that Fisher's work, as illustrated by the ideas he described and developed in this paper, underlies all modern science, and much more besides. This commentary was written to celebrate the 350th anniversary of the journal Philosophical Transactions of the Royal Society.

Evaluating diagnostic tests: The area under the ROC curve and the balance of errors.

Because accurate diagnosis lies at the heart of medicine, it is important to be able to evaluate the effectiveness of diagnostic tests. A variety of accuracy measures are used. One particularly widely used measure is the AUC, the area under the receiver operating characteristic (ROC) curve. This measure has a well-understood weakness when comparing ROC curves which cross. However, it also has the more fundamental weakness of failing to balance different kinds of misdiagnoses effectively. This is not merely an aspect of the inevitable arbitrariness in choosing a performance measure, but is a core property of the way the AUC is defined. This property is explored, and an alternative, the H measure, is described.

Investigation of the treatability of the primary indoor volatile organic compounds on activated carbon fiber cloths at typical indoor concentrations.

Volatile organic compounds (VOCs) are a major concern for indoor air pollution because of the impacts on human health. In recent years, interest has increased in the development and design of activated carbon filters for removing VOCs from indoor air. Although extensive information is available on sources, concentrations, and types of indoor VOCs, there is little or no information on the performance of indoor air adsorption systems for removing low concentrations of primary VOCs. Filter designs need to consider various factors such as empty bed contact time, humidity effects, competitive adsorption, and feed concentration variations, whereas adsorption capacities of the indoor VOCs at the indoor concentration levels are important parameters for filter design. A preliminary assessment of the feasibility of using adsorption filters to remove low concentrations of primary VOCs can be performed. This work relates the information (including VOC classes in indoor air, the typical indoor concentrations, and the adsorption isotherms) with the design of a particular adsorbent/adsorbates system. As groundwork for filter design and development, this study selects the primary VOCs in indoor air of residences, schools, and offices in different geographical areas (North America, Europe, and Asia) on the basis of occurrence, concentrations, and health effects. Activated carbon fiber cloths (ACFCs) are chosen as the adsorbents of interest. It is demonstrated that the isotherm of a VOC (e.g., toluene on the ACFC) at typical indoor concentrations-parts per billion by volume (ppbv) level-is different than the isotherm at parts per million by volume (ppmv) levels reported in the publications. The isotherms at the typical indoor concentrations for the selected primary VOCs are estimated using the Dubinin-Radushkevitch equation. The maximum specific throughput for an indoor VOC removal system to remove benzene is calculated as a worst-case scenario. It is shown that VOC adsorption capacity is an important indicator of a filter's lifetime and needs to be studied at the appropriate concentration range. Future work requires better understanding of the realistic VOC concentrations and isotherms in indoor environments to efficiently utilize adsorbents.

User-oriented batch reactor solutions to the homogeneous surface diffusion model for different activated carbon dosages.

This paper presents a simplified approach and user-oriented solutions to the homogeneous surface diffusion model (HSDM) equations for determining the surface diffusivity using a batch reactor system. Once the surface diffusivity is known, this model could also be used to estimate the performance of activated carbon (AC) applications as a function of contact time. In addition, fixed-bed performance can be predicted using the user-oriented solutions to the HSDM for fixed beds. The step-by-step procedure for determining surface diffusion coefficients of an activated carbon adsorber, which was initially developed by Hand, Crittenden and Thacker in 1983 for a carbon dose where C(equilibrium)/C(0)=0.5, is modified to allow calculations for different carbon dosages. This modification provides solutions to the HSDM equations for different activated carbon dosages. The solutions to the HSDM framework are provided as simplified algebraic equations suitable for quick and easy estimations of D(S). The excel spread sheet is provided in the supplemental information and a detailed example is discussed.

Predicting influent estradiol and estrone concentrations for wastewater treatment facilities.

An externally validated model was developed to predict influent concentrations of natural estrogens at municipal wastewater plants. The model kinetic constant for the first-order biotransformation of estradiol into estrone, k(E2), was calculated as 0.030 hour(-1) (ranging from -0.080 to 0.49 hour(-1), with a standard deviation of 0.23) and the model value for the biodegradation k of estrone, k(E1), was -0.18 hour(-1) (ranging from -0.44 to 0.38 hour(-1), with a standard deviation of 0.3). The k values were not found to be temperature-sensitive within the range 12 to 24.5 degrees C and displayed a weak trend of increasing with total suspended solids within the range 0.103 to 0.450 g/L. The new model improved on the predictions of a previous model for influent estrone concentrations, but gave comparable results for estradiol.

Adsorption and regeneration on activated carbon fiber cloth for volatile organic compounds at indoor concentration levels.

There are increasing concerns about indoor volatile organic compounds (VOCs) regarding their health effects and frequent occurrence. Adsorption using granular activated carbon (GAC) is a safe methodology for removing VOCs from indoor air. Although GAC has been widely used to remove VOCs from indoor air, the use of activated carbon fiber cloth (ACFC) is a promising substitute to the conventional activated carbon because of its regenerative properties; hence, this paper provides promising results for the application of ACFC as a regenerative adsorbent for the removal of VOCs from indoor air. The impacts of operating variables on the adsorption/regeneration performance of the ACFC were assessed. A single-layer ACFC exhibited remarkable adsorption and regenerative properties using 100 parts per billion by volume (ppbv) toluene as the indoor contaminant. The use of Joule heating regeneration technique showed that the ACFC was rapid and efficient in removing the low initial loading of toluene. Even after continuous adsorption/regeneration cycles the ACFC showed very good performance. After over 300 heating and cooling cycles the ACFC showed excellent durability and adsorption capacity.

Accumulation of calcium in the centre of leaves of coriander (Coriandrum sativum L.) is due to an uncoupling of water and ion transport.

The aim of this study is to understand the parameters regulating calcium ion distribution in leaves. Accumulation of ions in leaf tissue is in part dependent on import from the xylem. This import via the transpiration stream is more important for ions such as calcium that are xylem but not phloem mobile and cannot therefore be retranslocated. Accumulation of calcium was measured on bulk coriander leaf tissue (Coriandrum sativum L. cv. Lemon) using ion chromatography and calcium uptake was visualized using phosphor-images of (45)Ca(2+). Leaves of plants grown in hydroponics had elevated calcium in the centre of the leaf compared with the leaf margin, while K(+) was distributed homogeneously over the leaf. This calcium was shown to be localised to the mesophyll vacuoles using EDAX. Stomatal density and evapotranspiration (water loss per unit area of leaf) were equal at inner and outer sections of the leaf. Unequal ion distribution but uniformity of water loss suggested that there was a difference in the extent of uncoupling of calcium and water transport between the inner and outer leaf. Since isolated tissue from the inner and outer leaf were able to accumulate similar amounts of calcium, it is proposed that the spatial variation of leaf calcium concentration is due to differential ion delivery to the two regions rather than tissue/cell-specific differences in ion uptake capacity. There was a positive correlation between whole leaf calcium concentration and the difference in calcium concentration between inner and outer leaf tissue. Exposing the plants to increased humidity reduced transpiration and calcium delivery to the leaf and abolished this spatial variation of calcium concentration. Mechanisms of calcium delivery to leaves are discussed. An understanding of calcium delivery and distribution within coriander will inform strategies to reduce the incidence of calcium-related syndromes such as tip-burn and provides a robust model for the transport of ions and other substances in the leaf xylem.

Breast cancer diagnosis from proteomic mass spectrometry data: a comparative evaluation.

The performance results of a wide range of different classifiers applied to proteomic mass spectra data, in a blind comparative assessment organised by Bart Mertens, are reviewed. The different approaches are summarised, issues of how to evaluate and compare the predictions are described, and the results of the different methods are examined. Although the different methods perform differently, their rank ordering varies according to how one measures performance, so that one cannot draw unequivocal conclusions about which is 'best.' Instead, it is clear that what matters is not the method by itself, but the interaction of method and user - the degree of sophistication of the user with a method. Nevertheless, such competitions do serve the useful role of setting (constantly improving) baselines against which new researchers can pit their wits and methods, as well as providing standards against which new methods should be assessed.

Mathematical model for photocatalytic destruction of organic contaminants in air.

Photocatalytic oxidation (PCO) was investigated in a bench-scale reactor for the abatement of two airborne organic contaminants: toluene and ethanol. A mathematical model that includes the impacts of light intensity, initial contaminant concentration, catalyst thickness, and relative humidity (RH) on the degradation of organic contaminants in a photocatalytic reactor was developed to describe this process. The commercially available catalyst Degussa-PtTiO2 was selected to compare with the MTU-PtTiO2-350 catalyst, which was synthesized by the sol-gel process, platinized, and calcined at 350 degrees C. For toluene removal using the MTU-PtTiO2-350 catalyst, the degradation rate increased with increases in light intensity from 0.2 to 2.2 mW/cm2 and in catalyst thickness from 0.00037 to 0.00361 cm. However, further increases in light intensity and catalyst thickness had only slight effect on the toluene degradation rate. Increasing the initial concentration from 6.29 to 127.9 microg/L and the RH from 10 to 85% resulted in decreases in the toluene degradation rate. For ethanol removal using the MTU-PtTiO2-350 catalyst, the degradation rate increased more rapidly with an increase in RH from 17 to 56%; the RH had little effect on the ethanol degradation rate while it further increased from 56% to 82%. We discuss applicability of the model to estimate the influence of process variables and to evaluate photocatalyst performance.

Principles of data mining.

Data mining is the discovery of interesting, unexpected or valuable structures in large datasets. As such, it has two rather different aspects. One of these concerns large-scale, 'global' structures, and the aim is to model the shapes, or features of the shapes, of distributions. The other concerns small-scale, 'local' structures, and the aim is to detect these anomalies and decide if they are real or chance occurrences. In the context of signal detection in the pharmaceutical sector, most interest lies in the second of the above two aspects; however, signal detection occurs relative to an assumed background model, therefore, some discussion of the first aspect is also necessary. This paper gives a lightning overview of data mining and its relation to statistics, with particular emphasis on tools for the detection of adverse drug reactions.

Modeling of indoor air treatment of polychlorinated dibenzo-p-dioxins and dibenzofurans using high-efficiency particulate air-carbon filtration.

A high-efficiency particulate air (HEPA)-carbon filtration system was developed by the Access Business Group, LLC, to reduce the indoor levels of polychlorinated dibenzo-p-dioxins and dibenzofurans (PCDD/Fs). The HEPA filter removes the particle-bound PCDD/Fs, and the carbon filter removes the gaseous fraction. Because of the toxicity of PCDD/Fs, it is very difficult to handle them in the laboratory. In this study, mathematical modeling was performed to evaluate the performance of the HEPA-carbon filtration system for PCDD/Fs removal and to optimize its design and operation. The model was calibrated with experimental data conducted with toluene in a sealed room. Model simulations with four selected congeners demonstrated that it takes approximately 1 hr for the indoor air treatment system to reach the maximum removal efficiency and that the carbon air filter has a life time of 10(7) yr for dioxin removal. Given a zero emission from the HEPA filter, the overall removal efficiency is 78.7% for 2,3,7,8-tetrachloro dibenzo-p-dioxins, 89.8% for octa-chlorodibenzodioxin, 78% for tetra-chlorodibenzofuran, and 89.8% for octa-chlorodibenzofuran. The larger the mass emission from the HEPA filter, the lower the overall removal efficiency, and the larger the ratio of the filter flow rate (Q(f)) to the room flow rate (Q), the higher the overall removal efficiency. When the ratio of Q(f)/Q is 15, an overall removal efficiency of 90% can be reached for all four of the selected compounds. The removal of the four selected compounds does not change as the relative humidity increases < or = 90%.

On loss distributions from installment-repaid loans.

The banks have been accumulating huge data bases for many years and are increasingly turning to statistics to provide insight into customer behaviour, among other things. Credit risk is an important issue and certain stochastic models have been developed in recent years to describe and predict loan default. Two of the major models currently used in the industry are considered here, and various ways of extending their application to the case where a loan is repaid in installments are explored. The aspect of interest is the probability distribution of the total loss due to repayment default at some time. Thus, the loss distribution is determined by the distribution of times to default, here regarded as a discrete-time survival distribution. In particular, the probabilities of large losses are to be assessed for insurance purposes.

Is anoxic depolarisation associated with an ADC threshold? A Markov chain Monte Carlo analysis.

A Bayesian nonlinear hierarchical random coefficients model was used in a reanalysis of a previously published longitudinal study of the extracellular direct current (DC)-potential and apparent diffusion coefficient (ADC) responses to focal ischaemia. The main purpose was to examine the data for evidence of an ADC threshold for anoxic depolarisation. A Markov chain Monte Carlo simulation approach was adopted. The Metropolis algorithm was used to generate three parallel Markov chains and thus obtain a sampled posterior probability distribution for each of the DC-potential and ADC model parameters, together with a number of derived parameters. The latter were used in a subsequent threshold analysis. The analysis provided no evidence indicating a consistent and reproducible ADC threshold for anoxic depolarisation.