An investigation of new toxicity test method performance in validation studies: 1. Toxicity test methods that have predictive capacity no greater than chance.

An approach commonly used to measure new toxicity test method (NTM) performance in validation studies is to divide toxicity results into positive and negative classifications, and the identify true positive (TP), true negative (TN), false positive (FP) and false negative (FN) results. After this step is completed, the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are calculated. Although these statistics are widely used and often the only statistics used to assess the performance of toxicity test methods, there is little specific guidance in the validation literature on what values for these statistics indicate adequate performance. The purpose of this study was to begin developing data-based answers to this question by characterizing the CPS obtained from an NTM whose data have a completely random association with a reference test method (RTM). Determining the CPS of this worst-case scenario is useful because it provides a lower baseline from which the performance of an NTM can be judged in future validation studies. It also provides an indication of relationships in the CPS that help identify random or near-random relationships in the data. The results from this study of randomly associated tests show that the values obtained for the statistics vary significantly depending on the cut-offs chosen, that high values can be obtained for individual statistics, and that the different measures cannot be considered independently when evaluating the performance of an NTM. When the association between results of an NTM and RTM is random the sum of the complementary pairs of statistics (sensitivity + specificity, NPV + PPV) is approximately 1, and the prevalence (i.e., the proportion of toxic chemicals in the population of chemicals) and PPV are equal. Given that combinations of high sensitivity-low specificity or low specificity-high sensitivity (i.e., the sum of the sensitivity and specificity equal to approximately 1) indicate lack of predictive capacity, an NTM having these performance characteristics should be considered no better for predicting toxicity than by chance alone.

An investigation of new toxicity test method performance in validation studies: 2. Comparison of three measures of toxicity test performance.

An area that requires further research is how best to measure test method performance in validation studies and how to set criteria that should be used to judge the adequacy of this performance. The studies reported here were designed to begin an investigation of these questions. Computer simulations were used to generate data sets similar to those that might be obtained from a large validation study. These data were then analysed using three procedures including determination of the 95% prediction interval (PI), calculation of Pearson's correlation coefficient and calculation of the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The results of this work suggest that of the three approaches examined, quantitative measurements with calculation of the 95% PI provide the most information to allow discrimination between the performance of several different NTMs. The results also suggest that dividing data sets into positive and negative toxicity classifications followed by the calculation of CPS leads to considerable information loss. This loss of information may be so significant that it is not possible in certain circumstances to distinguish between NTMs that are adequate and those that are not.

An investigation of new toxicity test method performance in validation studies: 3. Sensitivity and specificity are not independent of prevalence or distribution of toxicity.

Often, the only measures of toxicity test performance provided in validation studies are the contingent probability statistics (CPS) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity and specificity are generally used in preference to NPV and PPV since NPV and PPV are assumed to vary with changes in prevalence while sensitivity and specificity are assumed to be independent of changes in prevalence. The purpose of the studies reported here was to test whether or not sensitivity and specificity are actually independent of changes in prevalence. Results derived from these studies indicate that sensitivity and specificity vary significantly depending on the prevalence of toxic substances in the set of chemicals being tested. This means sensitivity and specificity should not always be considered constant indicators of toxicity test performance.

Prediction of eye irritation from organic chemicals using membrane-interaction QSAR analysis.

Eye irritation potency of a compound or mixture has traditionally been evaluated using the Draize rabbit-eye test (Draize et al., 1944). In order to aid predictions of eye irritation and to explore possible corresponding mechanisms of eye irritation, a methodology termed "membrane-interaction QSAR analysis" (MI-QSAR) has been developed (Kulkarni and Hopfinger 1999). A set of Draize eye-irritation data established by the European Center for Ecotoxicology and Toxicology of Chemicals (ECETOC) (Bagley et al., 1992) was used as a structurally diverse training set in an MI-QSAR analysis. Significant QSAR models were constructed based primarily upon aqueous solvation-free energy of the solute and the strength of solute binding to a model phospholipid (DMPC) monolayer. The results demonstrate that inclusion of parameters to model membrane interactions of potentially irritating chemicals provides significantly better predictions of eye irritation for structurally diverse compounds than does modeling based solely on physiochemical properties of chemicals. The specific MI-QSAR models reported here are, in fact, close to the upper limit in both significance and robustness that can be expected for the variability inherent to the eye-irritation scores of the ECETOC training set. The MI-QSAR models can be used with high reliability to classify compounds of low- and high-predicted eye irritation scores. Thus, the models offer the opportunity to reduce animal testing for compounds predicted to fall into these two extreme eye-irritation score sets. The MI-QSAR paradigm may also be applicable to other toxicological endpoints, such as skin irritation, where interactions with cellular membranes are likely.

Eye Irritation Testing: The Way Forward. The Report and Recommendations of ECVAM Workshop 34.

This is the report of the thirty-fourth of a series of workshops organised by the European Centre for the Validation of Alternative Methods (ECVAM). ECVAM's main goal, as defined in 1993 by its Scientific Advisory Committee, is to promote the scientific and regulatory acceptance of alternative methods which are of importance to the biosciences and which reduce, refine or replace the use of laboratory animals. One of the first priorities set by ECVAM was the implementation of procedures which would enable it to become well-informed about the state-of-the-art of non-animal test development and validation, and the potential for the possible incorporation of alternative tests into regulatory procedures. It was decided that this would be best achieved by the organisation of ECVAM workshops on specific topics, at which small groups of invited experts would review the current status of various types of in vitro tests and their potential uses, and make recommendations about the best ways forward (1). The workshop on Eye Irritation Testing: The Way Forward was held in Egham, UK, on 15-17 June 1998, under the chairmanship of Michael Balls (ECVAM, Italy). The workshop had two aims, the first of which was to review some of the previous multi-laboratory validation studies on alternatives to the Draize eye test and assess why many promising alternative methods were not successful in these studies. The second aim was to discuss strategies for making progress toward the short-term reduction, refinement, and eventual replacement, of the Draize test, including: a new approach to the validation of in vitro tests for eye irritancy, based on the use of reference standards, which promises to overcome some of the problems encountered in previous studies; the use of stepwise testing strategies which reduce and refine the use of animals in eye irritation testing; the use of multivariate and other statistical techniques for the further analysis of data generated in previous validation studies; and a programme of research aimed at understanding the underlying mechanisms of eye irritation.

Workshop overview: scientific and regulatory challenges for the reduction, refinement, and replacement of animals in toxicity testing.

Public concern for animal welfare has been expressed through legislative control of animal use for experimental purposes since the first legislation was introduced in 1876 in the United Kingdom. Legislative control of animal use has been introduced in virtually every developed country, with major initiatives in Europe (1986) and the United States (1966 and 1985). Advances in scientific thinking resulted in the development of the concept of the three Rs--refinement, reduction, and replacement--by Russell and Burch in 1959. The field has expanded substantially since, with specialist scientific journals dedicated to alternatives, World Congresses organized to discuss the scientific and philosophical issues, and European and U.S. validation organizations being launched. Current scientific attention is focused on validation of alternative methods. The underlying scientific principles of chemical toxicity are complicated and insufficiently understood for alternative methods for all toxicity endpoints of importance in protecting human health to be available. Important lessons have been learned about how to validate methods, including the need to have prediction models available before the validation is undertaken, the need to understand the variability of the animal-based data which is to be used as the validation standard, and the need to have well-managed validation programs. Future progress will depend on the development of novel methods, which can now be validated through international collaborative efforts.

13th Meeting of the Scientific Group on Methodologies for the Safety Evaluation of Chemicals (SGOMSEC): validation and acute toxicity testing.

Scientific principles demand that before newly developed alternative methods for safety testing are fully embraced by the industrial or regulatory community, they reliably and reproducibly predict the designated toxic end point. The process used to determine reliability and reproducibility is termed validation, and it generally culminates with a highly controlled, blinded study using multiple chemicals and laboratories. It is imperative that the validation study is designed to confirm the previously established reproducibility and predictive power of the assay. Much has been learned recently about the practical aspects of validation through investigation of alternative methods for acute toxicity testing, i.e., those methods that assess acute systemic toxicity, skin irritation, and eye irritation. Although considerable progress has been made--many alternative tests are now commonly used in various industrial settings--there have been few tests that have successfully passed a complete validation. Some of the barriers to successful validation have been a) lack of high-quality, reproducible animal data; b) insufficient knowledge of the fundamental biologic processes involved in acute toxicity; and c) the development of truly robust in vitro assays that can accurately respond to materials with a wide range of chemical and physical characteristics. It is recommended that to progress in the areas of eye and skin irritation we need to expand our knowledge of toxic markers in humans and the biochemical basis of irritation; progress in the area of acute systemic toxicity will require the development of in vitro models to determine gastrointestinal uptake, blood-brain barrier passage, and biotransformation.

Validation of alternative methods for toxicity testing.

Before nonanimal toxicity tests may be officially accepted by regulatory agencies, it is generally agreed that the validity of the new methods must be demonstrated in an independent, scientifically sound validation program. Validation has been defined as the demonstration of the reliability and relevance of a test method for a particular purpose. This paper provides a brief review of the development of the theoretical aspects of the validation process and updates current thinking about objectively testing the performance of an alternative method in a validation study. Validation of alternative methods for eye irritation testing is a specific example illustrating important concepts. Although discussion focuses on the validation of alternative methods intended to replace current in vivo toxicity tests, the procedures can be used to assess the performance of alternative methods intended for other uses.

Synchronization of pacemaker cell firing by weak ELF fields: simulation by a circuit model.

Entrainment of output action potentials from repetitively firing pacemaker cells, brought about by regularly spaced excitatory or inhibitory postsynaptic inputs, is a well-known phenomenon. Synchronization of neural firing patterns by extremely low frequency (ELF) external electric fields has also been observed. Whereas current densities of approximately 10 A-m(-2) are required for direct excitation of otherwise quiescent neural tissue, much lower peak current densities (approximately 10[-2] A-m2) have been reported to entrain spontaneously firing molluscan pacemaker cells. We have developed a neural spike generator circuit model that simulates repetitive spike generation by a space clamped patch (area approximately 10[-7] m2) of excitable membrane subjected to depolarizing current. Picoampere (pA) range variation of DC depolarizing current causes a corresponding smooth variation of neural spike frequency, producing a physiologically realistic stimulus-response (S-R) characteristic. When lower pA range 60 Hz AC current is superposed upon the DC depolarizing current, smooth variation of the S-R characteristic is distorted by subharmonic locking of the spike generator at 30, 20, 15, 12, 10 Hz, and higher order subharmonic frequencies. Although the additional superposition of a physiologically realistic level of "white" current noise, covering the bandwidth 4-200 Hz, suffices to obscure higher order subharmonic locking, locking at 30, 20, and 15 Hz is still clearly evident in the presence of noise. Subharmonic locking is observed at an root mean square AC simulated tissue current density of approximately 10(-5) A-m(-2).

Investigation of ingredient interactions in cosmetic formulations using isolated bovine corneas.

The purpose of this paper is to report on use of a modified bovine cornea opacity and permeability assay (BCOP) to test the effects of several cosmetic formulations on eye-derived tissue in vitro. The results from these studies suggest that a BCOP protocol using prolonged exposure and repeated treatments may be useful for screening the eye effects of cosmetic formulations. Further work will be required, however, before the model is ready for formal validation. This series of experiments also provides an example of where the toxicity of one ingredient was significantly changed by its interaction with other ingredients in a mixture. As it was not possible to predict the highly reactive nature of the formulation in vitro based on an evaluation of ingredient toxicity data alone, this case illustrates the importance of obtaining adequate safety testing data on innovative mixtures of cosmetic ingredients before human exposure is allowed.

Report on the COLIPA Workshop on Mechanisms of Eye Irritation.

This report summarises the discussions of a workshop sponsored by the European Cosmetics, Toiletries and Perfumery Association (COLIPA). The workshop discussed the state-of-the-art of eye irritancy testing, and made recommendations as to the best ways in which to validate alternatives to the Draize eye irritation test. The importance of understanding the mechanisms of eye irritation, particularly when attempting to improve in vitro prediction of in vivo eye irritancy, was also emphasised.

Perspectives on alternatives to the eye irritation test: industry, public interest, government.

Government mandates are requiring serious consideration of alternatives to animal testing. For eye irritation testing, many non-whole animal alternatives exist that now need to be assessed as to their validity in replacing the animal model. The best promise for identifying useful alternatives comes from using both statistical and biological factors to evaluate results from formal validation studies. Industry submissions of side-by-side animal and alternative test results are also important. Empirical test results should be scrutinized first; mechanistic studies should follow, as needed. Co-operation is required by all parties to develop internationally harmonized test protocols and hazard classification systems.

A summary report of the COLIPA international validation study on alternatives to the draize rabbit eye irritation test.

The principal goal of this study was to determine whether the results from a set of selected currently available alternative methods as used by cosmetics companies are valid for predicting the eye irritation potential of cosmetics formulations and ingredients and, as a consequence, could be valid replacements for the Draize eye irritation test. For the first time in a validation study, prediction models (PMs) that convert the in vitro data from an assay to a prediction of eye irritation were developed for each alternative method before the study began. The PM is an unequivocal description of the relationship between the in vitro and the in vivo data and allows an objective assessment of the reliability and relevance of the alternative methods. In this study, 10 alternative methods were evaluated using 55 test substances selected as representative of substances commonly used in the cosmetics industry (23 ingredients and 32 formulations). Twenty of the single ingredients were common to the European Commission/British Home Office (EC/HO) eye irritation validation study (Balls et al., 1995b). The test substances were coded and supplied to the participating laboratories. The results were collected centrally and analysed independently, using statistical methods that had been agreed before the testing phase began. Each alternative method was then evaluated for reliability and relevance in assessing eye irritation potential. Using the criteria of both reliability and relevance as defined in the study, the preliminary results indicate that none of the alternative methods evaluated could be confirmed as a valid replacement for the Draize eye irritation test across the full irritation scale. However, three alternative methods-the fluorescein leakage test, the red blood cell assay (classification model) and the tissue equivalent assay-each satisfied one criterion of reliability or relevance. Further investigation of the decoded data from this study to explore more fully the relationship between the in vitro data and the in vivo data is recommended. Such a review may allow the development of new prediction models to be tested in a subsequent validation study.

Validation of alternative methods for toxicity testing.

Many studies have been conducted in order to assess the validity of alternative methods as replacements forin vivo toxicity tests. The purpose of this review is to build on what has been learned in the course of this work by presenting a practical process that can be used to conduct future validation programmes. The important role of a clearly stated prediction model, which defines how to use the results from an alternative method to predict anin vivo toxicity endpoint, has been introduced. Computer simulations have been used to demonstrate that data-based guidance can be developed to assist in judging the performance of alternative methods assessed in a validation study. Additionally, statistical procedures have been used in order to provide guidance on choosing the appropriate number of reference test substances and number of participating laboratories to include in a validation study. The validation of alternative methods for eye irritation testing is used as a specific example to illustrate important concepts. Although the focus of the discussion is on the validation of alternative methods intended to replace currentin vivo tests, the procedures can be used to assess the performance of alternative methods intended for other uses. This review will be particularly useful to those who require a practical guide for conducting a validation study and to those who must assess the results of such programmes.

Diabetes control programs: new directions.

The diabetes control programs (DCPs), which involve state and territorial health agencies working cooperatively with the Centers for Disease Control and Prevention, offer new opportunities for diabetes educators to expand their scope of practice into public health. The intervention activities of the 40 core-capacity DCPs that are proposed for the next fiscal year are summarized in this paper to provide insight into the new directions of the DCPs. These activities span a range of health-system and community-based approaches and are described by their potential impact on the audience. The interventions are aimed at a variety of intended audiences and partners, including minorities, individuals with diabetes, healthcare providers, and policymakers. As diabetes educators continue to expand their scope of practice, the DCPs offer opportunities to go beyond individual practice behaviors by participating in community development activities, disseminating practice guidelines, changing organizational practices, and advocating for health-related public policy and legislation.

Alternatives in cosmetics testing.

This paper represents a summary of presentations made during a round-table discussion at the ECVAM Opening Symposium. After introductory comments on the cosmetic industry's use of alternative methods in the safety assessment process, the use of alternative methods by L'Oréal and by the Japanese cosmetic industry is outlined, current validation studies in Japan are noted, and the involvement of COLIPA, the European Cosmetic, Toiletry and Perfumery Association, in promoting the use of alternative methods is discussed. Two final sections deal with the effect of data variability on the performance of alternative methods in validation studies and on the integrated use of quantitative structure-activity relationship (QSAR) analysis with other approaches in the safety assessment process.

The EC/HO international validation study on alternatives to the draize eye irritation test.

This is the final report of the Management Team for a European Commission/British Home Office (EC/HO) validation study on alternatives to the Draize eye irritation test. The principal goal of the study was to establish whether one or more of nine non-animal tests could be used to replace the Draize test for all severely irritating materials (or those belonging to specific classes) or the animal test completely for chemicals with or without regard to chemical class. Sixty chemicals were independently selected, coded and supplied, then the data obtained in 37 laboratories were analysed independently. The results of comparisons between 27 alternative test index scores and the Modified Maximum Average Scores (MMASs) obtained in the Draize eye test were compared. Tables of results showing Pearson's product moment correlation coefficients and Spearman's rank coefficients for each laboratory are provided, and correlation matrices of alternative test index scores among the different groups of laboratories are shown for each endpoint. Scatterplots are provided, in which the alternative test scores obtained by the lead laboratories for the nine tests are plotted against the MMAS for the full set of chemicals and 12 surfactants. It is concluded that, with the possible exception of predicting the irritancy of surfactants, none of the nine tests met any of the four performance targets. Possible reasons for this outcome are discussed.

Reducing the number of rabbits in the low-volume eye test.

Although the Draize eye irritation test has provided important and useful information for eye safety assessments, considerable effort has been directed toward refining the assay procedure, reducing the number of animals used, and replacing this assay with alternative methods. The low-volume eye test (LVET) is a refinement of the Draize eye irritation test that uses 1/10 the volume of test substance placed directly on the cornea. The level and duration of eye irritation in the LVET are less than those in the Draize procedure, which means that it is a less stressful test. Furthermore, LVETs are more predictive of human response. Statistical studies have been conducted to determine the effects of reducing the number of animals used in the Draize test. These results suggested that a three-animal test would provide essentially the same information as the six-animal test. A similar analysis has not been performed on results from the LVET. Accordingly, the present study was undertaken to evaluate previously existing LVET data to determine if the number of animals used in a LVET can be decreased as has been shown for the Draize test. The results of the analysis are consistent with the findings of earlier evaluations of classical Draize data. Three-animal subsets from 119 six-animal LVETs provided the correct classification greater than 92% of the time for three different classification schemes. Furthermore, the discrepancies between the three-animal subsets and the six-animal maximum average score tended to be smaller than those observed for the Draize test.(ABSTRACT TRUNCATED AT 250 WORDS)