A comparison of variability in Papanicolaou and liquid-based cytology inadequacy rates using Shewhart control charts.

OBJECTIVE: To use Shewhart control charts to compare variability in inadequacy rates from Papanicolaou (Pap) and liquid-based cytology (LBC). DESIGN: Retrospective analysis of quality assurance data. SETTING: Eleven Welsh cytology laboratories. METHODS: Shewhart 'p' charts were plotted for proportions of slides reported as inadequate. Charts were compared for statistical control. MAIN OUTCOME MEASURES: Evidence of statistical control in the processes. RESULTS: Control charts allowed easy interpretation of patterns in the data. Variability in inadequacy rates was much lower for LBC than for Pap cytology. CONCLUSION: Monitoring inadequate rates with Shewhart charts provides more information than tabular monitoring reports, assisting in quality improvement. With respect to inadequacy rates, LBC is less variable than Pap cytology.

Delta-NT or NT MoM: which is the most appropriate method for calculating accurate patient-specific risks for trisomy 21 in the first trimester?

OBJECTIVE: To assess whether in screening for trisomy 21 by nuchal translucency (NT) the delta or the multiples of the median (MoM) approach is the most appropriate method for calculating accurate individual patient-specific risks. METHODS: Data on fetal NT and crown-rump length from 128,030 unaffected and 428 trisomy 21 pregnancies, measured by sonographers who had obtained The Fetal Medicine Foundation Certificate of Competence in the 11-14-Week Scan, were used. We examined first, if the distribution of NT MoM and log(10)(NT MoM) was Gaussian; second, if the standard deviation of the distributions did not change with gestation; and third, if the median MoM in the affected population was a constant proportion of the median for unaffected pregnancies. All of these features are required to underpin the MoM approach. NT distributions and those of delta-NT were also analyzed. A non-parametric kernel density method was then used to assess the validity of both methods. Errors in the estimation of individual patient-specific risks using the MoM approach were assessed. RESULTS: In the unaffected pregnancies, the distributions of NT MoM and log(10)(NT MoM) were not Gaussian and the standard deviation of log(10)(NT MoM) decreased with gestation. In the trisomy 21 pregnancies, the median NT MoM decreased significantly with gestation, whereas the median delta-NT did not change with gestation. The non-parametric density approach showed that contours of constant likelihood ratio were parallel to the gestational age-dependent median NT values, thus supporting the delta-NT approach. The NT MoM approach resulted in women being given an overestimate of risk for trisomy at 11 weeks and a considerable underestimate of risk at 13 weeks. CONCLUSION: In the calculation of risk for trisomy 21 by NT the NT MoM approach is inaccurate and inappropriate because the underlying assumptions are not valid. In contrast, the delta-NT approach gives accurate estimates of risks.

The effect of temporal variation in biochemical markers of trisomy 21 across the first and second trimesters of pregnancy on the estimation of individual patient-specific risks and detection rates for Down's syndrome.

BACKGROUND: In a previous study we examined the changes in the median multiple of the median (MoM) with gestation of free beta human chorionic gonadotrophin (F beta-hCG), total human chorionic gonadotrophin (ThCG), alpha-fetoprotein (AFP) and pregnancy-associated plasma protein A (PAPP-A) in a large series of Down's syndrome pregnancies. Results showed that there was a significant temporal variation of the MoM for each marker. In this paper, we assess the impact of this temporal shift on the estimation of patient-specific risks and the detection rates (DRs) for Down's syndrome pregnancies. METHODS: Individual patient-specific risks, DRs and false positive rates were estimated using statistical modelling techniques and computer simulations. The data for these simulations were the regressed mean log(10) analyte MoMs, marker standard deviations (as log(10) MoM) and correlation coefficients derived from the analysis of over 1000 cases of Down's syndrome and 150,000 unaffected pregnancies between 6 and 20 weeks of gestation reported in our previous study. Two models were compared: the classical constant median separation model, which assumes no variation in median shift with gestation (model 1), and a variable median separation model (model 2), which takes account of the changes in median shift with gestation as described in our previous study. RESULTS: When individual patient-specific risks calculated for various MoM values using model 1 were compared with those derived from model 2, considerable differences in risk estimates were observed for all marker combinations, particularly in the first trimester. Using a 1 in 250 cut-off risk, DRs at each gestation in the second trimester for the AFP+F beta-hCG combination were maximized at 14-17 weeks of gestation and were virtually identical at 63-65% for model 1 and model 2. A similar trend was observed for the AFP+ThCG combination, with an optimum gestational range of 15-18 weeks and DRs of 66-68%. In the first trimester, using a 1 in 250 cut-off risk, DRs were more variable with gestation for the prime marker combination of F beta-hCG+PAPP-A, varying from 73% at 8 weeks to 65% at 13 weeks with model 1 and from 75% to 66% with model 2. CONCLUSION: Risk algorithms should take into account temporal variation in marker MoMs in order to produce accurate patient-specific risks. This also helps to maximize DRs, particularly when samples are taken out with the optimal gestational range.

Bias in risk estimation: application to Down's syndrome screening.

In this paper we consider the bias associated with parametric estimation of a univariate or bivariate Gaussian density, and also the induced bias when these Gaussian densities are used to determine a likelihood ratio. Algebraic approximations are derived that accurately predict the relative biases obtained, verification being achieved by a simulation exercise. The expressions confirm that when estimating a univariate Gaussian density there are four Z-scores for which there is zero bias and that relative bias increases rapidly beyond two standard deviations from the mean. The results are then extended to determine approximate confidence intervals for both the true density and the likelihood ratio. A simulation exercise confirms that the derived 95 per cent confidence intervals have coverage that ranges from 94 to 97 per cent. The results are applied to a Down's syndrome screening programme where 95 per cent confidence intervals are established for a woman's posterior odds of carrying a Down's foetus. It is shown that patients with similar posterior odds can give rise to confidence intervals for their true posterior odds that have very different widths, thus emphasizing that not all risks are of equal quality.

Temporal changes in maternal serum biochemical markers of trisomy 21 across the first and second trimester of pregnancy.

BACKGROUND: Many maternal serum markers show concentration changes in Down's syndrome pregnancies but the magnitude of the change in median marker levels varies with gestation. To date these changes have not been accurately specified. METHODS: The trends in marker median levels between 6 and 20 weeks of gestation were examined for alphafetoprotein (AFP), free beta human chorionic gonadotrophin (Fbeta-hCG), total human chorionic gonadotrophin (ThCG) and pregnancy-associated plasma protein A (PAPP-A) by a meta-analysis of data obtained from our collaborative studies and routine screening programmes for Down's syndrome over a 10-year period. Data were available from between 709 and 1082 Down's syndrome pregnancies and from between 14607 and 153909 unaffected pregnancies for each marker. The median multiple of the median (MoM) and mean log10MoM for each marker at each completed week of gestation were estimated and the trend with gestation smoothed using a weighted least squares regression model. RESULTS: The gestational ages at which maximum separation of marker levels occurred, comparing affected and unaffected pregnancies, and the respective regressed median MoMs and mean log10MoMs, were: for AFP at 16 weeks, 0.72 MoM, -0.14288log10MoM; for Fbeta-hCG at 15 weeks, 2-24MoM, 0.35034 log10MoM; for ThCG at 16 weeks, 1.93 MoM, 0.28548 log10MoM, as well as before 8 weeks (<0.65 MoM, -0.18853 log10MoM); and for PAPP-A before 8 weeks, <0.33 MoM, -0.47727 log10MoM. CONCLUSION: There is significant temporal variation in mean log10MoM values for the screening markers investigated. Screening algorithms, modified to take account of this variation, should allow more accurate gestation-specific risks to be calculated in individual pregnancies.

Correlation of marker values in Down's syndrome screening: the effect of dating error.

Central to any screening algorithm for Down's syndrome are the values used for the parameters in the multivariate Gaussian statistical model that is used to describe the joint distribution of the marker values. There has been much discussion about the values of the means and standard deviations which are appropriate but little interest has been shown in the values of the correlation coefficients between markers. There has been some speculation that the range of parameter values quoted in the literature arises from factors such as storage of samples, between-assay effects and differences in assay methodologies. We show that gestational dating error, among other factors, could be responsible for much of the variation that is present in quoted parameter values, though the factors mentioned above clearly have an effect.

Screening for Down's syndrome: the effect of test date on the detection rate.

Screening tests for Down's syndrome are carried out at different gestational ages. Because of fetal loss, crude estimates of their detection rates cannot be directly compared. We present methods for estimating the true detection rates along with their standard errors. This enables a proper statistical comparison of the true detection rates of tests carried out, for example, in different trimesters.

Detection rates and false positive rates for Down's syndrome screening: how precisely can they be estimated and what factors influence their value?

Down's syndrome screening is currently carried out using a combination of biochemical markers measured in maternal serum samples; these include MSAFP, Total hCG, uE3 and Free beta-hCG. Recently a number of papers have compared the effectiveness of different combinations of these markers. Some recommend MSAFP, Total hCG and uE3 (triple test) while others advocate MSAFP and Free beta-hCG (double test). The cases put forward to support these tests rely on estimated detection and false positive rates for the proposed test. A recent paper by Wright used simulation techniques to estimate the effects of sampling error on such error rates. In prospective studies there are two methods commonly used for estimating these rates. We obtain formulae for the standard deviations of these estimates and show that one of them always gives a smaller standard error than the other. We also show that in such studies the accuracy of estimating detection rates and false positive rates depends not only upon the method of calculation but also on the age distribution of pregnant women and the parameters used to calculate patient specific risk. We show that these effects can result in estimation errors of such magnitude that many observed differences in detection rates could be of questionable significance, a conclusion also reached by Wright.

The effects of gestation dating on the calculation of patient-specific risks in Down's syndrome screening: multivariate case.

When screening for Down's syndrome using biochemical markers, the measurements are adjusted for the gestational age of the fetus because the concentrations of the markers are known to change with gestational age. This adjustment is performed by referring each marker measurement to the population median for that marker for the appropriate estimated gestational age group. The measurement of gestational age is subject to error, whichever method is used, and so the population median used is usually the median of a mixture of distributions for different true gestational ages. Most screening programmes aim for a specific number of weeks and this produces a concentrated distribution of true gestational ages. This fact, combined with dating errors, leads to an asymmetric mixture for each gestational age group and hence to bias in the estimates of the medians. In a previous communication we have shown how the proportions in this mixture distribution can be estimated and how the true medians corresponding to a true gestational age can be estimated. The calculations presented were performed using a single marker, and the details of our method were restricted to this situation. This paper extends the method to the multimarker situation and, as expected, leads to a gain in the detection rate for a specified false positive rate. The true patient-specific risk estimates are again markedly different from the quoted nominal value obtained by ignoring the dating errors. The data set on which the method is illustrated uses two markers, although the technique generalises in an obvious way to more than two.

Comparison of quality-control rules used in clinical chemistry laboratories.

Numerous papers have been written to show which combinations of Shewhart-type quality-control charts are optimal for detecting systematic shifts in the mean response of a process, increases in the random error of a process, and linear drift effects in the mean response across the assay batch. One paper by Westgard et al. (Clin Chem 1977;23:1857-67) especially seems to have attracted the attention of users. Here we derive detailed results that enable the characteristics of the various Shewhart-type control schemes, including the multirule scheme (Clin Chem 1981;27:493-501), to be calculated and show that a fundamental formula proposed by Westgard et al. in the earlier paper is in error, although their derived results are not seriously wrong. We also show that, from a practical point of view, a suitably chosen Cusum scheme is near optimal for all the types and combinations of errors discussed, thereby removing the selection problem for the user.

Age-specific detection and false-positive rates: an aid to counseling in Down syndrome risk screening.

OBJECTIVE: To determine whether the serum screening test for Down syndrome provides equal detection efficacy for women of all ages, to improve the data available for patient counseling both before testing and afterward in the event of a positive result. METHODS: We examined the effect of age on Down screening by generating a set of "normal" and "Down syndrome" likelihood ratios by computer simulation. The expected false-positive and detection rates were derived for different age groups by counting the proportion of cases in which the likelihood ratio could modify the age-specific risk to be greater than the cutoff risk of one in 300 (equivalent to an incidence of 3.33 per 1000). The predictive value of a positive result was calculated using Baye's theorem. RESULTS: Detection rates, false-positive rates, and predictive values were shown to be age-dependent. CONCLUSIONS: Knowledge of the age dependency of Down syndrome screening results may be useful in explaining to patients that the Down screen can only detect a proportion of cases and that a negative result does not guarantee normality. This knowledge may also be helpful in minimizing psychological stress, as a positive result indicates only a small chance that the fetus will have Down syndrome.

Estimation of the mean and standard deviation using order statistics.

Estimation of the mean and standard deviation of a distribution using linear combinations of the order statistics of a censored sample is discussed. Existing methods are compared with the optimal estimator of this type and new approximations to the optimal estimator are proposed. Their performances in the presence of outliers are assessed by means of simulation.

Maximum likelihood techniques applied to method comparison studies.

One component of method comparison studies is the assessment of relative bias. Most authors use ordinary regression techniques for this purpose. The potential inadequacies in this approach are highlighted and a more generally applicable maximum likelihood technique proposed. The properties of the various methods of estimation are obtained by means of a simulation study, using a variety of different within-assay precision profiles.

Assay detection limits: concept, definition, and estimation.

This paper emphasizes the need for standardizing the definition and estimation of the minimum detectable analyte concentration in an assay. It reviews the key papers that are most frequently quoted in the literature to illustrate the variety of procedures currently in use.

Internal quality control in clinical chemistry: a teaching review.

This survey introduces the subject of internal quality assessment from a historical point of view, presents a unified approach to notation, concepts and definitions, and describes briefly those quality control procedures that are used most commonly in clinical chemistry. It is not the aim of this report to comment on all the individual contributions made in this field, but rather to concentrate on the principles. Particular emphasis is centered around assessment criteria to compare the efficiencies of selected control procedures for monitoring analyte concentrations in biological fluids. The question of whether to compare control schemes by means of average run lengths, run length distribution functions or average cost functions is considered. A rationalized approach based on the comparison of optimal procedures, using average run length, is adopted.

Dose-interpolation of immunoassay data: uncertainties associated with curve-fitting.

Estimates of analyte concentrations, obtained by immunoassay, have error distributions which are generally underestimated. Better estimates, which take into account the distribution of the response metameter of the calibration curve and uncertainties associated with the location of the fitted curve, have been obtained by computer simulation using practical immunoassay data. The increased use of curve-fitting software in automated immunoassay equipment has prompted the investigation of reported analyte concentration distributions for three commonly used functions, namely the logit-log, the 4-parameter logistic and the four-parameter Amersham models. The stimulation model is of potential value for a variety of situations.