Impact of the Prevalence of Cognitive Impairment on the Accuracy of the Montreal Cognitive Assessment: The Advantage of Using two MoCA Thresholds to Identify Error-prone Test Scores.

OBJECTIVES: The focus of this study is the classification accuracy of the Montreal Cognitive Assessment (MoCA) for the detection of cognitive impairment (CI). Classification accuracy can be low when the prevalence of CI is either high or low in a clinical sample. A more robust result can be expected when avoiding the range of test scores within which most classification errors are expected, with adequate predictive values for more clinical settings. METHODS: The classification methods have been applied to the MoCA data of 5019 patients in the Uniform Data Set of the University of Washington's National Alzheimer's Coordinating Center, to which 30 Alzheimer Disease Centers (ADCs) contributed. RESULTS: The ADCs show sample prevalence of CI varying from 0.22 to 0.87. Applying an optimal cutoff score of 23, the MoCA showed for only 3 of 30 ADCs both a positive predictive value (PPV) and a negative predictive value (NPV) ≥0.8, and in 18 cases, a PPV ≥0.8 and for 13 an NPV ≥0.8. Overall, the test scores between 22 and 25 have low odds of true against false decisions of 1.14 and contains 55.3% of all errors when applying the optimal dichotomous cut-point. Excluding the range 22 to 25 offers higher classification accuracies for the samples of the individual ADCs. Sixteen of 30 ADCs showed both NPV and PPV ≥0.8, 25 show a PPV ≥0.8, and 21 show an NPV ≥0.8. CONCLUSION: In comparison to a dichotomous threshold, considering the most error-prone test scores as uncertain enables a classification that offers adequate classification accuracies in a larger number of clinical settings.

The Clinical Relevance of Methods for Handling Inconclusive Medical Test Results: Quantification of Uncertainty in Medical Decision-Making and Screening.

BACKGROUND: although the existence of inconclusive medical test results or bio-markers is widely recognized, there are indications that this inherent diagnostic uncertainty is sometimes ignored. This paper discusses three methods for defining and determining inconclusive medical test results, which use different definitions and differ in clinical relevance. METHODS: the TG-ROC (two graphs receiver operating characteristics) method is the easiest to use, while the grey zone method and the uncertain interval method require more extensive calculations. RESULTS: this paper discusses the technical details of the methods, as well as advantages and disadvantages for their clinical use. TG-ROC and the grey zone method can help in the acquisition of high rates of diagnostic certainty, but can exclude large groups. The uncertain interval method can prevent decisions that are the most uncertain, invalid and unreliable, while excluding smaller groups. CONCLUSIONS: the identification of uncertain test scores is relevant, because these scores indicate the need to obtain better information or to await further developments. The methods presented help to determine inconclusive test scores and can help to reduce erroneous decisions. However, further research and development is desirable.

Interval of Uncertainty: An Alternative Approach for the Determination of Decision Thresholds, with an Illustrative Application for the Prediction of Prostate Cancer.

Often, for medical decisions based on test scores, a single decision threshold is determined and the test results are dichotomized into positive and negative diagnoses. It is therefore important to identify the decision threshold with the least number of misclassifications. The proposed method uses trichotomization: it defines an Uncertain Interval around the point of intersection between the two distributions of individuals with and without the targeted disease. In this Uncertain Interval the diagnoses are intermixed and the numbers of correct and incorrect diagnoses are (almost) equal. This Uncertain Interval is considered to be a range of test scores that is inconclusive and does not warrant a decision. It is expected that defining such an interval with some precision, prevents a relatively large number of false decisions, and therefore results in an increased accuracy or correct classifications rate (CCR) for the test scores outside this Uncertain Interval. Clinical data and simulation results confirm this. The results show that the CCR is systematically higher outside the Uncertain Interval when compared to the CCR of the decision threshold based on the maximized Youden index. For strong tests with a very small overlap between the two distributions, it can be difficult to determine an Uncertain Interval. In simulations, the comparison with an existing method for test-score trichotomization, the Two-graph Receiver Operating Characteristic (TG-ROC), showed smaller differences between the two distributions for the Uncertain Interval than for TG-ROC's Intermediate Range and consequently a more improved CCR outside the Uncertain Interval. The main conclusion is that the Uncertain Interval method offers two advantages: 1. Identification of patients for whom the test results are inconclusive; 2. A higher estimated rate of correct decisions for the remaining patients.

Unbalanced 2 x 2 factorial designs and the interaction effect: a troublesome combination.

In this power study, ANOVAs of unbalanced and balanced 2 x 2 datasets are compared (N = 120). Datasets are created under the assumption that H1 of the effects is true. The effects are constructed in two ways, assuming: 1. contributions to the effects solely in the treatment groups; 2. contrasting contributions in treatment and control groups. The main question is whether the two ANOVA correction methods for imbalance (applying Sums of Squares Type II or III; SS II or SS III) offer satisfactory power in the presence of an interaction. Overall, SS II showed higher power, but results varied strongly. When compared to a balanced dataset, for some unbalanced datasets the rejection rate of H0 of main effects was undesirably higher. SS III showed consistently somewhat lower power. When the effects were constructed with equal contributions from control and treatment groups, the interaction could be re-estimated satisfactorily. When an interaction was present, SS III led consistently to somewhat lower rejection rates of H0 of main effects, compared to the rejection rates found in equivalent balanced datasets, while SS II produced strongly varying results. In data constructed with only effects in the treatment groups and no effects in the control groups, the H0 of moderate and strong interaction effects was often not rejected and SS II seemed applicable. Even then, SS III provided slightly better results when a true interaction was present. ANOVA allowed not always for a satisfactory re-estimation of the unique interaction effect. Yet, SS II worked better only when an interaction effect could be excluded, whereas SS III results were just marginally worse in that case. Overall, SS III provided consistently 1 to 5% lower rejection rates of H0 in comparison with analyses of balanced datasets, while results of SS II varied too widely for general application.

Assignment refusal and its relation to outcome in a randomized controlled trial comparing Cognitive Therapy and Fluvoxamine in treatment-resistant patients with obsessive compulsive disorder.

The effectiveness of Fluvoxamine was compared to that of Cognitive Therapy (CT) in a 12-week randomized controlled trial (RCT) in 48 patients with obsessive-compulsive disorder (OCD), who were treatment-resistant to a previous behavior therapy (BT). A considerable amount of patients did not comply with the assigned treatment and switched treatments. The aim of this study was to identify patient characteristics predictive of assignment compliance and to study whether these characteristics were related to outcome. A logistic model, based on psychological and social patient characteristics, in addition to or in interaction with the assignment, was used for the explanation of compliance with treatment assignment. Especially patients who have a higher score on the Yale-Brown Obsessive Compulsive Scale (Y-BOCS) tend to comply with the effective Fluvoxamine treatment. The same set of variables was related to both compliance and outcome of therapy received. Therefore, the logistic model of compliance could be used to reduce the positive bias of As-Treated analysis (AT). The difference between the results of Fluvoxamine and Cognitive Therapy remained statistically significant after correcting for the positive bias as the result of assignment refusal and after applying the assumption that two drop-out patients needed imputation of lesser results.

Male and female development of delinquency during adolescence and early adulthood: a differential autoregressive model of delinquency using an overlapping cohort design.

Although it is well known that during adolescence the delinquent involvement of females is consistently less when compared to male involvement, it remains an important question whether the development of delinquency has a similar trajectory for both sexes. The main hypothesis tested is whether sex differences in delinquency, specifically growth, peak age, and decline, are constant. An autoregression model in continuous time, implemented as a structural equation model, is used for the description of the development of delinquency in males and females. The data are collected in an overlapping cohort design, and both within-person and between-persons data are integrated into a single model. The result shows that the involvement with delinquency over time is different for males and females. The main difference increases up to the age of 16, and decreases thereafter. The model indicates that both sexes reach the maximum in delinquency at the same age. It is concluded that males and females differ both in their start level at age 12 and in the amount of change with age.

Male and female delinquency trajectories from pre through middle adolescence and their continuation in late adolescence.

This study of male and female adolescent delinquency trajectories focuses on the prediction of late adolescence delinquency, based on earlier delinquency and social support. In this 3-wave longitudinal survey, 270 Dutch adolescents (113 males and 157 females) ages 12 to 14, were followed for a period of 6 years. For males, the level of delinquent activity in late adolescence strongly depends on earlier delinquent activities (R2 = .33, p < .0005). In contrast, the level of female delinquency in late adolescence is far less predictable (R2 = .18, p < .001), and could not be predicted from delinquent activities during pre and early adolescence, while support from the mother during late adolescence was associated with reduced delinquency for females. Different models may be needed to explain the development of delinquency for males versus females.

Fractional Design Wizard: a computer program for cost-effective experimental research design.

The computer program Fractional Design Wizard creates fractional factorial designs that are cost-effective and especially useful for discarding irrelevant factors from a large number of possible candidates. The program is intended for researchers who are relatively new to the field of fractional design and who want to acquaint themselves with the use of fractions for the reduction of large experimental designs. Fractional designs allow estimation of main effects, and sometimes two-way interactions, without one's having to examine all treatment conditions. The program needs Microsoft Windows 95 or better and 32 MB of memory. In a step-by-step fashion, the user can specify the required properties of the fractional design. When there are more valid designs, the user can generate these successively. If necessary, the user can go back to diminish the requirements. The output can be copied, printed, and saved. The program generates all the information that is needed for the use and interpretation of fractional designs. A help file explains the use of the program and also the purpose, the analysis, and the interpretation of fractional designs. The program, which is written in Object Pascal, is available as freeware on www.fss.uu.nl/ms/hl/fracdes.htm.