RESUMO
The Hosmer-Lemeshow (HL) test is a commonly used global goodness-of-fit (GOF) test that assesses the quality of the overall fit of a logistic regression model. In this paper, we give results from simulations showing that the type I error rate (and hence power) of the HL test decreases as model complexity grows, provided that the sample size remains fixed and binary replicates (multiple Bernoulli trials) are present in the data. We demonstrate that a generalized version of the HL test (GHL) presented in previous work can offer some protection against this power loss. These results are also supported by application of both the HL and GHL test to a real-life data set. We conclude with a brief discussion explaining the behavior of the HL test, along with some guidance on how to choose between the two tests. In particular, we suggest the GHL test to be used when there are binary replicates or clusters in the covariate space, provided that the sample size is sufficiently large.
RESUMO
Generalized linear models (GLMs) are very widely used, but formal goodness-of-fit (GOF) tests for the overall fit of the model seem to be in wide use only for certain classes of GLMs. We develop and apply a new goodness-of-fit test, similar to the well-known and commonly used Hosmer-Lemeshow (HL) test, that can be used with a wide variety of GLMs. The test statistic is a variant of the HL statistic, but we rigorously derive an asymptotically correct sampling distribution using methods of Stute and Zhu (Scand J Stat 29(3):535-545, 2002) and demonstrate its consistency. We compare the performance of our new test with other GOF tests for GLMs, including a naive direct application of the HL test to the Poisson problem. Our test provides competitive or comparable power in various simulation settings and we identify a situation where a naive version of the test fails to hold its size. Our generalized HL test is straightforward to implement and interpret and an R package is publicly available. Supplementary Information: The online version contains supplementary material available at 10.1007/s11749-023-00912-8.
RESUMO
OBJECTIVE: The aim of this study was to develop and validate a brief disability screen for children with JIA, the Kids Disability Screen (KDS). METHODS: A total of 216 children enrolled in the Canadian Alliance of Pediatric Rheumatology Investigators (CAPRI) Registry in 2017-2018 formed a development cohort, and 220 children enrolled in 2019-2020 formed a validation cohort. At every clinic visit, parents answered two questions derived from the Childhood Health Assessment Questionnaire (CHAQ): 'Is it hard for your child to run and play BECAUSE OF ARTHRITIS?' ('Hard' 0-10), and 'Does your child usually need help from you or another person BECAUSE OF ARTHRITIS?' ('Help', 0-10). We used 36-fold cross-validation and tested nine different mathematical methods to combine the answers and optimize psychometric properties. The results were confirmed in the validation cohort. RESULTS: Expressed as the mean of the two answers, KDS best balanced ease of use and psychometric properties, while a LASSO regression model combining the two answers with other patient characteristics [estimated CHAQ [eCHAQ]) had the highest responsiveness. In the validation cohort, 22.7%, 25.9% and 28.6% of patients had a score of 0 at enrolment for the KDS, eCHAQ and CHAQ, respectively. Responsiveness was 0.67, 0.74 and 0.62, respectively. Sensitivity to detect a CHAQ > 0 was 0.90 and specificity 0.56, KDS detecting some disability in 44% of children with a CHAQ = 0. CONCLUSION: This simple KDS has psychometric properties comparable with those of a full CHAQ and may be used at every clinic visit to identify those children who need a full disability assessment.
