Work experience and style explain variation among pediatricians in the detection of children with psychosocial problems.

OBJECTIVE: To assess whether variation in the proportion of children identified as having psychosocial problems by individual preventive pediatricians can be explained by pediatrician characteristics, over and above variations in the mix of children. Furthermore, to assess whether the characteristics of preventive pediatricians were related to the quality of problem identification. METHODS: We used data from approximately 3070 children ages 5 to 6 years who were assessed during a routine well-child visit by a preventive pediatrician in the Netherlands (response rate 85.2%). We obtained data about parent-reported child problems by using the Child Behavior Checklist (CBCL), sociodemographic background of the family, and characteristics of the preventive pediatrician. After each assessment, preventive pediatricians reported whether they had identified any psychosocial problem in the child. Multilevel logistic regression analyses were used to assess whether variation in the proportion of children identified by preventive pediatricians as having a psychosocial problem could be explained by the characteristics of preventive pediatricians and whether these characteristics were related to the quality of problem identification. RESULTS: Preventive pediatricians varied widely in the proportion of children identified as having psychosocial problems. Pediatrician characteristics such as work experience and work style (for example, on indication use of behavior questionnaires like the CBCL in routine care) explained about a quarter of this inter-pediatrician variation; child characteristics did not explain this variation even though characteristics like gender and parental education level were associated with likelihood of problem identification. More use of the CBCL and less use of the Teacher Report Form in routine care resulted in a better problem identification by preventive pediatricians. Work experience was not related to better problem identification. CONCLUSIONS: Preventive pediatricians identify psychosocial problems in children in a standardized way, but important inter-pediatrician variation remains. This variation may be reduced further and quality improved by changing their work style and targeted training.

An accurate and efficient identification of children with psychosocial problems by means of computerized adaptive testing.

BACKGROUND: Questionnaires used by health services to identify children with psychosocial problems are often rather short. The psychometric properties of such short questionnaires are mostly less than needed for an accurate distinction between children with and without problems. We aimed to assess whether a short Computerized Adaptive Test (CAT) can overcome the weaknesses of short written questionnaires when identifying children with psychosocial problems. METHOD: We used a Dutch national data set obtained from parents of children invited for a routine health examination by Preventive Child Healthcare with 205 items on behavioral and emotional problems (n = 2,041, response 84%). In a random subsample we determined which items met the requirements of an Item Response Theory (IRT) model to a sufficient degree. Using those items, item parameters necessary for a CAT were calculated and a cut-off point was defined. In the remaining subsample we determined the validity and efficiency of a Computerized Adaptive Test using simulation techniques, with current treatment status and a clinical score on the Total Problem Scale (TPS) of the Child Behavior Checklist as criteria. RESULTS: Out of 205 items available 190 sufficiently met the criteria of the underlying IRT model. For 90% of the children a score above or below cut-off point could be determined with 95% accuracy. The mean number of items needed to achieve this was 12. Sensitivity and specificity with the TPS as a criterion were 0.89 and 0.91, respectively. CONCLUSION: An IRT-based CAT is a very promising option for the identification of psychosocial problems in children, as it can lead to an efficient, yet high-quality identification. The results of our simulation study need to be replicated in a real-life administration of this CAT.

Comparing three short questionnaires to detect psychosocial dysfunction among primary school children: a randomized method.

BACKGROUND: Good questionnaires are essential to support the early identification of children with psychosocial dysfunction in community based settings. Our aim was to assess which of three short questionnaires was most suitable for this identification among school-aged children METHODS: A community-based sample of 2,066 parents of children aged 7-12 years (85% of those eligible) filled out the Child Behavior Checklist (CBCL) and - randomly determined - one of three questionnaires to be compared: the Strengths and Difficulties Questionnaire with Impact Supplement (SDQ), the Pediatric Symptom Checklist (PSC) and the PSYBOBA, a Dutch-origin questionnaire. Preventive Child Healthcare professionals assessed children's psychosocial functioning during routine health examinations. We assessed the scale structure (by means of Structural Equation Modelling), validity (correlation coefficients, sensitivity and specificity) and usability (ratings by parents and professionals) of each questionnaire and the degree to which they could improve the identification based only on clinical assessment (logistic regression). RESULTS: For the three questionnaires, Cronbach's alphas varied between 0.80 and 0.89. Sensitivities for a clinical CBCL at a cut off point with specificity = 0.90 varied between 0.78 and 0.86 for the three questionnaires. Areas under the Receiver Operating Curve, using the CBCL as criterion, varied between 0.93 and 0.96. No differences were statistically significant. All three questionnaires added information to the clinical assessment. Odds ratios (95% confidence intervals) for added information were PSC: 29.3 (14.4-59.8), SDQ: 55.0 (23.1-131.2) and PSYBOBA: 68.5 (28.3-165.6). Parents preferred the SDQ and PSYBOBA. Preventive Child Health Care professionals preferred the SDQ. CONCLUSIONS: This randomized comparison of three questionnaires shows that each of the three questionnaires can improve the detection of psychosocial dysfunction among children substantially.

Identification of children with psychosocial problems differed between preventive child health care professionals.

OBJECTIVE: To assess whether differences between individual Preventive Child Health Care (PCH) professionals in the percentage of children they identify as having psychosocial problems are larger than expected based on chance and whether such differences can be explained by differences in parent-reported problems or risk indicators. STUDY DESIGN: We used data from three community-based studies among 3,170 Dutch children aged 8 till 12 years, invited for a routine health examination. Parents filled in the Child Behavior Checklist (CBCL) and questions on demographic characteristics. After the examination, PCH professionals registered whether they had identified any psychosocial problem in the child. We examined differences between professionals in the rate of identified children. We used multilevel logistic regression analysis to assess whether such differences could be explained by parent-reported problems or risk indicators. RESULTS: Significant differences between PCH professionals were found (P<0.001). These differences could not be explained by parent-reported problems or risk indicators. The differences were largest for children with a score above the CBCL clinical cut-off point. CONCLUSION: Some PCH professionals are more likely to identify psychosocial problems than others, independently from parent-reported problems or other risk indicators.