Assessment of interobserver reliability in nutrition studies that use direct observation of school meals.

This article (a) provides a general review of interobserver reliability (IOR) and (b) describes our method for assessing IOR for items and amounts consumed during school meals for a series of studies regarding the accuracy of fourth-grade children's dietary recalls validated with direct observation of school meals. A widely used validation method for dietary assessment is direct observation of meals. Although many studies utilize several people to conduct direct observations, few published studies indicate whether IOR was assessed. Assessment of IOR is necessary to determine that the information collected does not depend on who conducted the observation. Two strengths of our method for assessing IOR are that IOR was assessed regularly throughout the data collection period and that IOR was assessed for foods at the item and amount level instead of at the nutrient level. Adequate agreement among observers is essential to the reasoning behind using observation as a validation tool. Readers are encouraged to question the results of studies that fail to mention and/or to include the results for assessment of IOR when multiple people have conducted observations.

Quality control for interviews to obtain dietary recalls from children for research studies.

Quality control is an important aspect of a study because the quality of data collected provides a foundation for the conclusions drawn from the study. For studies that include interviews, establishing quality control for interviews is critical in ascertaining whether interviews are conducted according to protocol. Despite the importance of quality control for interviews, few studies adequately document the quality control procedures used during data collection. This article reviews quality control for interviews and describes methods and results of quality control for interviews from two of our studies regarding the accuracy of children's dietary recalls; the focus is on quality control regarding interviewer performance during the interview, and examples are provided from studies with children. For our two studies, every interview was audio recorded and transcribed. The audio recording and typed transcript from one interview conducted by each research dietitian either weekly or daily were randomly selected and reviewed by another research dietitian, who completed a standardized quality control for interviews checklist.Major strengths of the methods of quality control for interviews in our two studies include: (a) interviews obtained for data collection were randomly selected for quality control for interviews, and (b) quality control for interviews was assessed on a regular basis throughout data collection. The methods of quality control for interviews described may help researchers design appropriate methods of quality control for interviews for future studies.

Recency affects reporting accuracy of children's dietary recalls.

PURPOSE: This study evaluated the effect of recency on accuracy of fourth-graders' dietary reports. METHODS: Each of 60 randomly selected children was observed eating school meals (breakfast, lunch) and interviewed to obtain a 24-hour dietary recall using one of six conditions generated by crossing two target periods (previous day, prior 24 hours) with three interview times (morning, afternoon, evening), with 10 children (5 males) per condition. Accuracy of the school meal portions of each recall was assessed by comparing reports to observations. Rates for omissions (items observed but not reported) and intrusions (items reported but not observed) were calculated to determine accuracy for reporting items. A measure of total inaccuracy combined errors for reporting items and amounts. RESULTS: Using the prior 24 hours as the target period yielded better performance than did using the previous day: Omission rates were lower by about one-third, intrusion rates by about one-half, and total inaccuracy by about one-third (all p's<0.01). A marginally significant interaction of target period by interview time was found for omission rate (p=0.08), but not for intrusion rate (p=0.15) or for total inaccuracy (p=0.47). CONCLUSIONS: This provides evidence that recency influences children's recall accuracy and demonstrates the importance of an awareness of principles of memory when designing what are essentially memory tests for epidemiologic studies.

Children's Social Desirability and Dietary Reports.

We investigated telephone administration of the Children's Social Desirability (CSD) scale and our adaptation for children of the Social Desirability for Food scale (C-SDF). Each of 100 4th-graders completed 2 telephone interviews 28 days apart. CSD scores had adequate internal consistency and test-retest reliability, and a 14-item subset was identified that sufficiently measures the same construct. Our C-SDF scale performed less well in terms of internal consistency and test-retest reliability; factor analysis revealed 2 factors, 1 of which was moderately related to the CSD. The 14-item subset of the CSD scale may help researchers understand error in children's dietary reports.

Accuracy of fourth-graders' dietary recalls of school breakfast and school lunch validated with observations: in-person versus telephone interviews.

OBJECTIVE: To investigate the accuracy of children's dietary recalls of school breakfast and school lunch validated with observations and obtained during in-person versus telephone interviews. DESIGN: Each child was observed eating school breakfast and school lunch and was interviewed that evening about that day's intake. SETTING: Ten elementary schools. PARTICIPANTS: A sample of fourth-graders was randomly selected within race (black, white) and gender strata, observed, and interviewed in person (n = 33) or by telephone (n = 36). MAIN OUTCOMES MEASURED: Rates for omissions (items observed but not reported) and intrusions (items reported but not observed) were calculated to determine accuracy for reporting items. A measure of total inaccuracy was calculated to determine inaccuracy for reporting items and amounts combined. ANALYSIS: Analysis of variance; chi-square. RESULTS: Interview type (in person, telephone) did not significantly affect recall accuracy. For omission rate, intrusion rate, and total inaccuracy, means were 34%, 19%, and 4.6 servings for in-person recalls and 32%, 16%, and 4.3 servings for telephone recalls of school breakfast and school lunch. CONCLUSIONS AND IMPLICATIONS: The accuracy of children's recalls of school breakfast and school lunch is not significantly different whether obtained in person or by telephone. Whether interviewed in person or by telephone, children reported only 67% of items observed; furthermore, 17% of items reported were not observed.

Reverse versus forward order reporting and the accuracy of fourth-graders' recalls of school breakfast and school lunch.

BACKGROUND: Do children recall school breakfast and school lunch intake during 24-h recalls more accurately when prompted to report meals and snacks in reverse versus forward order? METHODS: One hundred twenty-one fourth-graders stratified by race (Black, White) and gender were each observed and interviewed twice (once per order) regarding the previous day's intake. Omission and intrusion rates determined accuracy for reporting items. Total inaccuracy determined accuracy for reporting items and amounts. RESULTS: Results failed to indicate significant effects of interviewer, weekday, sequence (first or second recall), or race on omission rates, intrusion rates, or total inaccuracy. A significant order by gender interaction was found for omission rates, which were lower (i.e., better) for males for reverse (53%) versus forward recalls (62%), but not females (61 versus 53%) (P < 0.008). Intrusion rates were acceptable for males for 54% of reverse recalls and 40% of forward recalls (P = 0.095). Means were 57 and 32%, and 6.4 servings for omission rate, intrusion rate, and total inaccuracy for reverse recalls, and 56 and 39%, and 6.9 servings for forward recalls. CONCLUSIONS: Prompting children to report in reverse versus forward order improved omission and intrusion rates for males more so than females. Regardless of reverse or forward order, children reported <50% of items observed; furthermore, >30% of items reported were not observed. Research is needed to enhance accuracy of children's dietary recalls.

Differences in Fourth-Graders' Participation Rates Across Four School-Based Nutrition Studies.

Federal policy has encouraged researchers to include children in research studies; thus, it is important to report experiences recruiting children to participate in studies. This article compares fourth-graders' participation rates across four school-based nutrition studies conducted in one school district in a southeastern state. For each study, children were observed eating school meals (breakfast and lunch); interviewed regarding dietary intake; and weighed and measured. For Study 1, children from 11 schools received $10 per interview for up to two interviews conducted in the morning at school. For Study 2, children from 10 schools received $25 if interviewed once in the evening, either by telephone or in a van parked outside the child's home. For Study 3, children from three schools received $10 per interview for up to three interviews held in the evening by telephone. For Study 4, children from six schools received $15 per interview for up to two interviews conducted either in the morning or afternoon at school, or in the evening by telephone. Recruitment procedures were similar for all studies.Participation rates were 73% (n=635) for Study 1, 57% (n=432) for Study 2, 66% (n=158) for Study 3, and 71% (n=296) for Study 4. Logistic regression was used to determine whether study (1, 2, 3, 4), race (black, white), or gender (male, female) were significant predictors of participation (agreed, denied). The results indicated that study (p<0.0001), race (p=0.0198), and gender (p=0.0188) were significant predictors, however, no two-factor interactions among these effects were significant. Post hoc pairwise comparisons with Bonferroni adjustment indicated that agreement to participate for Study 2 was lower (p<0.0001) than that for Studies 1, 3, and 4, which did not differ. Agreement to participate across all four studies was higher for black (69%) than white (63%; p=0.0054) children and for females (69%) than males (64%; p=0.0209). Schools provide a natural environment for nutrition research because school foodservice programs feed millions of children one or two meals (breakfast and/or lunch) each school day. Observations of children eating school meals provide a convenient and relatively unobtrusive means of validating children's dietary recalls. Thus, at some point, most child nutrition professionals are likely to be involved in research either directly (i.e. by conducting studies themselves) or indirectly (i.e. by allowing others access to their school cafeterias to collect data).This paper references the following data:Table 1. Similarities and Differences in the Designs for Each of the Four Studies.Table 2. Number and Percent of Fourth-Grade Children Who Agreed or Denied to Participate by Race and Gender Across all Four Studies Combined.Table 3. Number and Percent of Fourth-Grade Children Who Agreed or Denied to Participate by Gender and Race for Each of the Four Studies Separately.