Estimating vaccination coverage: validity of household-retained vaccination cards and parental recall.

Public health programs rely on household-survey estimates of vaccination coverage as a basis of programmatic and policy decisions; however, the validity of estimates derived from household-retained vaccination cards and parental recall has not been thoroughly evaluated. Using data from a vaccination coverage survey conducted in the Western Pacific's Northern Mariana Islands, we compared results from household data sources to medical record sources for the same children. We calculated the percentage of children aged 1, 2, and 6 years who received all vaccines recommended by age 12 months, 24 months, and for school entry, respectively. Coverage estimates based on vaccination cards ranged from 14% to 30% in the three age groups compared to 78-91% for the same children based on medical records. When cards were supplemented by parental recall, estimates were 51-53%. Concordance, sensitivity, specificity, positive and negative predictive values, and kappa statistics generally indicated poor agreement between household and medical record sources. Household-retained vaccination cards and parental recall were insufficient sources of information for estimating vaccination coverage in this population. This study emphasizes the importance of identifying reliable sources of vaccination history information and reinforces the need for awareness of the potential limitations of vaccination coverage estimated from surveys that rely on household-retained cards and/or parental recall.

Impact of methodological "shortcuts" in conducting public health surveys: results from a vaccination coverage survey.

BACKGROUND: Lack of methodological rigor can cause survey error, leading to biased results and suboptimal public health response. This study focused on the potential impact of 3 methodological "shortcuts" pertaining to field surveys: relying on a single source for critical data, failing to repeatedly visit households to improve response rates, and excluding remote areas. METHODS: In a vaccination coverage survey of young children conducted in the Commonwealth of the Northern Mariana Islands in July 2005, 3 sources of vaccination information were used, multiple follow-up visits were made, and all inhabited areas were included in the sampling frame. Results are calculated with and without these strategies. RESULTS: Most children had at least 2 sources of data; vaccination coverage estimated from any single source was substantially lower than from all sources combined. Eligibility was ascertained for 79% of households after the initial visit and for 94% of households after follow-up visits; vaccination coverage rates were similar with and without follow-up. Coverage among children on remote islands differed substantially from that of their counterparts on the main island indicating a programmatic need for locality-specific information; excluding remote islands from the survey would have had little effect on overall estimates due to small populations and divergent results. CONCLUSION: Strategies to reduce sources of survey error should be maximized in public health surveys. The impact of the 3 strategies illustrated here will vary depending on the primary outcomes of interest and local situations. Survey limitations such as potential for error should be well-documented, and the likely direction and magnitude of bias should be considered.

Vaccination coverage in the US Commonwealth of the Northern Mariana Islands, 2005.

BACKGROUND: In July 2005, a house-to-house survey was conducted to determine vaccination coverage achieved through routine health services on the three inhabited islands (Saipan, Rota, and Tinian) of the US Commonwealth of the Northern Mariana Islands (CNMI). METHODS: A population-based cluster survey was conducted on Saipan; clusters and households were selected by systematic random sampling. On the smaller islands of Rota and Tinian, all households were visited. Vaccination histories and demographic information were obtained during household interview for all children aged 19-35 months, children aged 6 years, and adults aged 65 years and older. Vaccination histories for children were supplemented by hospital/clinic records and an electronic vaccination registry. RESULTS: Among 295 children aged 19-35 months, estimated coverage with the primary vaccination series was 80 percent; coverage with individual vaccines was generally higher. Among 193 children aged 6 years, coverage for vaccines required at school-aged was 83 percent. Among 226 adults aged 65 years and older, 52 percent received influenza vaccine during the previous season while 21 percent had ever received pneumococcal vaccine. CONCLUSIONS: The CNMI has achieved the US Healthy People 2010 objective of 80 percent coverage for the standard vaccination series among children aged 19-35 months. High coverage levels among 6-year-old children may reflect the benefit of school entry requirements. Influenza and pneumococcal vaccination among older adults remains low. Efforts to ensure that children and older adults throughout the CNMI are equally well-protected should continue. Strategies to address parental awareness of vaccinations that are due should be explored and may be facilitated by upgrading the electronic vaccination registry.