Identifying Cases of Shoulder Injury Related to Vaccine Administration (SIRVA) in the United States: Development and Validation of a Natural Language Processing Method.

BACKGROUND: Shoulder injury related to vaccine administration (SIRVA) accounts for more than half of all claims received by the National Vaccine Injury Compensation Program. However, due to the difficulty of finding SIRVA cases in large health care databases, population-based studies are scarce. OBJECTIVE: The goal of the research was to develop a natural language processing (NLP) method to identify SIRVA cases from clinical notes. METHODS: We conducted the study among members of a large integrated health care organization who were vaccinated between April 1, 2016, and December 31, 2017, and had subsequent diagnosis codes indicative of shoulder injury. Based on a training data set with a chart review reference standard of 164 cases, we developed an NLP algorithm to extract shoulder disorder information, including prior vaccination, anatomic location, temporality and causality. The algorithm identified 3 groups of positive SIRVA cases (definite, probable, and possible) based on the strength of evidence. We compared NLP results to a chart review reference standard of 100 vaccinated cases. We then applied the final automated NLP algorithm to a broader cohort of vaccinated persons with a shoulder injury diagnosis code and performed manual chart confirmation on a random sample of NLP-identified definite cases and all NLP-identified probable and possible cases. RESULTS: In the validation sample, the NLP algorithm had 100% accuracy for identifying 4 SIRVA cases and 96 cases without SIRVA. In the broader cohort of 53,585 vaccinations, the NLP algorithm identified 291 definite, 124 probable, and 52 possible SIRVA cases. The chart-confirmation rates for these groups were 95.5% (278/291), 67.7% (84/124), and 17.3% (9/52), respectively. CONCLUSIONS: The algorithm performed with high sensitivity and reasonable specificity in identifying positive SIRVA cases. The NLP algorithm can potentially be used in future population-based studies to identify this rare adverse event, avoiding labor-intensive chart review validation.

Risk for Shoulder Conditions After Vaccination: A Population-Based Study Using Real-World Data.

BACKGROUND: Although shoulder conditions have been reported as an adverse event after intramuscular vaccination in the deltoid muscle, epidemiologic data on shoulder conditions after vaccination are limited. OBJECTIVE: To estimate the risk for shoulder conditions after vaccination and assess possible risk factors. DESIGN: Retrospective cohort study. SETTING: Kaiser Permanente Southern California, a large integrated health care organization. PARTICIPANTS: Kaiser Permanente Southern California members aged 3 years or older who had an intramuscular vaccination administered in the deltoid muscle between 1 April 2016 and 31 December 2017. MEASUREMENTS: A natural language processing (NLP) algorithm was used to identify potential shoulder conditions among vaccinated persons with shoulder disorder diagnosis codes. All NLP-identified cases were manually chart confirmed on the basis of our case definition. The characteristics of vaccinated persons with and without shoulder conditions were compared. RESULTS: Among 3 758 764 administered vaccinations, 371 cases of shoulder condition were identified, with an estimated incidence of 0.99 (95% CI, 0.89 to 1.09) per 10 000 vaccinations. The incidence was 1.22 (CI, 1.10 to 1.35) for the adult (aged ≥18 years) and 0.05 (CI, 0.02 to 0.14) for the pediatric (aged 3 to 17 years) vaccinated populations. In the adult vaccinated population, advanced age, female sex, an increased number of outpatient visits in the 6 months before vaccination, lower Charlson Comorbidity Index, and pneumococcal conjugate vaccine were associated with a higher risk for shoulder conditions. Among influenza vaccines, quadrivalent vaccines were associated with an increased risk for shoulder conditions. Simultaneous administration of vaccines was associated with a higher risk for shoulder conditions among elderly persons. LIMITATION: Generalizability to other health care settings, use of administrative data, and residual confounding. CONCLUSION: These population-based data suggest a small absolute risk for shoulder conditions after vaccination. Given the high burden of shoulder conditions, clinicians should pay attention to any factors that may further increase risks. PRIMARY FUNDING SOURCE: Centers for Disease Control and Prevention.