Evaluation of four machine learning models for signal detection.

Background: Logistic regression-based signal detection algorithms have benefits over disproportionality analysis due to their ability to handle potential confounders and masking factors. Feature exploration and developing alternative machine learning algorithms can further strengthen signal detection. Objectives: Our objective was to compare the signal detection performance of logistic regression, gradient-boosted trees, random forest and support vector machine models utilizing Food and Drug Administration adverse event reporting system data. Design: Cross-sectional study. Methods: The quarterly data extract files from 1 October 2017 through 31 December 2020 were downloaded. Due to an imbalanced outcome, two training sets were used: one stratified on the outcome variable and another using Synthetic Minority Oversampling Technique (SMOTE). A crude model and a model with tuned hyperparameters were developed for each algorithm. Model performance was compared against a reference set using accuracy, precision, F1 score, recall, the receiver operating characteristic area under the curve (ROCAUC), and the precision-recall curve area under the curve (PRCAUC). Results: Models trained on the balanced training set had higher accuracy, F1 score and recall compared to models trained on the SMOTE training set. When using the balanced training set, logistic regression, gradient-boosted trees, random forest and support vector machine models obtained similar performance evaluation metrics. The gradient-boosted trees hyperparameter tuned model had the highest ROCAUC (0.646) and the random forest crude model had the highest PRCAUC (0.839) when using the balanced training set. Conclusion: All models trained on the balanced training set performed similarly. Logistic regression models had higher accuracy, precision and recall. Logistic regression, random forest and gradient-boosted trees hyperparameter tuned models had a PRCAUC ⩾ 0.8. All models had an ROCAUC ⩾ 0.5. Including both disproportionality analysis results and additional case report information in models resulted in higher performance evaluation metrics than disproportionality analysis alone.

Performance of subgrouped proportional reporting ratios in the US Food and Drug Administration (FDA) adverse event reporting system.

BACKGROUND: Many signal detection algorithms give the same weight to information from all products and patients, which may result in signals being masked or false positives being flagged as potential signals. Subgrouped analysis can be used to help correct for this. RESEARCH DESIGN AND METHODS: The publicly available US Food and Drug Administration Adverse Event Reporting System quarterly data extract files from 1 January 2015 through 30 September 2017 were utilized. A proportional reporting ratio (PRR) analysis subgrouped by either age, sex, ADE report type, seriousness of ADE, or reporter was compared to the crude PRR analysis using sensitivity, specificity, precision, and c-statistic. RESULTS: Subgrouping by age (n = 78, 34.5% increase), sex (n = 67, 15.5% increase), and reporter (n = 64, 10.3% increase) identified more signals than the crude analysis. Subgrouping by either age or sex increased both the sensitivity and precision. Subgrouping by report type or seriousness resulted in fewer signals (n = 50, -13.8% for both). Subgrouped analyses had higher c-statistic values, with age having the highest (0.468). CONCLUSIONS: Subgrouping by either age or sex produced more signals with higher sensitivity and precision than the crude PRR analysis. Subgrouping by these variables can unmask potentially important associations.

Direct-acting antiviral retreatment patterns for hepatitis C.

BACKGROUND: Despite the strong efficacy of direct-acting antivirals (DAAs) against the hepatitis C virus, many patients require a second regimen of DAA treatment. However, limited research exists to characterize rates of retreatment across different DAA agents or potential factors that may increase retreatment risk. OBJECTIVE: To characterize patterns and predictors of DAA retreatment among a large, generalizable, commercially insured US population of patients. METHODS: Using the IBM MarketScan Commercial Claims and Encounters data source, this retrospective cohort study examined retreatment patterns among patients receiving DAAs between 2013 and 2019. Descriptive statistics were used to compare patient characteristics predictive of retreatment risk and to examine rates of retreatment in patients initiating different DAA treatments. RESULTS: Among 31,553 DAA users, a total of 1,017 (3.2%) required DAA retreatment. Among the 1,017 patients re-treated, 44 (4.3%) received a third treatment regimen and 2 patients received a fourth treatment regimen. The average total cost for a retreatment regimen was $109,683, with patient out-of-pocket costs totaling $1,287 Patients requiring retreatment had higher rates of hypertension (32.0% vs 26.7%; P < 0.001), diabetes (16.9% vs 11.9%; P < 0.001), coagulopathy (9.9% vs 4.5%; P < 0.001), deficiency anemia (11.1% vs 7.4%; P < 0.001), alcohol abuse (3.3% vs 2.3%; P = 0.038), prior liver transplantation (3.4% vs 2.3%; P = 0.024), and hepatocellular carcinoma (6.1% vs 1.9%; P < 0.001) compared with patients not requiring retreatment. CONCLUSIONS: Although uncommon, some patients receiving DAAs require a second regimen of DAA treatment at substantial cost to both health plans and patients. These patients tend to have more comorbidities and markers of hepatic disease severity. Patients with high retreatment risk may benefit from careful monitoring for occurrences of retreatment.

Summary of adverse drug events for hydroxychloroquine, azithromycin, and chloroquine during the COVID-19 pandemic.

OBJECTIVE: Given the increased use of hydroxychloroquine (HCQ), chloroquine (CQ), and azithromycin (AZM) during the early months of the coronavirus disease 2019 (COVID-19) pandemic, there is a need to evaluate the associated safety concerns. The objective of this study was to summarize the adverse drug events (ADEs) associated with HCQ, CQ, and AZM use during the national COVID-19 emergency and compare the results with known adverse reactions listed in the drugs' package inserts. METHODS: A cross-sectional study design was used. The publicly available Food and Drug Administration Adverse Event Reporting System quarterly data extract files from January 1, 2020 to June 30, 2020 were downloaded. A disproportionality analysis was conducted using the proportional reporting ratio to identify possible ADE signals. A Poisson regression was used to assess if the number of ADE reports for the 3 drugs increased over time. RESULTS: There was a statistically significant increasing trend in the reported ADEs for both HCQ (P < 0.001) and AZM (P < 0.001). Before the declaration of the national emergency, there were 592 reported drug-ADE pairs for the 3 drugs compared with 2492 drug-ADE pairs reported after March 13, 2020. These 2492 drug-ADE pairs represented 848 ADEs across the 3 drugs, of which 114 (13.4%) were identified as potential signals including 55 (48.2%) that were not listed in the prescribing information. CONCLUSIONS: Our results showed that the reported ADEs for HCQ and AZM have increased during the COVID-19 pandemic. Differences were observed in both the type of and frequency of the highest reported ADEs for the 3 selected drugs before and after the national emergency declaration. Although causation cannot be determined from ADE reports, further investigation of some reports may be warranted. Our results highlight the need for pharmacovigilance and education of health care professionals on the safety of these drugs when being used for COVID-19 prophylaxis or treatment.

Comparing the use of individual and composite terms to evaluate adverse drug event disproportionality: a focus on glucagon-like peptide-1 receptor agonists and diabetic retinopathy.

Background: The SUSTAIN-6 trial showed significantly higher rates of retinopathy complications in the semaglutide group compared to placebo. Observational studies have not consistently corroborated this finding, raising questions about the appropriateness of composite variables and whether the relationship exists across the entire drug class or is limited to individual glucagon-like peptide 1 agonists (GLP-1RAs). The study objective was to evaluate the difference between using individual and composite terms to assess associations between GLP-1RAs and diabetic retinopathy events.Research Design and Methods: Reports from the US Food and Drug Administration Adverse Event Reporting System were utilized to examine relationships between GLP-1RAs and diabetic retinopathy events. A disproportionality analysis was conducted using the proportional reporting ratio.Results: Four GLP-1RAs demonstrated signals for diabetic retinopathy events. The GLP-1RA drug class had four diabetic retinopathy signals. Only semaglutide had a signal for the composite diabetic retinopathy outcome. The GLP-1RA drug class and the composite diabetic retinopathy outcome did not meet the PRR signal thresholds.Conclusions: The use of drug class level and composite outcome variables may mask diabetic retinopathy signals in comparison to individual drug assessments. Our results support the SUSTAIN-6 trial findings and suggest an association between four GLP-1RAs and diabetic retinopathy events.

FDA's Unimproved Enforcement of Postmarketing Requirements and Commitments: Implications for Providers and Patients.

BACKGROUND: The 2007 reauthorization of the Prescription Drug User Fee Act empowered the Food and Drug Administration (FDA) to require and enforce postmarketing studies to monitor the safety of prescription drugs, an increasing number of which are approved under expedited development or review programs. However, compliance rates for postmarketing requirements are low, and the FDA has not exercised its enforcement authority, allowing the very safety concerns that prompted the expansion of its power to continue. Prior evaluations have found that the FDA lacks reliable, timely, and readily accessible data for tracking postmarket safety issues, and that it has failed to enforce the postmarket surveillance measures it requires of the pharmaceutical industry. OBJECTIVE: This study provides an updated evaluation of FDA oversight of postmarketing requirements and commitments and assesses whether there have been improvements since 2010. METHODS: This study utilized data from the FDA's annual Report on the Performance of Drug and Biologics Firms in Conducting Postmarketing Requirements and Commitments. It evaluated studies opened in FY 2011-2014 and tracked their status through the FDA's FY 2018 report (data as of September 30, 2018), thereby allowing the pharmaceutical industry 4-7 years to complete a given PMR/PMC. Descriptive statistics were calculated, and differences in the status of postmarketing requirements and postmarketing commitments between FYs 2011-2014 and FY 2009-2010 were evaluated. RESULTS: During fiscal years 2011-2014, there was little difference in the FDA's oversight of postmarketing studies compared to fiscal years 2009-2010. While there were some year-to-year significant differences, the overall trend indicated no change. CONCLUSIONS: The FDA's oversight of postmarketing studies has not improved since 2010. This paper discusses implications for providers and patients who unduly assume the responsibility of postmarketing surveillance due to the lack of FDA oversight.

Ceftobiprole: A novel, broad-spectrum cephalosporin with activity against methicillin-resistant Staphylococcus aureus.

PURPOSE: The pharmacology, antimicrobial activity, pharmacokinetics, pharmacodynamics, clinical efficacy, safety, and place in therapy of ceftobiprole are reviewed. SUMMARY: Ceftobiprole, a novel, broad-spectrum, parenteral cephalosporin, inhibits the cell-wall synthesis of penicillin-binding proteins (PBPs) PBP2a and PBP2x, responsible for the resistance in staphylococci and pneumococci, respectively. Ceftobiprole has good activity against gram-positive aerobes and anaerobes, and its activity against gram-negative aerobes and anaerobes is species dependent. Ceftobiprole is relatively inactive against Acinetobacter species. Its ability to bind relevant PBPs of resistant gram-positive and gram-negative bacteria indicates its potential use in the treatment of hospital-acquired pneumonia and complicated skin and skin-structure infections (cSSSIs). Ceftobiprole is primarily excreted unchanged by the kidneys and exhibits linear pharmacokinetics. The half-life of the drug is approximately 3-4 hours. It exhibits minimal plasma protein binding (16%). Ceftobiprole does not inhibit the cytochrome P-450 isoenzyme system, so the possibility of drug-drug interactions is low. The drug has not been approved for use in the United States but has been approved in Canada and elsewhere. Ceftobiprole is currently undergoing Phase III clinical trials and has demonstrated activity against methicillin-resistant Staphylococcus aureus, penicillin-resistant Streptococcus pneumoniae, and Pseudomonas aeruginosa. Completed Phase III trials used i.v. dosages of 500 mg every 8-12 hours. The most commonly observed adverse effects of ceftobiprole included headache and gastrointestinal upset. CONCLUSION: Ceftobiprole is a novel, broad-spectrum, parenteral cephalosporin undergoing Phase III clinical trials. Its broad spectrum of activity makes it a candidate for monotherapy of cSSSIs and pneumonias that have required combination therapy in the past.

Statewide sentinel surveillance for antibiotic nonsusceptibility among streptococcus pneumoniae isolates in South Carolina, 2003-2004.

BACKGROUND: In 2003, the South Carolina Department of Health and Environmental Control established the Carolina Antibiotic Resistance Surveillance System (CARSS), an active sentinel surveillance system for antibiotic-resistant Streptococcus pneumoniae. METHODS: CARSS includes twelve hospitals. Each hospital was assigned a weighted sample size. Minimum inhibitory concentrations were determined using the E-test method. RESULTS: A total of 452 isolates were collected. The prevalence of penicillin nonsusceptibility in the study was 44.9%. Penicillin intermediate resistance (PCN-I) was 33.2%, and penicillin high-level resistance (PCN-R) was 11.7%. One hundred six (23.5%) isolates were nonsusceptible to one antibiotic. One hundred twenty-four (27.4%) isolates were nonsusceptible to three or more antibiotics. CONCLUSIONS: CARSS confirmed the prevalences of antibiotic nonsusceptibility previously reported for South Carolina. However, CARSS suggests resistance is shifting from PCN-R to PCN-I in South Carolina. There is a high prevalence of multidrug nonsusceptibility in South Carolina. CARSS will continue to monitor these trends.

Laboratory survey of antibiotic nonsusceptibility among Streptococcus pneumoniae isolates in South Carolina, 1998 versus 2000.

BACKGROUND: In 1998, the South Carolina Department of Health and Environmental Control surveyed clinical microbiology laboratories statewide to determine the prevalence of antibiotic nonsusceptibility among isolates of Streptococcus pneumoniae. A follow-up study was conducted in 2001. METHODS: A cross-sectional study was conducted to estimate the prevalence of penicillin nonsusceptibility (PCN-N), extended-spectrum cephalosporin nonsusceptibility (ESC-N), and levofloxacin nonsusceptibility (LEV-N) in South Carolina. A standardized questionnaire was mailed to 89 laboratories. RESULTS: The prevalence of penicillin intermediate resistance increased from 1998 (17.6%) to 2000 (20.9%, chi2 P = 0.008). Furthermore, the prevalence of PCN-N increased from 1998 (34.5%) to 2000 (38.4%, chi2 P = 0.01). The prevalence of ECN-N decreased from 1998 (19.1%) to 2000 (17.7%), but the difference was not significant (chi2 P = 0.25). CONCLUSION: The laboratory survey was a low-cost method of estimating the change in prevalence of antibiotic nonsusceptibility, and it emphasizes regional surveillance because the prevalence of antibiotic nonsusceptibility varied geographically.