Getting Rigor Right: A Framework for Methodological Choice in Adaptive Monitoring and Evaluation.

The field of global development has embraced the idea that programs require agile, adaptive approaches to monitoring, evaluation, and learning. But considerable debate still exists around which methods are most appropriate for adaptive learning. Researchers have a range of proven and novel tools to promote a culture of adaptation and learning. These tools include lean testing, rapid prototyping, formative research, and structured experimentation, all of which can be utilized to generate responsive feedback (RF) to improve social change programs. With such an extensive toolkit, how should one decide which methods to employ? In our experience, the level of rigor used should be responsive to the team's level of certainty about the program design being investigated-how certain-or confident-are we that a program design will produce its intended results? With less certainty, less rigor is needed; with more certainty, more rigor is needed. In this article, we present a framework for getting rigor right and illustrate its use in 3 case studies. For each example, we describe the feedback methods used and why, how the approach was implemented (including how we conducted cocreation and ensured buy-in), and the results of each engagement. We conclude with lessons learned from these examples and how to use the right kind of RF mechanism to improve social change programs.

Correct diagnosis of childhood pneumonia in public facilities in Tanzania: a randomised comparison of diagnostic methods.

OBJECTIVE: This study compares two methods for clinical diagnosis of childhood pneumonia that aim to estimate rates of underdiagnosis and overdiagnosis of childhood pneumonia by examining the sensitivity of Integrated Management of Childhood Diseases implementation in routine care against lung ultrasound (LUS) diagnosis. SETTING: We conducted observations in 83 public health facilities (dispensaries, health centres and district hospitals) in Pwani, Dodoma and Tabora, Tanzania between October and December 2017. METHODS: We used a novel method to estimate rates of underdiagnosis and overdiagnosis of childhood pneumonia by comparing directly observed public provider diagnoses to the results of diagnoses made by trained clinicians using Mindray DP-10 ultrasound machines. We perform multivariate analysis to identify confounding effects and robustness checks to bound the result. We also explore a number of observable characteristics correlated with higher rates of agreement between provider diagnoses and ultrasound diagnoses. RESULTS: We observed 93 providers conducting exams on patients aged 2 months-5 years who presented respiratory symptoms or were given a respiratory diagnosis by the provider. Of these 957 patients, 110 were excluded from analysis resulting in a final sample of 847.17.6% of cases identified as pneumonia via LUS examinations in our sample were diagnosed as pneumonia by providers, suggesting that a significant number of pneumonia cases for which care is sought in the public sector go undiagnosed. Provider knowledge of breath counting and years of experience are positively correlated with higher agreement. While clinical examination rates are not statistically correlated with agreement, it is notable that providers conducted a clinical examination on only about one-third of patients in the sample. CONCLUSION: Our results suggest that provider training and knowledge of clinical examination protocols for pneumonia diagnosis are predictive of correct diagnosis of pneumonia and should be further explored in future research as a tool for improving quality of care.

Prepregnancy Obesity and Birth Outcomes.

OBJECTIVE: To investigate the association between prepregnancy obesity and birth outcomes using fixed effect models comparing siblings from the same mother. METHODS: A total of 7496 births to 3990 mothers from the National Longitudinal Survey of Youth 1979 survey are examined. Outcomes include macrosomia, gestational length, incidence of low birthweight, preterm birth, large and small for gestational age (LGA, SGA), c-section, infant doctor visits, mother's and infant's days in hospital post-partum, whether the mother breastfed, and duration of breastfeeding. Association of outcomes with maternal pre-pregnancy obesity was examined using Ordinary Least Squares (OLS) regression to compare across mothers and fixed effects to compare within families. RESULTS: In fixed effect models we find no statistically significant association between most outcomes and prepregnancy obesity with the exception of LGA, SGA, low birth weight, and preterm birth. We find that prepregnancy obesity is associated with a with lower risk of low birthweight, SGA, and preterm birth but controlling for prepregnancy obesity, increases in GWG lead to increased risk of LGA. CONCLUSIONS: Contrary to previous studies, which have found that maternal obesity increases the risk of c-section, macrosomia, and LGA, while decreasing the probability of breastfeeding, our sibling comparison models reveal no such association. In fact, our results suggest a protective effect of obesity in that women who are obese prepregnancy have longer gestation lengths, and are less likely to give birth to a preterm or low birthweight infant.

Characterization of the salivary microbiome in patients with pancreatic cancer.

Clinical manifestations of pancreatic cancer often do not occur until the cancer has undergone metastasis, resulting in a very low survival rate. In this study, we investigated whether salivary bacterial profiles might provide useful biomarkers for early detection of pancreatic cancer. Using high-throughput sequencing of bacterial small subunit ribosomal RNA (16S rRNA) gene, we characterized the salivary microbiota of patients with pancreatic cancer and compared them to healthy patients and patients with other diseases, including pancreatic disease, non-pancreatic digestive disease/cancer and non-digestive disease/cancer. A total of 146 patients were enrolled at the UCSD Moores Cancer Center where saliva and demographic data were collected from each patient. Of these, we analyzed the salivary microbiome of 108 patients: 8 had been diagnosed with pancreatic cancer, 78 with other diseases and 22 were classified as non-diseased (healthy) controls. Bacterial 16S rRNA sequences were amplified directly from salivary DNA extractions and subjected to high-throughput sequencing (HTS). Several bacterial genera differed in abundance in patients with pancreatic cancer. We found a significantly higher ratio of Leptotrichia to Porphyromonas in the saliva of patients with pancreatic cancer than in the saliva of healthy patients or those with other disease (Kruskal-Wallis Test; P < 0.001). Leptotrichia abundances were confirmed using real-time qPCR with Leptotrichia specific primers. Similar to previous studies, we found lower relative abundances of Neisseria and Aggregatibacter in the saliva of pancreatic cancer patients, though these results were not significant at the P < 0.05 level (K-W Test; P = 0.07 and P = 0.09 respectively). However, the relative abundances of other previously identified bacterial biomarkers, e.g., Streptococcus mitis and Granulicatella adiacens, were not significantly different in the saliva of pancreatic cancer patients. Overall, this study supports the hypothesis that bacteria abundance profiles in saliva are useful biomarkers for pancreatic cancer though much larger patient studies are needed to verify their predictive utility.

Necessity of lumbar puncture in patients presenting with new onset complex febrile seizures.

INTRODUCTION: This study aims to characterize the population of patients presenting to a pediatric emergency department (ED) for a first complex febrile seizure, and subsequently assess the rate of acute bacterial meningitis (ABM) occurrence in this population. Furthermore, this study seeks to identify whether a specific subset of patients may be at lesser risk for ABM or other serious neurological disease. METHODS: This retrospective cohort study reviewed the charts of patients between the ages of 6 months to 5 years of age admitted to an ED between 2005 and 2010 for a first complex febrile seizure (CFS). The health information department generated a patient list based on admission and discharge diagnoses, which was screened for patient eligibility. Exclusion criteria included history of a complex febrile seizure, history of an afebrile seizure, trauma, or severe underlying neurological disorder. Data extracted included age, gender, relevant medical history, descriptions of seizure, treatment received, and follow-up data. Patients presenting with two short febrile seizures within 24 hours were then analyzed separately to assess health outcomes in this population. RESULTS: There were 193 patients were eligible. Lumbar puncture was performed on 136 subjects; it was significantly more likely to be performed on patients that presented with seizure focality, status epilepticus, or a need for intubation. Fourteen patients were found to have pleocytosis following white blood cell count correction, and 1 was diagnosed with ABM (0.5% [95% confidence interval: 0.0-1.5, n=193]). Forty-three patients had 2 brief febrile seizures within 24 hours. Of the 43, 17 received lumbar puncture while in the ED. None of these patients were found to have ABM or other serious neurological disease. CONCLUSION: ABM is rare in patients presenting with a first complex febrile seizure. Patients presenting only with 2 short febrile seizures within 24 hours may be less likely to have ABM, and may not require lumbar puncture without other clinical symptoms of neurological disease.

Group B Streptococcus CovR regulation modulates host immune signalling pathways to promote vaginal colonization.

Streptococcus agalactiae (Group B Streptococcus, GBS) is a frequent commensal organism of the vaginal tract of healthy women. However, GBS can transition to a pathogen in susceptible hosts, but host and microbial factors that contribute to this conversion are not well understood. GBS CovR/S (CsrR/S) is a two component regulatory system that regulates key virulence elements including adherence and toxin production. We performed global transcription profiling of human vaginal epithelial cells exposed to WT, CovR deficient, and toxin deficient strains, and observed that insufficient regulation by CovR and subsequent increased toxin production results in a drastic increase in host inflammatory responses, particularly in cytokine signalling pathways promoted by IL-8 and CXCL2. Additionally, we observed that CovR regulation impacts epithelial cell attachment and intracellular invasion. In our mouse model of GBS vaginal colonization, we further demonstrated that CovR regulation promotes vaginal persistence, as infection with a CovR deficient strainresulted in a heightened host immune response as measured by cytokine production and neutrophil activation. Using CXCr2 KO mice, we determined that this immune alteration occurs, at least in part, via signalling through the CXCL2 receptor. Taken together, we conclude that CovR is an important regulator of GBS vaginal colonization and loss of this regulatory function may contribute to the inflammatory havoc seen during the course of infection.

Do bites of kissing bugs cause unexplained allergies? Results from a survey in triatomine-exposed and unexposed areas in southern california.

BACKGROUND: The bite of Triatominae can cause cutaneous allergic reactions and even anaphylaxis. Since the early 1980s, no population-based surveys have been done in Southern California, and none was ever carried out in inland Los Angeles or Riverside Counties. OBJECTIVES: To measure the frequency of insect sightings, bites and allergic reactions in a suburban area of eastern Los Angeles County and along with rural and urban sites in Riverside County. METHODS: A door-to-door survey was done in triatomine exposed and unexposed areas. Logistic regression modeling was used for the analysis, and study participants were asked to collect insects. RESULTS: Out of the 221 study participants in the exposed areas, 46 (20%) knew about the presence of Triatominae in their neighborhood. Fifteen (7%) persons reported triatomine sightings in their house during the month preceding the interview. Also, 15 (7%) participants reported ever being bitten by a triatomine. Ten (5%) participants collected either a Triatoma protracta Uhler and/or Paratriatoma hirsuta Barber in and around their house. Twenty-nine (13%) persons in the rural Riverside County reported symptoms compatible with allergy to triatomine bites. This was 4 times higher than in the urban control area where only 4 (3%) of 115 persons reported these symptoms. The association between living in a triatomine-exposed area and self-reported symptoms suggestive of allergies increased slightly when adjusted for the participant's sex and the age of their house (adjusted odds ratio: 5.1, 95% confidence interval: 1.2 to 22.0). Reporting these symptoms was associated with seeing Triatominae in the neighborhood and having been bitten. CONCLUSION: Allergies to triatomine bites could be a significant problem in inland Southern California. Further investigations, a diagnostic test and better information of persons living in triatomine-exposed areas are needed.

Impact of endovascular simulator training on vascular surgery as a career choice in medical students.

OBJECTIVE: The primary goal of this study was to determine whether exposure to endovascular simulator training increases interest in vascular surgery among medical students. Secondary goals were to determine whether interest in vascular surgery is inversely related to the time after exposure, to identify factors associated with interest, and to identify students' characteristics that positively influence performance metrics. METHODS: This was a prospective, randomized, crossover study comprising 80 medical students who were randomized into group A (n = 40) and group B (n = 40). Participants completed a survey of their interest in vascular surgery and attitudinal factors using Vascular Surgery Interest Form (VSIF) before exposure to the simulator (pretest). At 1 month after exposure of group A to the simulator, both groups were tested using VSIF (test). Upon completion of testing, group B was exposed to simulator training, whereas group A received no further training. At 2 months after exposure of group B to the simulator, both groups were posttested using VSIF, which asked the students' level of interest in vascular surgery using a 1 to 10 scale. Performance metrics were recorded during each exposure. Differences among cohort demographics were determined using Pearson χ(2) analysis. Differences in interest were determined with paired sample correlations. Linear regression and analysis of variance were used to correlate VSIF responses with interest and the performance metrics. RESULTS: Both student cohorts had significant increases in interest after exposure to simulation. In group A, test interest (mean ± standard deviation) was significantly higher than pretest and posttest interests (5.51 ± 1.73 vs 4.00 ± 1.88 vs 4.18 ± 1.82; P < .05). In group B, posttest interest was significantly higher than pretest and test interests (5.62 ± 2.03 vs 3.96 ± 1.61 vs 4.08 ± 1.64; P < .05). The increase in interest was reciprocally related to the time passed since the initial exposure. Resident and attending lifestyle, length of training, radiation concerns, gender identification of a mentor, and personality fit with occupation were not correlated with interest. Sex, medical school year, comfort with endovascular procedures, willingness to work long hours, interest in performing percutaneous procedures, and commitment to surgical career did not affect impact performance metrics. CONCLUSIONS: One exposure of students to endovascular simulator training is associated with an increase in vascular surgery interest. Acquired interest is reciprocally related to the time demonstrating the temporal importance of the exposure.