Designing evaluation studies to optimally inform policy: what factors do policy-makers in China consider when making resource allocation decisions on healthcare worker training programmes?

BACKGROUND: In light of the gap in evidence to inform future resource allocation decisions about healthcare provider (HCP) training in low- and middle-income countries (LMICs), and the considerable donor investments being made towards training interventions, evaluation studies that are optimally designed to inform local policy-makers are needed. The aim of our study is to understand what features of HCP training evaluation studies are important for decision-making by policy-makers in LMICs. We investigate the extent to which evaluations based on the widely used Kirkpatrick model - focusing on direct outcomes of training, namely reaction of trainees, learning, behaviour change and improvements in programmatic health indicators - align with policy-makers' evidence needs for resource allocation decisions. We use China as a case study where resource allocation decisions about potential scale-up (using domestic funding) are being made about an externally funded pilot HCP training programme. METHODS: Qualitative data were collected from high-level officials involved in resource allocation at the national and provincial level in China through ten face-to-face, in-depth interviews and two focus group discussions consisting of ten participants each. Data were analysed manually using an interpretive thematic analysis approach. RESULTS: Our study indicates that Chinese officials not only consider information about the direct outcomes of a training programme, as captured in the Kirkpatrick model, but also need information on the resources required to implement the training, the wider or indirect impacts of training, and the sustainability and scalability to other settings within the country. In addition to considering findings presented in evaluation studies, we found that Chinese policy-makers pay close attention to whether the evaluations were robust and to the composition of the evaluation team. CONCLUSIONS: Our qualitative study indicates that training programme evaluations that focus narrowly on direct training outcomes may not provide sufficient information for policy-makers to make decisions on future training programmes. Based on our findings, we have developed an evidence-based framework, which incorporates but expands beyond the Kirkpatrick model, to provide conceptual and practical guidance that aids in the design of training programme evaluations better suited to meet the information needs of policy-makers and to inform policy decisions.

Large funding inflows, limited local capacity and emerging disease control priorities: a situational assessment of tuberculosis control in Myanmar.

There are numerous challenges in planning and implementing effective disease control programmes in Myanmar, which is undergoing internal political and economic transformations whilst experiencing massive inflows of external funding. The objective of our study-involving key informant discussions, participant observations and linked literature reviews-was to analyse how tuberculosis (TB) control strategies in Myanmar are influenced by the broader political, economic, epidemiological and health systems context using the Systemic Rapid Assessment conceptual and analytical framework. Our findings indicate that the substantial influx of donor funding, in the order of one billion dollars over a 5-year period, may be too rapid for the country's infrastructure to effectively utilize. TB control strategies thus far have tended to favour medical or technological approaches rather than infrastructure development, and appear to be driven more by perceived urgency to 'do something' rather informed by evidence of cost-effectiveness and sustainable long-term impact. Progress has been made towards ambitious targets for scaling up treatment of drug-resistant TB, although there are concerns about ensuring quality of care. We also find substantial disparities in health and funding allocation between regions and ethnic groups, which are related to the political context and health system infrastructure. Our situational assessment of emerging TB control strategies in this transitioning health system indicates that large investments by international donors may be pushing Myanmar to scale up TB and drug-resistant TB services too quickly, without due consideration given to the health system (service delivery infrastructure, human resource capacity, quality of care, equity) and epidemiological (evidence of effectiveness of interventions, prevention of new cases) context.

Patterns of medication use and factors associated with antibiotic use among adult fever patients at Singapore primary care clinics.

BACKGROUND: Antimicrobial resistance is a public health problem of global importance. In Singapore, much focus has been given to antibiotic usage patterns in hospital settings. Data on antibiotic use in primary care is lacking. We describe antibiotic usage patterns and assess factors contributing to antibiotic usage among adults presenting with acute febrile illness (AFI) in primary care settings in Singapore. METHODS: We analyzed data from the Early Dengue infection and outcome study. Adults with AFI presenting at 5 Singapore polyclinics were included. We used multivariable logistic regression to assess demographic, clinical and laboratory factors associated with antibiotic usage among adults with AFI. RESULTS: Between December 2007 and February 2013, 1884 adult AFI patients were enrolled. Overall, 16% of adult AFI patients reported antibiotic use. We observed a rise in the use of over-the-counter medications in late 2009 and a decrease in antibiotic use during 2010, possibly related to the outbreak of pandemic influenza A H1N1 virus. After adjusting for age, gender, polyclinic and year of enrolment, the following factors were associated with higher odds of antibiotic use: living in landed property (compared to public housing) (OR = 1.73; 95% CI: 1.06-2.80); body mass index (BMI) <18.5 (OR = 1.87; 95% CI: 1.19-2.93); elevated white blood cell (WBC) count (OR = 1.98; 95% CI: 1.42-2.78); and persistence of initial symptoms at 2-3 days follow-up with OR (95% CI) for categories of 1, 2, 3, and ≥4 persisting symptoms being 2.00 (1.38-2.92), 2.67 (1.80-3.97), 4.26 (2.73-6.64), and 2.79 (1.84-4.24) respectively. CONCLUSIONS: Our study provides insights on antibiotic usage among adult patients presenting to primary care clinics with febrile illness, and suggests that high socio-economic status, and risk factors of a severe illness, that is, low BMI and persistence of initial symptoms, are associated with higher antibiotic use. Further work to understand trends of antibiotic usage in both private and public primary care clinics, and factors that influence patient expectation and physician prescribing of antibiotics is important.

Characteristics of acute febrile illness and determinants of illness recovery among adults presenting to Singapore primary care clinics.

BACKGROUND: Undifferentiated acute febrile illness (AFI) is a common presentation among adults in primary care settings in Singapore but large gaps exist in the understanding of the characteristics of these patients. We studied clinical and epidemiological characteristics of AFI patients and factors associated with delayed recovery from AFI. METHODS: We performed a secondary data analysis using data from the Early DENgue infection and outcome (EDEN) study on 2046 adult patients presenting at 5 Singapore polyclinics between December 2007 and February 2013 with a history of fever (≥38 °C) for less than 72 h. We used an accelerated failure time model to investigate factors associated with delayed recovery from AFI. RESULTS: The mean age of patients was 36.6 years, 65 % were male, 51 % were of Chinese ethnicity, and 75 % lived in public housing. Median illness duration was 5 days (interquartile range, 3-7). In multivariable analysis, the unemployed and white collar workers had longer illness duration compared with blue collar workers (time ratio (TR), 1.10; 95 % confidence interval (CI), 1.03-1.17 and TR, 1.08; 95 % CI, 1.02-1.15, respectively). Patients with more symptoms at initial consultation had slower recovery (TR, 1.03 per additional symptom; 95 % CI, 1.02-1.03). Other clinical factors were also associated with longer duration of illness, including use of analgesics (TR, 1.21; 95 % CI, 1.15-1.28); use of cough medicines (TR, 1.14; 95 % CI, 1.08-1.20); use of antibiotics (TR, 1.14; 95 % CI, 1.07-1.21); and hospitalization (TR, 1.59; 95 % CI, 1.39-1.82). Compared to patients with normal WBC count at first consultation, those with low WBC count had slower recovery (TR, 1.14; 95 % CI, 1.07-1.21), while the reverse was observed among patients with high WBC count (TR, 0.94; 95 % CI, 0.88-1.00). CONCLUSIONS: Differences in illness duration among different types of employment may reflect differences in their underlying general health status. Early identification of factors delaying recovery could help triage management in a primary care setting. In-depth characterization of fever etiology in Singapore will improve surveillance and control activities.