Second Global Symposium on Health Systems Research: a conference impact evaluation.

Evaluation researchers have confirmed the importance of conference evaluation, but there remains little research on the topic, perhaps in part because evaluation methodology related to conference impact is underdeveloped. We conducted a study evaluating a 4-day long health conference, the Second Global Symposium on Health Systems Research (HSR), which took place in Beijing in November 2012. Using a conference evaluation framework and a mixed-methods approach that involved in-conference surveys, in-conference interviews and 7-month post-conference interviews, we evaluated the impact of the Symposium on attendees' work and the field of health systems research. The three major impacts on participants' work were new knowledge, new skills and new networks, and many participants were able to provide examples of how obtaining new knowledge, skills or collaborations had changed the way they conduct their work. Participants noted that the Symposium influenced the field of HSR only in so far as it influenced the capacity of stakeholders, but did not lead to any high level agenda or policy changes, perhaps due to the insufficient length of time (7 months) between the Symposium and post-conference follow-up. This study provides an illustration of a framework useful for conference organizers in the evaluation of future conferences, and of a unique methodology for evaluation researchers.

Evaluation of the international forum on evidence informed health policymaking: Addis Ababa, Ethiopia - 27 to 31 August 2012.

BACKGROUND: Meetings and conferences are often used as a tool to disseminate information, network with colleagues, and/or set direction for a field of study, but there is little evidence to support whether such events achieve their objectives. This study evaluates the International Forum on Evidence Informed Health Policymaking (EIHP), a three-day meeting held in Addis Ababa, Ethiopia, in 2012, to determine the success of the meeting based on pre-determined objectives. METHODS: The evaluation strategy was developed based on a previously published conference evaluation framework and operationalized as an end-of-conference participant survey that incorporated both process (programme/organization) and outcome measures (potential changes in behaviour). RESULTS: Sixty seven of approximately 121 attendees filled out a questionnaire (a 55% response rate) and, overall, participants rated the programme components and plenary sessions very highly. The top three benefits reported by participants were: i) sharing experiences and lessons learned (75%); ii) new opportunities for future collaboration (69%); and iii) new knowledge (67%). Conversely, only 25% or less of meeting participants reported an intent to utilize any of the potential benefits highlighted in the questionnaire, with the notable exception of pursuing new opportunities for future collaboration. CONCLUSIONS: The evaluation findings suggest that the International Forum achieved its objectives of sharing experiences with EIHP and providing opportunities for networking among EIHP initiatives, although there are limited prospects for direct improvements to efforts to support EIHP.

A scoping review about conference objectives and evaluative practices: how do we get more out of them?

Large multi-day conferences have often been criticized as ineffective ways to improve social outcomes and to influence policy or practice. Unfortunately, many conference evaluations have also been inadequate in determining the impact of a conference on its associated social sector, with little evidence gathered or analyzed to substantiate or refute these criticisms. The aim of this scoping review is to investigate and report stakeholders' objectives for planning or participating in large multi-day conferences and how these objectives are being evaluated. We conducted a scoping review supplemented by a small number of key informant interviews. Eight bibliographic databases were systematically searched to identify papers describing conference objectives and/or evaluations. We developed a conference evaluation framework based on theoretical models and empirical findings, which structured the descriptive synthesis of the data. We identified 3,073 potential papers for review, of which 44 were included in this study. Our evaluation framework connects five key elements in planning a conference and its evaluation (number in brackets refers to number of themes identified): conference objectives (8), purpose of evaluation (7), evaluation methods (5), indicators of success (9) and theories/models (8). Further analysis of indicators of success identified three categories of indicators with differing scopes (i.e. immediate, prospective or follow-up) as well as empirical links between the purpose of evaluations and these indicators. Conference objectives and evaluations were largely correlated with the type of conference (i.e. academic, political/governmental or business) but diverse overall. While much can be done to improve the quality and usefulness of conference evaluations, there are innovative assessments that are currently being utilized by some conferences and warrant further investigation. This review provides conference evaluators and organizers a simple resource to improve their own assessments by highlighting and categorizing potential objectives and evaluation strategies.