Community-researcher partnerships at NIAID HIV/AIDS clinical trials sites: insights for evaluation and enhancement.

BACKGROUND: Community engagement has been a cornerstone of National Institute of Allergy and Infectious Diseases (NIAID)'s HIV/AIDS clinical trials programs since 1990. Stakeholders now consider this critical to success, hence the impetus to develop evaluation approaches. OBJECTIVES: The purpose was to assess the extent to which community advisory boards (CABs) at HIV/AIDS trials sites are being integrated into research activities. METHODS: CABs and research staff (RS) at NIAID research sites were surveyed for how each viewed (a) the frequency of activities indicative of community involvement, (b) the means for identifying, prioritizing, and supporting CAB needs, and (c) mission and operational challenges. RESULTS: Overall, CABs and RS share similar views about the frequency of community involvement activities. Cluster analysis reveals three groups of sites based on activity frequency ratings, including a group notable for CAB-RS discordance. CONCLUSIONS: Assessing differences between community and researcher perceptions about the frequency of and challenges posed by specific engagement activities may prove useful in developing evaluation tools for assessing community engagement in collaborative research settings.

Evaluating research and impact: a bibliometric analysis of research by the NIH/NIAID HIV/AIDS clinical trials networks.

Evaluative bibliometrics uses advanced techniques to assess the impact of scholarly work in the context of other scientific work and usually compares the relative scientific contributions of research groups or institutions. Using publications from the National Institute of Allergy and Infectious Diseases (NIAID) HIV/AIDS extramural clinical trials networks, we assessed the presence, performance, and impact of papers published in 2006-2008. Through this approach, we sought to expand traditional bibliometric analyses beyond citation counts to include normative comparisons across journals and fields, visualization of co-authorship across the networks, and assess the inclusion of publications in reviews and syntheses. Specifically, we examined the research output of the networks in terms of the a) presence of papers in the scientific journal hierarchy ranked on the basis of journal influence measures, b) performance of publications on traditional bibliometric measures, and c) impact of publications in comparisons with similar publications worldwide, adjusted for journals and fields. We also examined collaboration and interdisciplinarity across the initiative, through network analysis and modeling of co-authorship patterns. Finally, we explored the uptake of network produced publications in research reviews and syntheses. Overall, the results suggest the networks are producing highly recognized work, engaging in extensive interdisciplinary collaborations, and having an impact across several areas of HIV-related science. The strengths and limitations of the approach for evaluation and monitoring research initiatives are discussed.

Integrating utilization-focused evaluation with business process modeling for clinical research improvement.

New discoveries in basic science are creating extraordinary opportunities to design novel biomedical preventions and therapeutics for human disease. But the clinical evaluation of these new interventions is, in many instances, being hindered by a variety of legal, regulatory, policy and operational factors, few of which enhance research quality, the safety of study participants or research ethics. With the goal of helping increase the efficiency and effectiveness of clinical research, we have examined how the integration of utilization-focused evaluation with elements of business process modeling can reveal opportunities for systematic improvements in clinical research. Using data from the NIH global HIV/AIDS clinical trials networks, we analyzed the absolute and relative times required to traverse defined phases associated with specific activities within the clinical protocol lifecycle. Using simple median duration and Kaplan-Meyer survival analysis, we show how such time-based analyses can provide a rationale for the prioritization of research process analysis and re-engineering, as well as a means for statistically assessing the impact of policy modifications, resource utilization, re-engineered processes and best practices. Successfully applied, this approach can help researchers be more efficient in capitalizing on new science to speed the development of improved interventions for human disease.

Developing a conceptual framework for an evaluation system for the NIAID HIV/AIDS clinical trials networks.

Globally, health research organizations are called upon to re-examine their policies and practices to more efficiently and effectively address current scientific and social needs, as well as increasing public demands for accountability.Through a case study approach, the authors examine an effort undertaken by the National Institute of Allergy & Infectious Diseases (part of the National Institutes of Health, Department of Health & Human Services, United States Government) to develop an evaluation system for its recently restructured HIV/AIDS clinical trials program. The challenges in designing, operationalizing, and managing global clinical trials programs are considered in the context of large scale scientific research initiatives.Through a process of extensive stakeholder input, a framework of success factors was developed that enables both a prospective view of the elements that must be addressed in an evaluation of this research and a current state assessment of the extent to which the goals of the restructuring are understood by stakeholders across the DAIDS clinical research networks.

The critical role of conflict resolution in teams: a close look at the links between conflict type, conflict management strategies, and team outcomes.

This article explores the linkages between strategies for managing different types of conflict and group performance and satisfaction. Results from a qualitative study of 57 autonomous teams suggest that groups that improve or maintain top performance over time share 3 conflict resolution tendencies: (a) focusing on the content of interpersonal interactions rather than delivery style, (b) explicitly discussing reasons behind any decisions reached in accepting and distributing work assignments, and (c) assigning work to members who have the relevant task expertise rather than assigning by other common means such as volunteering, default, or convenience. The authors' results also suggest that teams that are successful over time are likely to be both proactive in anticipating the need for conflict resolution and pluralistic in developing conflict resolution strategies that apply to all group members.

An examination of community members', researchers' and health professionals' perceptions of barriers to minority participation in medical research: an application of concept mapping.

OBJECTIVE: Some of the most promising medical treatments are currently being developed and used in clinical trials. In the US, rates of chronic disease among racial/ethnic minorities are disproportionately high. Unfortunately, the rates of minority participation in medical research are low, and the reasons are unclear. This study seeks to contribute to the body of knowledge that is currently available relating to the specific barriers to racial/ethnic minority participation in medical research through the conceptualization and measurement of these barriers. DESIGN: Study participants included a convenience sample obtained from the National Cancer Institute's Special Populations Networks, and consisted of practitioners, researchers and community members who specialize in research related to the treatment and prevention of cancer. A structured form of concept mapping (Trochim 1989) was the methodology used in this study. The concept mapping process has three specific phases: (1) project planning - development of project focus statements and sample selection (2) idea generation and structuring and (3) analysis and interpretation. This method is analogous to a more formalized and structured focus group approach, and involved the gathering of 149 ideas and the sorting of 70 statements. Comparisons across participant demographics were conducted and are presented in the form of pattern matches. RESULTS: The findings of this study suggest that there are two specific areas where barriers to minority participation may be addressed. The first area is the research system, specifically, the manner in which research studies are designed and implemented, including referral, recruitment and retention of racial/ethnic minorities. The data suggest that recruitment and retention will be aided by addressing patient concerns regarding the research process, and assuaging fears about clinical trials. The second area pertains to minority perceptions of the research process based on history and personal experiences. CONCLUSION: There appears to be a difference in the barriers to participation as defined by community members themselves, and health professionals' perceptions of these barriers. Increased inclusion of minorities in the design, management, and implementation of medical research studies would help mitigate negative perceptions of the research process, and serve to increase participation among racial/ethnic minorities.

Evaluating program effectiveness using the regression point displacement design.

Most health care initiatives are evaluated using observational study designs in lieu of randomized controlled trials (RCT) due primarily to resource limitations. However, although observational studies are less expensive to implement and evaluate, they are also more problematic in determining causality than the RCT. This trade off is most apparent in the initial planning stage of program development. An RCT is generally preferred though the cost of implementing a pilot program using the RCT might outstrip the potential benefit if the desired results are not obtained. This article describes a simple quasi-experimental model called the regression point displacement (RPD) design, which compares the prepost results of a single or multiple treatment groups to that of a control population. This design has shown great potential in evaluating health care pilot programs or demonstration projects-especially those that are community based-due to its relative ease of implementation and low cost of analysis.

An introduction to concept mapping as a participatory public health research method.

In this article, the authors introduce concept mapping as a useful participatory research method for public health researchers interested in generating hypotheses and developing theory. The authors first provide an overview of concept mapping, which combines qualitative approaches with quantitative analytical tools to produce visual displays of the relationship between ideas. Then, they present an illustrative research application of the method to the exploration of women's perceptions of the relationship between residential neighborhood factors and intimate partner violence experiences. They give attention to the data collection and analysis procedures and to demonstrating the intricacies of using concept mapping for public health research purposes. Finally, the article concludes with a discussion of the unique contributions and challenges associated with concept mapping.

Setting objectives for community and systems change: an application of concept mapping for planning a statewide health improvement initiative.

The Hawaii Department of Health (HDOH) used concept mapping techniques to engage local stakeholders and national subject area experts in defining the community and system factors that affect individuals' behaviors related to tobacco, nutrition, and physical activity. Over eight working days, project participants brainstormed 496 statements (edited to a final set of 90), which were then sorted and rated for their importance and feasibility. A sequence of multivariate statistical analyses, including multidimensional scaling and hierarchical cluster analysis, generated maps and figures that were then interpreted by project stakeholders. The results were immediately incorporated into an official plan, approved by the governor and state legislature, recommending how Hawaii's tobacco settlement resources could be used to create sustainable changes in population health. The results also provide empirical support for the premise that both community and systems factors ought to be considered when planning comprehensive health improvement initiatives.