Feasibility of common bibliometrics in evaluating translational science.

INTRODUCTION: A pilot study by 6 Clinical and Translational Science Awards (CTSAs) explored how bibliometrics can be used to assess research influence. METHODS: Evaluators from 6 institutions shared data on publications (4202 total) they supported, and conducted a combined analysis with state-of-the-art tools. This paper presents selected results based on the tools from 2 widely used vendors for bibliometrics: Thomson Reuters and Elsevier. RESULTS: Both vendors located a high percentage of publications within their proprietary databases (>90%) and provided similar but not equivalent bibliometrics for estimating productivity (number of publications) and influence (citation rates, percentage of papers in the top 10% of citations, observed citations relative to expected citations). A recently available bibliometric from the National Institutes of Health Office of Portfolio Analysis, examined after the initial analysis, showed tremendous potential for use in the CTSA context. CONCLUSION: Despite challenges in making cross-CTSA comparisons, bibliometrics can enhance our understanding of the value of CTSA-supported clinical and translational research.

Mapping tobacco industry strategies in South East Asia for action planning and surveillance.

OBJECTIVE: To develop a comprehensive conceptual framework of tobacco industry tactics in four countries in South East Asia for the purpose of: (1) generating consensus on key areas of importance and feasibility for regional and cross country tobacco industry monitoring and surveillance; (2) developing measures to track and monitor the effects of the tobacco industry and to design counterstrategies; and (3) building capacity to improve tobacco control planning in the participating countries. DESIGN: A structured conceptualisation methodology known as concept mapping was used. The process included brainstorming, sorting and rating of statements describing industry activities. Statistical analyses used multidimensional scaling and cluster analysis. Interpretation of the maps was participatory, using regional tobacco control researchers, practitioners, and policy makers during a face to face meeting. PARTICIPANTS: 31 participants in this study come from the four countries represented in the project along with six people from the Johns Hopkins Blomberg School of Public Health. CONCLUSIONS: The map shows eight clusters of industry activities within the four countries. These were arranged into four general sectors: economics, politics, public relations and deception. For project design purposes, the map indicates areas of importance and feasibility for monitoring tobacco industry activities and serves as a basis for an initial discussion about action planning. Furthermore, the development of the map used a consensus building process across different stakeholders or stakeholder agencies and is critical when developing regional, cross border strategies for tracking and surveillance.

Development of a model of the tobacco industry's interference with tobacco control programmes.

OBJECTIVE: To construct a conceptual model of tobacco industry tactics to undermine tobacco control programmes for the purposes of: (1) developing measures to evaluate industry tactics, (2) improving tobacco control planning, and (3) supplementing current or future frameworks used to classify and analyse tobacco industry documents. DESIGN: Web based concept mapping was conducted, including expert brainstorming, sorting, and rating of statements describing industry tactics. Statistical analyses used multidimensional scaling and cluster analysis. Interpretation of the resulting maps was accomplished by an expert panel during a face-to-face meeting. SUBJECTS: 34 experts, selected because of their previous encounters with industry resistance or because of their research into industry tactics, took part in some or all phases of the project. RESULTS: Maps with eight non-overlapping clusters in two dimensional space were developed, with importance ratings of the statements and clusters. Cluster and quadrant labels were agreed upon by the experts. CONCLUSIONS: The conceptual maps summarise the tactics used by the industry and their relationships to each other, and suggest a possible hierarchy for measures that can be used in statistical modelling of industry tactics and for review of industry documents. Finally, the maps enable hypothesis of a likely progression of industry reactions as public health programmes become more successful, and therefore more threatening to industry profits.

Ethical and scientific features of cutoff-based designs of clinical trials: a simulation study.

Cutoff-based clinical trial designs are geared towards balancing ethical and scientific concerns when it is deemed unethical or infeasible to randomize all patients to study treatments. In a cutoff-based design with randomization, patients who are the least sick based on a quantitative baseline indicator are assigned the control treatment, patients who are the most sick based on the same indicator are assigned to test treatment, and patients who are moderately sick based on the indicator are randomly assigned. Simulations were conducted to examine statistical efficiency and potential bias for designs with varying amounts of cutoff-based assignment and randomization. All design variations yielded unbiased estimates of a main treatment effect and a linear interaction effect. While randomization tends to lead to greater efficiency (or lower standard errors of treatment effect), the correlation between the binary treatment variable and baseline assignment variable completely determines the efficiency of a design.

Using concept mapping to develop a conceptual framework of staff's views of a supported employment program for individuals with severe mental illness.

This article describes the use of concept mapping to develop a pictorial multivariate conceptual framework of staff views of a program of supported employment (SE) for individuals with severe mental illness. The SE program involves extended individualized supported employment for clients through a mobile job support worker (MJSW) who maintains contact with the client after job placement and supports the client in various ways. All 14 staff members of a psychiatric rehabilitation agency with assignments associated with the SE program. They brainstormed a large number of specific program activity statements (N = 96), sorted and rated the statements, and interpreted the map that was produced through multidimensional scaling and hierarchical cluster analysis. The resulting map enabled identification of 4 issues that should be included in any theory of SE programs--the specific activity sequences that characterize the program itself; the pattern of local program evolution; the definition of program staff roles; and the influence of key contextual factors such as the client's family or the program's administrative structure. The implications of concept mapping methodology for theory development and program evaluation are considered.

An illustrative statistical analysis of cutoff-based randomized clinical trials.

Cutoff-based randomized clinical trials (RCTs) are designed to balance ethical and scientific concerns. Patients scoring below a cutoff score on a baseline measure (i.e. the least severely ill) are assigned to the control-treated group, those scoring above a second cutoff score (i.e. the most severely ill) are assigned to the test-treated group, and those scoring within the interval (i.e. the moderately ill) are randomly assigned. This paper provides a formal illustration on the statistical analysis of cutoff-based RCTs using data from the Xanax Cross-National Collaborative Study. To overcome problems specific to cutoff-based designs, we generally recommend a backward elimination approach that tests interactions before main effects.

Cutoff assignment strategies for enhancing randomized clinical trials.

The randomized clinical trial (RCT) is the preferred method for assessing the efficacy of treatments. Recent ethical and logistical criticisms suggest that new variations of the traditional RCT are needed. Some of these criticisms may be addressed with new hybrid designs that combine random assignment with assignment by one or more cutoff values on a baseline variable (e.g., severity of illness). In a simple version of such a "cutoff-based" RTC, persons scoring below a cutoff score on a baseline measure (i.e., the least severely ill) are automatically assigned to the control-treated group, those scoring above a second, higher cutoff (i.e., the most ill) are automatically assigned to the test-treated group, and those scoring in the interval between the cutoff scores (i.e., the moderately ill) are randomly assigned to either group. Depending on the baseline score, the patient is assigned to treatment either randomly or by the need-based, clinically related baseline score. Six cutoff-based design variations are studied via simulations and compared with the traditional RCT and the single-cutoff (i.e., regression-discontinuity) design. All variations yield unbiased estimates of the treatment effect but estimates differ in efficiency, with the RCT being most efficient and the single-cutoff design being least efficient. Secondary analyses of data from the Cross-National Collaborative Study of the Effects of Alprazolam (Xanax) on panic are conducted for each variation by selectivity discarding cases from the original dataset to stimulate cutoff-based assignment. The results confirm the simulations and illustrate how cutoff-based designs might look with real data.

Conceptualization for planning and evaluation.

Conceptualization is a central task in planning and evaluation. There is a need to develop conceptualization methods which can help multiple constituency groups collaborate on the development of conceptual frameworks which can guide the planning and evaluation effort. A general model for conceptualization methods is presented along with considerations for implementation. This model is illustrated in two planning and evaluation studies which involved a broad range of constituent groups within the organizations in the development of "concept maps" which could guide subsequent work. Some general implications of conceptualization methods for planning and evaluation are discussed.