Evaluation of the long-term outcomes of the University of Michigan's Practice-Oriented Research Training (PORT) program.

The University of Michigan created the Practice-Oriented Research Training (PORT) program and implemented it between 2008 and 2018. The PORT program provided research training and funding opportunities for allied healthcare professionals. The program consisted of weekly didactics and group discussion related to topics relevant to developing specific research ideas into projects and funding for a mentored research project for those who submitted a competitive grant application. The goal of this evaluation was to assess the long-term impact of the PORT program on the research careers of the participants. Ninety-two participants (74 staff and 18 faculty) participated in both phases of the program. A mixed-methods approach to evaluation was used; 25 participants who received funding for their research completed surveys, and semi-structured interviews were conducted with eight program participants. In addition, data were collected on participants' publication history. Fifteen out of the 74 staff participants published 31 first-authored papers after participating in PORT. Twelve out of 15 staff participants who published first-authored papers did so for the first time after participating in the PORT program. Results of quantitative and qualitative analyses suggest that the PORT program had positive impacts on both participants and the research community.

Evaluation of an online research best practices training for community health workers and promotoras.

Introduction: Community health workers and promotoras (CHW/Ps) have a fundamental role in facilitating research with communities. However, no national standard training exists as part of the CHW/P job role. We developed and evaluated a culturally- and linguistically tailored online research best practices course for CHW/Ps to meet this gap. Methods: After the research best practices course was developed, we advertised the opportunity to CHW/Ps nationwide to complete the training online in English or Spanish. Following course completion, CHW/Ps received an online survey to rate their skills in community-engaged research and their perceptions of the course using Likert scales of agreement. A qualitative content analysis was conducted on open-ended response data. Results: 104 CHW/Ps completed the English or Spanish course (n = 52 for each language; mean age 42 years SD ± 12); 88% of individuals identified as female and 56% identified as Hispanic, Latino, or Spaniard. 96%-100% of respondents reported improvement in various skills. Nearly all CHW/Ps (97%) agreed the course was relevant to their work, and 96% felt the training was useful. Qualitative themes related to working more effectively as a result of training included enhanced skills, increased resources, and building bridges between communities and researchers. Discussion: The CHW/P research best practices course was rated as useful and relevant by CHW/Ps, particularly for communicating about research with community members. This course can be a professional development resource for CHW/Ps and could serve as the foundation for a national standardized training on their role related to research best practices.

Reimagining a summer research program during COVID: Strategies for enhancing research workforce diversity.

Well-designed, accessible short-term research training programs are needed to recruit and retain underrepresented persons into clinical and translational research training programs and diversify the workforce. The Michigan Institute for Clinical and Health Research developed a summer research program, training over 270 students in 15 years. In response to the 2020 COVID-19 pandemic, we pivoted swiftly from an in-person format to a fully remote format. We describe this process, focusing on factors of diversity, equity, and inclusion including enabling student participation in remote research activities. We collected data about students' learning experiences since the program's inception; therefore, we could evaluate the impact of remote vs. in-person formats. We examined data from five cohorts: three in-person (2017-2019; n = 57) and two remote (2020-2021; n = 45). While there was some concern about the value of participating in a remote format, overall students in both formats viewed the program favorably, with students in the remote cohorts rating some aspects of the program significantly more favorably. In addition, more students who identified as Black or African American participated in the remote format than in the in-person format. We describe lessons learned from this unprecedented challenge and future program directions.

Assessments of Research Competencies for Clinical Investigators: A Systematic Review.

Although there is extensive research literature on clinical skill competencies and the use of competency-based frameworks for clinical research, the appropriate methods to assess these competencies are not as well understood. Our goal in this systematic literature review is to identify, compare, and critique assessments of clinical research competencies. Articles were included in this review if they examined clinical investigators or clinical investigators in training, focused on research-based skills, and included some form of assessment of research-based competencies. A total of 76 articles were identified as part of the initial search; 16 met the criteria for inclusion. Two types of assessments of clinical research competence were identified: subjective self-assessments (n = 13) and objective tests (n = 6). These assessments covered a wide range of competencies, but there were no competency domains common to all. Most assessments had limited validation. Training was consistently associated with self-assessed competence but had little relationship to objective measures of competence. In contrast, experience was consistently associated with objectively assessed competence but not with self-assessed competence. These findings have important implications for those interested in assessing medical education programs. We describe a recommended standard for validity for assessments used for the purposes of summative program assessment.

The Research Objective Structured Clinical Exam (R-OSCE): An Innovative Tool to Assess Clinical and Translational Research Competencies.

This article was migrated. The article was marked as recommended. Introduction The Research Objective Structured Clinical Exam (R-OSCE) described in this paper was designed as part of a comprehensive program to assess competency in specific domains of clinical and translational research (CTR) for students enrolled in a 12-week introductory summer research program. Methods The program curriculum was mapped to core competencies developed by the National Center for Translational Science (NCATS) and used to develop R-OSCE stations. Twelve stations were developed, with five administered during orientation as a practice test and seven administered post-program. A scoring rubric using an anchored scale of 1-5 was developed and six qualified raters were trained in its use. The exam was self-paced and delivered through a secure online computer-based platform. Results Forty-seven students (three cohorts) completed the post-program R-OSCE. Most respondents scored at 3 (developing competence) or higher on most stations for both the practice and post-program exams, the exceptions being the stations involving writing research questions and engaging communities in research. Students indicated they liked demonstrating CTR skills through the R-OSCE. Most participants agreed that exam tasks were related to stated program competencies and that stations were realistic. Discussion The R-OSCE is best used as part of a comprehensive assessment program and may be useful in providing formative feedback to trainees that they can share with their mentors. Additionally, this study demonstrated that it could feasibly be used to evaluate the effectiveness of research education programs. However, additional time was needed to train raters and score the R-OSCE. Modifications were made to administer the exam through use of an online format with a modest budget. The computer-based format provides a solution to the current need for assessments that can be administered remotely.

The DIAMOND portal: An innovative platform for sharing translational research workforce training and assessment resources.

Although several initiatives have produced core competency domains for training the translational science workforce, training resources to help clinical research professionals advance these skills reside primarily within local departments or institutions. The Development, Implementation, and AssessMent of Novel Training in Domain (DIAMOND) project was designed to make this training more readily and publicly available. DIAMOND includes a digital portal to catalog publicly available educational resources and an ePortfolio to document professional development. DIAMOND is a nationally crowdsourced, federated, online catalog providing a platform for practitioners to find and share training and assessment materials. Contributors can share their own educational materials using a simple intake form that creates an electronic record; the portal enables users to browse or search this catalog of digital records and access the resources. Since September 2018, the portal has been visited more than 5,700 times and received over 280 contributions from professionals. The portal facilitates opportunities to connect and collaborate regarding future applications of these resources. Consequently, growing the collection and increasing numbers of both contributors and users remains a priority. Results from a small subset of users indicated over half accomplished their purpose for visiting the site, while qualitative results showed that users identified several benefits and helpful features of the ePortfolio.

An Internet-based tailored hearing protection intervention for firefighters: development process and users' feedback.

BACKGROUND: Noise-induced hearing loss is a significant occupational injury for firefighters exposed to intermittent noise on the job. It is important to educate firefighters about using hearing protection devices whenever they are exposed to loud noise. Computer technology is a relatively new health education approach and can be useful for tailoring specific aspects of behavioral change training. The purpose of this study is to present the development process of an Internet-based tailored intervention program and to assess its efficacy. METHOD: The intervention programs were implemented for 372 firefighters (mean age = 44 years, Caucasian = 82%, male = 95%) in three states (California, Illinois, and Indiana). The efficacy was assessed from firefighters' feedback through an Internet-based survey. RESULTS: A multimedia Internet-based training program was developed through (a) determining program content and writing scripts, (b) developing decision-making algorithms for tailoring, (c) graphic design and audio and video productions, (d) creating computer software and a database, and (e) postproduction quality control and pilot testing. Participant feedback regarding the training has been very positive. Participants reported that they liked completing the training via computer (83%) and also that the Internet-based training program was well organized (97%), easy to use (97%), and effective (98%) and held their interest (79%). Almost all (95%) would recommend this Internet training program to other firefighters. CONCLUSION: Interactive multimedia computer technology using the Internet was a feasible mode of delivery for a hearing protection intervention among firefighters. Participants' favorable feedback strongly supports the continued utilization of this approach for designing and developing interventions to promote healthy behaviors.

From research to practice: the importance of community collaboration in the translation process.

In order for effective interventions to make an impact on their target population, they must be successfully translated and disseminated to the organizations that will ultimately deliver them to those in need. Cuídate!, a culturally based intervention to reduce HIV sexual risk among Latino youth, was identified by the Centers for Disease Control and Prevention's (CDC) Prevention Research Synthesis (PRS) project as "best evidence" of intervention efficacy and selected as part of the CDC's Replicating Effective Programs (REP). The REP process consisted of the design, development, and field-testing of the Cuídate! program package in community-based, nonacademic settings. Project staff worked with CDC and community-based partners throughout the REP process. Community partners included a community advisory board (CAB) and four case agencies. Case agency staff participated in a facilitator training and subsequently implemented the Cuídate! program at their respective agencies. Process evaluation findings showed that facilitators were able to effectively use program materials and implement the program with fidelity. Adolescent participants reported they liked the program and would recommend the project to others. Only slight modifications to program and training materials were necessary following evaluation. Lessons learned included the importance of interdisciplinary collaboration and utilizing the resources available from each collaborative partner.

Recruitment and retention of Latino adolescents to a research study: lessons learned from a randomized clinical trial.

PURPOSE: To describe facilitators and barriers to participation and retention of Latino adolescents in a randomized clinical trial. DESIGN AND METHODS: Participants were part of a randomized clinical trial designed to reduce HIV sexual risk behavior among Latino youth. Responses from 106 randomly selected respondents from the 3-month follow-up were content analyzed. RESULTS: Four main facilitator patterns emerged: peer/family support, program incentives, commitment, and desire to help. Participation barriers included conflicts with other commitments, embarrassment, and lack of peer support. PRACTICE IMPLICATIONS: Recruitment and retention of Latino adolescents in research studies is critical to building a research base for nursing practice.

Selecting control interventions for clinical outcome studies.

In the current research environment the design and management of control groups is becoming more complex. The selection of a control group design is dependent on study goals, presence and quality of existing interventions, urgency of the problem or issue being addressed by the intervention, and factors related to the study site. The purpose of the presentation is to identify various approaches to the design of control groups in experimental studies and to identify their strengths, limitations, and applications. A case study exemplifies the issues associated with control group selection and design.

Effects of booster interventions on factory workers' use of hearing protection.

BACKGROUND: The provision of reinforcements or boosters to interventions is seen as a logical approach to enhancing or maintaining desired behavior. Empirical studies, however, have not confirmed the effectiveness of boosters nor assessed the optimum number of boosters or the timing for their delivery. OBJECTIVES: This randomized controlled trial contrasted the effect of four booster conditions (a). 30 days; (b). 90 days; (c). 30 and at 90 days; and (d). no boosters of the intervention to increase the use of hearing protection devices (HPDs). METHODS: A total of 1325 factory workers completed a computerized questionnaire and were randomly assigned to one of three computer-based (tailored, nontailored predictor-based, or control) multimedia interventions designed to increase the use of hearing protection devices. After the intervention, colorful boosters specific to the type of training received were mailed to workers' homes. Posttest measures of use were administered at the time of their next annual audiogram 6 to 18 months after the intervention. RESULTS Repeated measures of analysis of variance (ANOVA) showed a significant main effect for the booster (after 30 days) in the group that received tailored training (F[3442] = 2.722; p =.04). However, in the assessment of the interaction between time (pretest and posttest) and boosters (four groups), the ANOVA did not find significant differences in hearing protection device use for any of the training groups. To assess for significant differences between groups, post hoc comparisons were conducted at the pretest and posttest for the total sample and for the subsample of workers who reported using hearing protection devices less than 100% of the time needed. Sheffé contrasts by intervention group, gender, ethnicity, and hearing ability found no significant changes in the mean use of hearing protection devices for the booster groups. CONCLUSIONS: Although the provision of boosters represented a considerable commitment of resources, their use was not effective in this study. However, it would be premature to eliminate boosters of interventions. Further study is needed to explore the effects of different booster types for increasing the use of hearing protection devices, and to assess carefully the effects of boosters on other health behaviors in studies with controlled designs.

Effectiveness of a tailored intervention to increase factory workers' use of hearing protection.

BACKGROUND: In the United States it is estimated that more than 30 million workers are exposed to harmful levels of noise on the job. When engineering or administrative controls cannot be used to reduce noise, workers should always use hearing protection devices (HPDs) when exposed to loud noise to prevent noise-induced hearing loss (NIHL). Previous research has shown that workers do not always use HPDs when required; therefore, it is essential that workers assume personal responsibility for preventing NIHL by increasing their use of HPDs. OBJECTIVES: This study tested the effectiveness of an individually tailored multimedia intervention to increase use of HPDs by factory workers. METHODS: A randomized controlled design was used to compare the effects of a tailored intervention (n= 446) with two other interventions (a nontailored predictor-based intervention (n= 447) and a control intervention (n= 432)) on workers' self-reported use of HPDs 6 to 18 months following the intervention. RESULTS: Only those workers receiving the tailored intervention significantly increased their use of HPDs from pretest to posttest. However, this increase significantly differed from the nontailored group but not from the control group. CONCLUSIONS: Individually-tailored interventions offer promise for changing behavior. In light of the similarity between the results for the tailored intervention and the control intervention groups, further research is needed to understand barriers to HPD use and how to maximize the benefits of individually tailored interventions in this setting.