Evaluation of the long-term outcomes of the University of Michigan's Practice-Oriented Research Training (PORT) program.

The University of Michigan created the Practice-Oriented Research Training (PORT) program and implemented it between 2008 and 2018. The PORT program provided research training and funding opportunities for allied healthcare professionals. The program consisted of weekly didactics and group discussion related to topics relevant to developing specific research ideas into projects and funding for a mentored research project for those who submitted a competitive grant application. The goal of this evaluation was to assess the long-term impact of the PORT program on the research careers of the participants. Ninety-two participants (74 staff and 18 faculty) participated in both phases of the program. A mixed-methods approach to evaluation was used; 25 participants who received funding for their research completed surveys, and semi-structured interviews were conducted with eight program participants. In addition, data were collected on participants' publication history. Fifteen out of the 74 staff participants published 31 first-authored papers after participating in PORT. Twelve out of 15 staff participants who published first-authored papers did so for the first time after participating in the PORT program. Results of quantitative and qualitative analyses suggest that the PORT program had positive impacts on both participants and the research community.

Implementing a mentoring program for clinical research professionals: A novel professional development initiative for university health research staff.

Clinical and translational research relies on a well-trained workforce, but mentorship programs designed expressly for this workforce are lacking. This paper presents the development of a mentoring program for research staff and identifies key programmatic outcomes. Research staff participating in this program were matched with a senior mentor. Focus groups were conducted to identify key program outcomes. Surveys were administered throughout the program period to assess participants' experience, gains in skill, and subsequent careers. Analysis of the resultant qualitative and quantitative data are used to characterize the implementation and impact of the program. A total of 47 mentees and 30 mentors participated in program between 2018 and 2023. A comprehensive logic model of short-, intermediate- and long-term outcomes was developed. Participants reported positive valuations of every programmatic outcome assessed including their program experience, learning and research careers. The pool of available mentors also grew as new mentors were successfully recruited for each cohort. This mentorship program developed and implemented by senior research staff successfully provided junior research staff with professional development support, mentorship, and professional development opportunities. Junior and senior health research staff built mentoring relationships that advanced their clinical and translational research careers.

Evaluation of an online research best practices training for community health workers and promotoras.

Introduction: Community health workers and promotoras (CHW/Ps) have a fundamental role in facilitating research with communities. However, no national standard training exists as part of the CHW/P job role. We developed and evaluated a culturally- and linguistically tailored online research best practices course for CHW/Ps to meet this gap. Methods: After the research best practices course was developed, we advertised the opportunity to CHW/Ps nationwide to complete the training online in English or Spanish. Following course completion, CHW/Ps received an online survey to rate their skills in community-engaged research and their perceptions of the course using Likert scales of agreement. A qualitative content analysis was conducted on open-ended response data. Results: 104 CHW/Ps completed the English or Spanish course (n = 52 for each language; mean age 42 years SD ± 12); 88% of individuals identified as female and 56% identified as Hispanic, Latino, or Spaniard. 96%-100% of respondents reported improvement in various skills. Nearly all CHW/Ps (97%) agreed the course was relevant to their work, and 96% felt the training was useful. Qualitative themes related to working more effectively as a result of training included enhanced skills, increased resources, and building bridges between communities and researchers. Discussion: The CHW/P research best practices course was rated as useful and relevant by CHW/Ps, particularly for communicating about research with community members. This course can be a professional development resource for CHW/Ps and could serve as the foundation for a national standardized training on their role related to research best practices.

Reimagining a summer research program during COVID: Strategies for enhancing research workforce diversity.

Well-designed, accessible short-term research training programs are needed to recruit and retain underrepresented persons into clinical and translational research training programs and diversify the workforce. The Michigan Institute for Clinical and Health Research developed a summer research program, training over 270 students in 15 years. In response to the 2020 COVID-19 pandemic, we pivoted swiftly from an in-person format to a fully remote format. We describe this process, focusing on factors of diversity, equity, and inclusion including enabling student participation in remote research activities. We collected data about students' learning experiences since the program's inception; therefore, we could evaluate the impact of remote vs. in-person formats. We examined data from five cohorts: three in-person (2017-2019; n = 57) and two remote (2020-2021; n = 45). While there was some concern about the value of participating in a remote format, overall students in both formats viewed the program favorably, with students in the remote cohorts rating some aspects of the program significantly more favorably. In addition, more students who identified as Black or African American participated in the remote format than in the in-person format. We describe lessons learned from this unprecedented challenge and future program directions.

Assessments of Research Competencies for Clinical Investigators: A Systematic Review.

Although there is extensive research literature on clinical skill competencies and the use of competency-based frameworks for clinical research, the appropriate methods to assess these competencies are not as well understood. Our goal in this systematic literature review is to identify, compare, and critique assessments of clinical research competencies. Articles were included in this review if they examined clinical investigators or clinical investigators in training, focused on research-based skills, and included some form of assessment of research-based competencies. A total of 76 articles were identified as part of the initial search; 16 met the criteria for inclusion. Two types of assessments of clinical research competence were identified: subjective self-assessments (n = 13) and objective tests (n = 6). These assessments covered a wide range of competencies, but there were no competency domains common to all. Most assessments had limited validation. Training was consistently associated with self-assessed competence but had little relationship to objective measures of competence. In contrast, experience was consistently associated with objectively assessed competence but not with self-assessed competence. These findings have important implications for those interested in assessing medical education programs. We describe a recommended standard for validity for assessments used for the purposes of summative program assessment.

The Research Objective Structured Clinical Exam (R-OSCE): An Innovative Tool to Assess Clinical and Translational Research Competencies.

This article was migrated. The article was marked as recommended. Introduction The Research Objective Structured Clinical Exam (R-OSCE) described in this paper was designed as part of a comprehensive program to assess competency in specific domains of clinical and translational research (CTR) for students enrolled in a 12-week introductory summer research program. Methods The program curriculum was mapped to core competencies developed by the National Center for Translational Science (NCATS) and used to develop R-OSCE stations. Twelve stations were developed, with five administered during orientation as a practice test and seven administered post-program. A scoring rubric using an anchored scale of 1-5 was developed and six qualified raters were trained in its use. The exam was self-paced and delivered through a secure online computer-based platform. Results Forty-seven students (three cohorts) completed the post-program R-OSCE. Most respondents scored at 3 (developing competence) or higher on most stations for both the practice and post-program exams, the exceptions being the stations involving writing research questions and engaging communities in research. Students indicated they liked demonstrating CTR skills through the R-OSCE. Most participants agreed that exam tasks were related to stated program competencies and that stations were realistic. Discussion The R-OSCE is best used as part of a comprehensive assessment program and may be useful in providing formative feedback to trainees that they can share with their mentors. Additionally, this study demonstrated that it could feasibly be used to evaluate the effectiveness of research education programs. However, additional time was needed to train raters and score the R-OSCE. Modifications were made to administer the exam through use of an online format with a modest budget. The computer-based format provides a solution to the current need for assessments that can be administered remotely.

The DIAMOND portal: An innovative platform for sharing translational research workforce training and assessment resources.

Although several initiatives have produced core competency domains for training the translational science workforce, training resources to help clinical research professionals advance these skills reside primarily within local departments or institutions. The Development, Implementation, and AssessMent of Novel Training in Domain (DIAMOND) project was designed to make this training more readily and publicly available. DIAMOND includes a digital portal to catalog publicly available educational resources and an ePortfolio to document professional development. DIAMOND is a nationally crowdsourced, federated, online catalog providing a platform for practitioners to find and share training and assessment materials. Contributors can share their own educational materials using a simple intake form that creates an electronic record; the portal enables users to browse or search this catalog of digital records and access the resources. Since September 2018, the portal has been visited more than 5,700 times and received over 280 contributions from professionals. The portal facilitates opportunities to connect and collaborate regarding future applications of these resources. Consequently, growing the collection and increasing numbers of both contributors and users remains a priority. Results from a small subset of users indicated over half accomplished their purpose for visiting the site, while qualitative results showed that users identified several benefits and helpful features of the ePortfolio.

Guidelines for Evaluating Clinical Research Training using Competency Assessments.

This article was migrated. The article was marked as recommended. Effective training programs in clinical and translational research (CTR) are critical to the development of the research workforce. The evolution of global CTR competencies frameworks motivates many CTR institutions to align their training offerings with these professional standards. Guidelines for integrating competency-based frameworks and assessments into rigorous program evaluations are needed in order to promote the quality and impact of these training programs. These guidelines provide practical suggestions for how to ensure that subjective and objective assessments of CTR knowledge and skill can be effectively integrated in the evaluations used to improve these essential training programs. The approach presented here necessarily involves the systematic and deliberate incorporation of these particular types of assessments into comprehensive evaluation plans. While these guidelines are broadly applicable to the work of those charged with developing, administering and evaluating CTR training programs, they have been specifically designed for use by program directors.

Best practices in social and behavioral research: report from the Enhancing Clinical Research Professional's Training and Qualifications project.

INTRODUCTION: This article discusses the process of defining competencies and development of a best practices training course for investigators and clinical research coordinators who conduct social and behavioral research. METHODS: The first project phase established recommendations for training in Good Clinical Practice (GCP) and was done in conjunction with representatives from 62 Clinical and Translational Science Award (CTSA) hubs. Diversity in behavioral clinical trials and differences in regulation of behavioral trials compared with clinical trials involving drugs, devices, or biologics necessitated a separate Social and Behavioral Work Group. This group worked with CTSA representatives to tailor competencies and fundamental GCP principles into best practices for social and behavioral research. RESULTS: Although concepts underlying GCP were deemed similar across all clinical trials, not all areas were equally applicable and the ways in which GCP would be enacted differ for behavioral trials. It was determined that suitable training in best practices for social and behavioral research was lacking. DISCUSSION: Based on the training need, an e-learning course for best practices is available to all CTSA sites. Each institution is able to track outcomes for its employees to help achieve standardized competency-based best practices for social and behavioral investigators and staff.

Piloting a nationally disseminated, interactive human subjects protection program for community partners: design, content, and evaluation.

Funders, institutions, and research organizations are increasingly recognizing the need for human subjects protections training programs for those engaged in academic research. Current programs tend to be online and directed toward an audience of academic researchers. Research teams now include many nonacademic members, such as community partners, who are less likely to respond to either the method or the content of current online trainings. A team at the CTSA-supported Michigan Institute for Clinical and Health Research at the University of Michigan developed a pilot human subjects protection training program for community partners that is both locally implemented and adaptable to local contexts, yet nationally consistent and deliverable from a central administrative source. Here, the developers and the analysts of this program discuss its development, its content, and the results of its evaluation.

Piloting a nationally disseminated, interactive human subjects protection program for community partners: unexpected lessons learned from the field.

Funders, institutions, and research organizations are increasingly recognizing the need for human subjects protections training programs for those engaged in academic research. Current programs tend to be online and directed toward an audience of academic researchers. Research teams now include many nonacademic members, such as community partners, who are less likely to respond to either the method or the content of current online trainings. A team at the CTSA-supported Michigan Institute for Clinical and Health Research at the University of Michigan developed a pilot human subjects protection training program for community partners that is both locally implemented and adaptable to local contexts, yet nationally consistent and deliverable from a central administrative source. Here, the developers of the program and the collaborators who participated in the pilot across the United States describe 10 important lessons learned that align with four major themes: The distribution of the program, the implementation of the program, the involvement of community engagement in the program, and finally lessons regarding the content of the program. These lessons are relevant to anyone who anticipates developing or improving a training program that is developed in a central location and intended for local implementation.

An Internet-based tailored hearing protection intervention for firefighters: development process and users' feedback.

BACKGROUND: Noise-induced hearing loss is a significant occupational injury for firefighters exposed to intermittent noise on the job. It is important to educate firefighters about using hearing protection devices whenever they are exposed to loud noise. Computer technology is a relatively new health education approach and can be useful for tailoring specific aspects of behavioral change training. The purpose of this study is to present the development process of an Internet-based tailored intervention program and to assess its efficacy. METHOD: The intervention programs were implemented for 372 firefighters (mean age = 44 years, Caucasian = 82%, male = 95%) in three states (California, Illinois, and Indiana). The efficacy was assessed from firefighters' feedback through an Internet-based survey. RESULTS: A multimedia Internet-based training program was developed through (a) determining program content and writing scripts, (b) developing decision-making algorithms for tailoring, (c) graphic design and audio and video productions, (d) creating computer software and a database, and (e) postproduction quality control and pilot testing. Participant feedback regarding the training has been very positive. Participants reported that they liked completing the training via computer (83%) and also that the Internet-based training program was well organized (97%), easy to use (97%), and effective (98%) and held their interest (79%). Almost all (95%) would recommend this Internet training program to other firefighters. CONCLUSION: Interactive multimedia computer technology using the Internet was a feasible mode of delivery for a hearing protection intervention among firefighters. Participants' favorable feedback strongly supports the continued utilization of this approach for designing and developing interventions to promote healthy behaviors.

From research to practice: the importance of community collaboration in the translation process.

In order for effective interventions to make an impact on their target population, they must be successfully translated and disseminated to the organizations that will ultimately deliver them to those in need. Cuídate!, a culturally based intervention to reduce HIV sexual risk among Latino youth, was identified by the Centers for Disease Control and Prevention's (CDC) Prevention Research Synthesis (PRS) project as "best evidence" of intervention efficacy and selected as part of the CDC's Replicating Effective Programs (REP). The REP process consisted of the design, development, and field-testing of the Cuídate! program package in community-based, nonacademic settings. Project staff worked with CDC and community-based partners throughout the REP process. Community partners included a community advisory board (CAB) and four case agencies. Case agency staff participated in a facilitator training and subsequently implemented the Cuídate! program at their respective agencies. Process evaluation findings showed that facilitators were able to effectively use program materials and implement the program with fidelity. Adolescent participants reported they liked the program and would recommend the project to others. Only slight modifications to program and training materials were necessary following evaluation. Lessons learned included the importance of interdisciplinary collaboration and utilizing the resources available from each collaborative partner.

Mexican adolescents' alcohol use, family intimacy, and parent-adolescent communication.

Despite widespread adolescent alcohol use, research on individual and contextual factors among Mexican adolescents is limited. This study describes the relationship between adolescent risk/protective factors, parent-adolescent communication, and their effects on alcohol use of 14- to 17-year-old adolescents living in Mexico (N = 829; 458 girls, 371 boys). In this study, adolescents reported that 55% ever used alcohol, 24% used alcohol in the past 30 days, and 10% reported binge drinking. Adolescents with high family intimacy were less likely to report ever using alcohol and binge drinking. Regression analysis revealed that parent-adolescent communication mediated the effect of family intimacy on overall and binge drinking. Alcohol use prevention with Mexican adolescents should focus on family intimacy and parent-adolescent communication.

Perception and attitudes of firefighters on noise exposure and hearing loss.

Noise exposure of firefighters results in increased risk of hearing loss. Noise-induced hearing loss (NIHL) can be prevented by the consistent use of hearing protection devices (HPDs). However, little is known about firefighters' perceptions and attitudes on NIHL and HPD use. The purposes of the study were (1) to identify common noise sources of fire fighting equipment and activities; and (2) to describe the attitudes and beliefs of firefighters concerning the importance of their hearing, occupational noise exposure, NIHL, and HPD use. A nonexperimental, descriptive focus group design was used to gather qualitative data from firefighters at two fire houses in southeast Michigan. The study identified sources of noise unique to fire fighting, such as driving fire/emergency trucks with sirens on; operating water pumps, saws, and other equipment; and shouting to communicate with civilians and fellow firefighters. Firefighters acknowledged the significance of good hearing in fire fighting service but perceived NIHL was unavoidable on the job and viewed it as a small risk compared with other hazards. HPDs were not used regularly because firefighters believed they interfered with the ability to hear commands during emergency conditions, interfered with other required safety equipment, and were generally forgotten when gearing up. Firefighters recognized the importance of hearing health and the benefit of a hearing conservation program, including annual training, and identified several topics that should be included in a training program. These findings will provide critical information in planning what type of program should be developed and carried out for the fire fighting population.

Recruitment and retention of Latino adolescents to a research study: lessons learned from a randomized clinical trial.

PURPOSE: To describe facilitators and barriers to participation and retention of Latino adolescents in a randomized clinical trial. DESIGN AND METHODS: Participants were part of a randomized clinical trial designed to reduce HIV sexual risk behavior among Latino youth. Responses from 106 randomly selected respondents from the 3-month follow-up were content analyzed. RESULTS: Four main facilitator patterns emerged: peer/family support, program incentives, commitment, and desire to help. Participation barriers included conflicts with other commitments, embarrassment, and lack of peer support. PRACTICE IMPLICATIONS: Recruitment and retention of Latino adolescents in research studies is critical to building a research base for nursing practice.

Selecting control interventions for clinical outcome studies.

In the current research environment the design and management of control groups is becoming more complex. The selection of a control group design is dependent on study goals, presence and quality of existing interventions, urgency of the problem or issue being addressed by the intervention, and factors related to the study site. The purpose of the presentation is to identify various approaches to the design of control groups in experimental studies and to identify their strengths, limitations, and applications. A case study exemplifies the issues associated with control group selection and design.

Effects of booster interventions on factory workers' use of hearing protection.

BACKGROUND: The provision of reinforcements or boosters to interventions is seen as a logical approach to enhancing or maintaining desired behavior. Empirical studies, however, have not confirmed the effectiveness of boosters nor assessed the optimum number of boosters or the timing for their delivery. OBJECTIVES: This randomized controlled trial contrasted the effect of four booster conditions (a). 30 days; (b). 90 days; (c). 30 and at 90 days; and (d). no boosters of the intervention to increase the use of hearing protection devices (HPDs). METHODS: A total of 1325 factory workers completed a computerized questionnaire and were randomly assigned to one of three computer-based (tailored, nontailored predictor-based, or control) multimedia interventions designed to increase the use of hearing protection devices. After the intervention, colorful boosters specific to the type of training received were mailed to workers' homes. Posttest measures of use were administered at the time of their next annual audiogram 6 to 18 months after the intervention. RESULTS Repeated measures of analysis of variance (ANOVA) showed a significant main effect for the booster (after 30 days) in the group that received tailored training (F[3442] = 2.722; p =.04). However, in the assessment of the interaction between time (pretest and posttest) and boosters (four groups), the ANOVA did not find significant differences in hearing protection device use for any of the training groups. To assess for significant differences between groups, post hoc comparisons were conducted at the pretest and posttest for the total sample and for the subsample of workers who reported using hearing protection devices less than 100% of the time needed. Sheffé contrasts by intervention group, gender, ethnicity, and hearing ability found no significant changes in the mean use of hearing protection devices for the booster groups. CONCLUSIONS: Although the provision of boosters represented a considerable commitment of resources, their use was not effective in this study. However, it would be premature to eliminate boosters of interventions. Further study is needed to explore the effects of different booster types for increasing the use of hearing protection devices, and to assess carefully the effects of boosters on other health behaviors in studies with controlled designs.

Effectiveness of a tailored intervention to increase factory workers' use of hearing protection.

BACKGROUND: In the United States it is estimated that more than 30 million workers are exposed to harmful levels of noise on the job. When engineering or administrative controls cannot be used to reduce noise, workers should always use hearing protection devices (HPDs) when exposed to loud noise to prevent noise-induced hearing loss (NIHL). Previous research has shown that workers do not always use HPDs when required; therefore, it is essential that workers assume personal responsibility for preventing NIHL by increasing their use of HPDs. OBJECTIVES: This study tested the effectiveness of an individually tailored multimedia intervention to increase use of HPDs by factory workers. METHODS: A randomized controlled design was used to compare the effects of a tailored intervention (n= 446) with two other interventions (a nontailored predictor-based intervention (n= 447) and a control intervention (n= 432)) on workers' self-reported use of HPDs 6 to 18 months following the intervention. RESULTS: Only those workers receiving the tailored intervention significantly increased their use of HPDs from pretest to posttest. However, this increase significantly differed from the nontailored group but not from the control group. CONCLUSIONS: Individually-tailored interventions offer promise for changing behavior. In light of the similarity between the results for the tailored intervention and the control intervention groups, further research is needed to understand barriers to HPD use and how to maximize the benefits of individually tailored interventions in this setting.