A participatory approach to evaluating a national training and institutional change initiative: the BUILD longitudinal evaluation.

BACKGROUND AND PURPOSE: The National Institutes of Health (NIH) funds training programs to increase the numbers and skills of scientists who obtain NIH research grants, but few programs have been rigorously evaluated. The sizeable recent NIH investment in developing programs to increase the diversity of the NIH-funded workforce, implemented through the Diversity Program Consortium (DPC), is unusual in that it also funds a Consortium-wide evaluation plan, which spans the activities of the 10 BUilding Infrastructure Leading to Diversity (BUILD) awardees and the National Research Mentoring Network (NRMN). The purpose of this article is to describe the evaluation design and innovations of the BUILD Program on students, faculty, and institutions of the 10 primarily undergraduate BUILD sites. KEY HIGHLIGHTS OF THE PROJECT: Our approach to this multi-methods quasi-experimental longitudinal evaluation emphasizes stakeholder participation and collaboration. The evaluation plan specifies the major evaluation questions and key short- to long-term outcome measures (or Hallmarks of Success). The Coordination and Evaluation Center (CEC) embarked on a comprehensive evaluation strategy by developing a set of logic models that incorporate the Hallmarks of Success and other outcomes that were collaboratively identified by the DPC. Data were collected from each BUILD site through national surveys from the Higher Education Research Institute at UCLA (HERI), annual followup surveys that align with the HERI instruments, site visits and case studies, program encounter data ("tracker" data), and institutional data. The analytic approach involves comparing changes in Hallmarks (key outcomes) within institutions for biomedical students who participated versus those who did not participate in the BUILD program at each institution, as well as between institution patterns of biomedical students at the BUILD sites, and matched institutions that were not BUILD grantees. Case studies provide insights into the institutionalization of these new programs and help to explain the processes that lead to the observed outcomes. IMPLICATIONS: Ultimately, the results of the consortium-wide evaluation will be used to inform national policy in higher education and will provide relevant examples of institutional and educational programmatic changes required to diversify the biomedical workforce in the USA.

Neighborhood socio-economic disadvantage and race/ethnicity as predictors of breast cancer stage at diagnosis.

BACKGROUND: This study investigated the role of key individual- and community-level determinants to explore persisting racial/ethnic disparities in breast cancer stage at diagnosis in California during 1990 and 2000. METHODS: We examined socio-demographic determinants and changes in breast cancer stage at diagnosis in California during 1990 and 2000. In situ, local, regional, and distant diagnoses were examined by individual (age, race/ethnicity, and marital status) and community (income and education by zip code) characteristics. Community variables were constructed using the California Cancer Registry 1990-2000 and the 1990 and 2000 U.S. Census. RESULTS: From 1990 to 2000, there was an overall increase in the percent of in situ diagnoses and a significant decrease in regional and distant diagnoses. Among white and Asian/Pacific Islander women, a significant percent increase was observed for in situ diagnoses, and significant decreases in regional and distant diagnoses. Black women had a significant decrease in distant -stage diagnoses, and Hispanic women showed no significant changes in any diagnosis during this time period. The percent increase of in situ cases diagnosed between 1990 and 2000 was observed even among zip codes with low income and education levels. We also found a significant percent decrease in distant cases for the quartiles with the most poverty and least education. CONCLUSIONS: Hispanic women showed the least improvement in breast cancer stage at diagnosis from 1990 to 2000. Breast cancer screening and education programs that target under-served communities, such as the rapidly growing Hispanic population, are needed in California.

Effects of laboratory data exchange in the care of patients with HIV.

BACKGROUND: Electronic health record (EHR) systems are often modified through the addition of new features over time. Few studies have examined the specific effects of such changes. We examined whether implementation of a bidirectional laboratory interface for order entry and data reporting within an existing ambulatory EHR would result in more prompt responses to laboratory indications for antiretroviral therapy (ART) changes or in improved communication with HIV+ patients about relevant laboratory results. METHODS: We conducted a single-arm intervention study comparing the timeliness of ART regimen changes, HIV viral load (VL) outcomes and patient-reported assessments of care before and after implementation of a laboratory data exchange interface within an existing EHR, without changing the EHR ordering or results reporting user interface. Patient data was extracted from the EHR covering the period from 1 year before to 2 years after the intervention for a cohort of 1181 patients who had received care during the baseline year. The timeliness of ART changes was represented by the days from a laboratory-result "signal" (CD4 dropping below 350 or 200 or VL increasing by a half-log or to a value over 100,000) to an ART-change "response". Patient assessments of care were collected by interviewing 100 anonymous patients at baseline and another 125 at 2 years post-intervention. RESULTS: A total of 171 laboratory "signal" events were followed within 80 days by a change in ART therapy. The mean time from signal to therapy change (adjusted for clustering by patient) initially increased, from 37.7 days during the pre-intervention year to 48.2 days during the quarter immediately following activation of the lab intervention. It then declined to a mean of 31.4 days over the remaining 21 months of observation (P=0.03 for the 6-day improvement from the pre-period). A majority of patients (65%) achieved undetectable VL values by the end of the observation period; faster signal-response times were not associated with greater achievement of undetectable VL. Patients rated communication about laboratory tests more highly after implementation of the interface (91 vs. 83 on a 100-point scale, P=0.01); ratings were not higher for other aspects of care. CONCLUSIONS: Adding laboratory data exchange interfaces within existing EHRs holds promise for improving HIV care, both in the timeliness of responses to important laboratory results and in the quality of provider communication about lab tests. However, the benefits from this incremental change may be modest unless more extensive redesign of laboratory follow-up workflows is undertaken, with support from enhanced user interfaces that take advantage of the laboratory information delivered. Providers should also consider increased staffing to compensate for dips in follow-up performance during the initial post-implementation months.

Comparison of extramural clinical rotation days: did the Pipeline program make a difference?

Extramural clinical rotations are implemented by dental schools for a combination of clinical and didactic or behavioral goals. In the United States, the Pipeline, Profession, and Practice: Community-Based Dental Education program was launched to increase the number of underrepresented minority students who are recruited and retained in dental education, to expand the dental curriculum in cultural competence, and to incorporate community-based extramural rotations into the dental schools' clinical curriculum. The objective of this study was to conduct an impact analysis regarding the change in number of extramural clinical rotation weeks for Pipeline and non-Pipeline program students over the time period of 2003 to 2007. National data from the American Dental Education Association's senior survey and other secondary sources were used to determine what student, school, and community characteristics are associated with a difference in the student report of the number of required weeks they expect to spend during their last year in dental school providing care at extramural clinic settings. Students reported a mean of 7.2 weeks for Pipeline students and 6.4 weeks for non-Pipeline students in 2003, increasing to 8.2 weeks for Pipeline students and 6.6 weeks for non-Pipeline students (p<0.05) in 2007. The multivariable model showed the Pipeline program increased significantly the number of rotational weeks reported by students. Three other variables significantly increased rotation weeks: 1) a lower baseline number of reported weeks in community rotations; 2) a lower level of debt student reported upon graduation; and 3) student reports of a higher orientation toward service to others as a reason to enter dentistry.

Who will serve? Assessing recruitment of underrepresented minority and low-income dental students to increase access to dental care.

This article uses data from the 2007 American Dental Education Association survey of dental school seniors to assess their intentions to serve underserved populations according to the students' underrepresented minority (URM) and income status. Dental school recruitment and retention programs that concentrate exclusively on URM students will not benefit most low-income students since 83 percent of them are not URM. Recruiting URM students leads to more graduating students with intentions to serve minorities. Whether the income of URM students was high or low, about half in each income group stated that more than 25 percent of their patients would be underserved minorities, compared to 28 percent of the low-income non-URM students and 17 percent of the higher income non-URM students. However, our multivariable results suggest that recruitment of both low-income groups (URM and non-URM) rather than high income regardless of ethnicity might be especially helpful in producing graduates who choose public service. URM/income status was not significantly related to serving special care or rural populations.

Associations of eye diseases and symptoms with self-reported physical and mental health.

PURPOSE: To study the associations of eye diseases and visual symptoms with the most widely used health-related quality-of-life (HRQOL) generic profile measure. DESIGN: HRQOL was assessed using the short form-36 (SF-36) version 1 survey administered to a sample of patients receiving care provided by a physician group practice association. METHODS: Eye diseases, ocular symptoms, and general health were assessed in a sample of patients from 48 physician groups. A total of 18,480 surveys were mailed out and 7,093 returned; 5,021 of these had complete data. Multiple linear regression models were used to examine the decrements in self-reported physical and mental health associated with eye diseases and symptoms, including trouble seeing and blurred vision. RESULTS: Nine percent of the respondents had cataracts, 2% had age-related macular degeneration, 2% glaucoma, 8% blurred vision, and 13% trouble seeing. Trouble seeing and blurred vision both had statistically unique associations with worse scores on the SF-36 mental health summary score. Only trouble seeing had a significant association with the SF-36 physical health summary score. While these ocular symptoms were significantly associated with SF-36 scores, having an eye disease (cataracts, glaucoma, and macular degeneration) was not, after adjusting for other variables in the model. CONCLUSIONS: Our results suggest an important link between visual symptoms and general HRQOL. The study extends the findings of prior research to show that both trouble seeing and blurred vision have independent, measurable associations with HRQOL, while the presence of specific eye diseases may not.

Reforming dental education: faculty members' perceptions on the continuation of pipeline program changes.

In 2002-03 the Robert Wood Johnson Foundation (RWJF) established the Pipeline, Profession, and Practice: Community-Based Dental Education program to change dental education in the United States. In partnership with The California Endowment, the RWJF awarded grants to fifteen U.S. dental schools that would support them in efforts to recruit more underrepresented minority/low-income (URM/LI) students, add cultural competence training, and increase extramural rotations to sixty days. As the program evaluator, the University of California, Los Angeles, School of Public Health (UCLA-SPH) conducted a survey of dental faculty in 2006 related to the goals of the Pipeline program. In this article, we report faculty perceptions pertaining to the extramural rotations and URM/LI recruitment. The survey was conducted in fourteen U.S. dental schools that received Pipeline grants and had an overall response rate of 60 percent (n=1,027) from the 1,713 faculty members who received the survey. A majority (57 percent) of faculty members strongly agreed that extramural rotations should continue as an integral part of students' education; 51 percent felt the same about the continuation of URM/LI recruitment programs. Multivariate logistic regression analyses revealed that faculty type, perception of extramural rotations being a positive experience, increased student productivity, and school culture were significant determinants of support for continuation of the extramural rotation programs. Determinants of support for continuation of the URM/LI recruitment programs were faculty type, perception of URM/LI recruitment effectiveness, perception of students from diverse backgrounds improving educational experience, and having a school mission statement that supports URM/LI recruitment. Pipeline schools should ensure that their extramural faculty remain key players in the Pipeline programs, widely publicize the programs' successes, and develop a service-oriented culture in order to build and sustain faculty perceptions that these programs should continue as integral parts of the schools' educational mission.

What enhances underrepresented minority recruitment to dental schools?

This study examined the factors influencing the proportion of underrepresented minority students (URM) in dental schools. Using a comprehensive recruitment model, it considered the relative importance of community characteristics (population demographics, oral health policies, dental care system, and university environment), dental school characteristics (Pipeline-supported, mission, and financing), and community-based dental education (CBDE) characteristics of the dental school on recruitment of URM students. Data come from a national survey of dental school seniors and a variety of publicly available sources. Three outcome variables measure URM recruitment: percent URM, percent Hispanic, and percent African American in the first year of dental school. Multivariable results revealed that the most important factors predicting a higher percent URM in first-year classes were a higher proportion of URM clinical faculty and graduating students' perceptions that their clinical rotation experience improved their ability to care for diverse groups. For percent Hispanic in the first year, a higher proportion of URM clinical faculty and students spending more time in clinical rotations predicted greater Hispanic recruitment. Graduating students' perceptions that they were less prepared to treat diverse groups were directly associated with the proportion of Hispanic students in the class. For a higher percent of African Americans in the first-year class, the most important factors were a higher proportion of blacks in the county, support from the national Pipeline program, and graduating students' perceptions of better preparedness to integrate cultural differences into treatment planning. Higher total financial aid awarded by the school was negatively associated with recruitment of African Americans. Results suggest some improved URM recruitment strategies for dental schools.

Effect of school environment on dental students' perceptions of cultural competency curricula and preparedness to care for diverse populations.

Reports of oral health disparities among racial, ethnic, and socioeconomic sectors of the U.S. population have hastened development of strategies to address this issue. Among these strategies is revising dental school curricula in order to develop more culturally competent graduates. The present study uses data from the 2003 American Dental Education Association (ADEA) survey of dental school seniors to assess students' perceptions of the adequacy of their cultural competency training. We hypothesize that these perceptions are influenced by multiple student characteristics and contextual factors, including a school's status with respect to the Pipeline, Professions, and Practice initiative of the Robert Wood Johnson Foundation and The California Endowment. Response data from ADEA survey items reflecting student perceptions of adequacy of curriculum time devoted to cultural competency and their preparedness to treat an ethnically and culturally diverse population were analyzed to assess the influence of selected student and contextual factors. Student gender, marital status, and level of educational debt were found to influence the curriculum time variable, and students at California schools reported higher perceived preparedness levels than students at dental schools nationwide. Dental school environments promoting acceptance and respect of diverse ethnicities/cultures and student race/ethnicity were significantly associated with students' perception of the adequacy of curriculum time for cultural competency and students' perception of their preparedness to provide oral health care for racially and culturally diverse groups. The findings provide insight for development of cultural competency curricula and direction for future study in this area.

What influences dental students to serve special care patients?

This study examines factors associated with graduating dental students' motivation to deliver services to special care patients. We investigated community context and student characteristics, which would influence potential behavior. Higher percentages of older adults and low-income residents in the community were positively correlated with interest in serving special care populations. Factors which correlated with individual student characteristics included having a father with at least a college education, a higher number of weeks spent in extramural clinical rotations, preparedness to provide care to disabled patients, and service orientation and socially conscious attitudes. Frail elderly and disabled persons have limited access to dental care, which is compounded by a shortage of skilled dental professionals who are willing to treat these populations. Our findings suggest that interest in special care dentistry is partly conditioned by the dental school's demographic and dental market context. This study is important to dental educators and policymakers because the challenge of providing care to the "special patient" will increase in the future.

Influence of contextual environment and community-based dental education on practice plans of graduating seniors.

This study investigated senior dental students' plans to provide care to underserved racial/ethnic minority populations. Three sets of determinants were analyzed: contextual environment, community-based dental education (CBDE), and student characteristics. We analyzed data from the ADEA Survey of Dental School Seniors and administrative data sources to construct contextual variables. Multivariable results show three contextual variables predicted practice plans: greater numbers of federally qualified health centers, higher percentages of underrepresented minorities, and attending a California Pipeline dental school. Regarding CBDE predictors, it was alarming to find seniors who viewed the cultural competency curriculum as inadequate and perceived themselves as less prepared to provide oral health care to diverse populations were also those most likely to serve minority patients. Significant student characteristics included racial/ethnic minority, female gender, older age, lower parent's income, and socially conscious orientation. The study provides evidence that contextual environment, CBDE, and student characteristics were significantly associated with plans to care for underserved patients. Findings suggest if the Pipeline initiative is successful in stimulating reform in U.S. dental schools, future students will develop greater awareness regarding critical access problems and the competencies required to effectively care for diverse populations. In the long term, addressing the problem of dental care access will require the creation of policy, financial, and structural interventions to motivate providers to care for the underserved.