Assessment Structure, Culture, and Resources at US AAVMC-Accredited Colleges of Veterinary Medicine.

While outcomes assessment is commonplace in colleges of veterinary medicine, no information is published on how veterinary colleges resource, administer, and view assessment. Consequently, this article has the following objectives: (a) to determine the current level of resources (personnel, committees, software) allocated toward education assessment and program evaluation in colleges of veterinary medicine, (b) to characterize any common organizational structures within colleges of veterinary medicine for assessment, (c) to determine assessment personnel (faculty and staff) perceptions regarding whether existing assessment resources and structures are sufficient, and (d) to examine the perceived strength of the culture of assessment. Our survey found that most assessment professionals had been in their position for 4 years or less and over 50% did not have formal assessment training. A majority of respondents agreed that assessment was encouraged and supported at their institution, but there was much less agreement on items related to formal plans and structures. For example, only one quarter of respondents reported that assessment was connected to planning and budgeting, and only one third reported having a formal assessment plan. We hope that our survey will be a resource tracking the development of assessment resources and climate at American colleges of veterinary medicine.

Multi-institutional Analysis of Student and Program Variables as Predictors of Performance on the NAPLEX.

Objective. Understanding student characteristics and risk factors for performance on the North American Pharmacist Licensure Examination (NAPLEX) may help schools and colleges of pharmacy target limited resources. This study aimed to evaluate two sets of Doctor of Pharmacy (PharmD) student variables for their effect on NAPLEX performance.Methods. Data were collected from five schools and colleges of pharmacy over three years (2016-2018). Relationships between variables were first examined with bivariate correlations, and then by a series of multiple regressions using a structural equation modeling approach. Two sets of analyses were conducted using the outcome variable of NAPLEX scaled score. The first set examined variables at admission to the program while the second set examined performance variables during the PharmD program.Results. Students' score on the Pharmacy Curriculum Outcomes Assessment (PCOA) had the highest bivariate correlation with their NAPLEX score, with area 4 and area 2 having the strongest correlations. The correlations of Pharmacy College Admission Test (PCAT) subtest scores with NAPLEX scores were significant but smaller. Admissions variables of incoming science grade point average (GPA) and PCAT area scores were significant predictors of performance on the NAPLEX, but these variables were entered in separate models as they explained the same variance (each approximately 15%). The PCOA area scores and GPA before starting advanced pharmacy practice experiences (pre-APPE GPA) were significant predictors and explained about 25% of the variance in NAPLEX scores.Conclusion Pharmacy students' PCOA area scores and pre-APPE GPA were significant predictors of their NAPLEX scores. Information in this study can be used by pharmacy schools to efficiently target support services to students for enhanced NAPLEX performance.

Association Between NAPLEX Preparation Program Characteristics and First-Time Pass Rates.

Objective. The objectives of this study were to (1) describe characteristics of preparation programs for the North American Pharmacist Licensure Examination (NAPLEX) that are currently used by colleges and schools of pharmacy and (2) evaluate these program characteristics in relation to first-attempt NAPLEX pass rates.Methods. This cross-sectional study was based on an online survey administered between February and March 2020. Assessment leads from 143 PharmD programs were invited to answer questions on their schools' PharmD program characteristics and various aspects of NAPLEX preparation programs. The study included regression analyses to investigate associations between the NAPLEX first-attempt pass rates and PharmD demographic characteristics as well as between first-attempt pass rates and various aspects of the NAPLEX preparation programs. Finally, common themes from open-ended questions were identified.Results. Fifty-eight participants completed the survey out of 132 successfully delivered email invitations (response rate = 44%). Fifty participants (86%) indicated that their PharmD program offers a NAPLEX preparation program. Our data indicate that offering a NAPLEX preparation program was not significantly associated with higher NAPLEX first-attempt pass rates. The analysis identified possible explanations for this lack of association, including student concerns with balancing a preparation program with advanced pharmacy practice experiences (APPEs) and the faculty workload associated with delivering such programs.Conclusion. The current findings show no association between offering a NAPLEX preparation program and NAPLEX first-attempt pass rates. Future research should continue to examine the impact of these programs on individual school pass rates and factors that may enhance student motivation to engage in these programs.

Systematic Review of Predictors of Success for the North American Pharmacist Licensure Examination.

Objective. Studies have examined possible predictors of success on the North American Pharmacist Licensure Examination (NAPLEX). This systematic review investigated the literature on potential predictors of success on the NAPLEX.Findings. Articles were included in the review if they studied student characteristics and academic performance as independent variables and NAPLEX scores and/or pass rates as an outcome. Data were extracted from each article for students' demographics or variables, sample size, methods of statistical analyses, and results reporting correlation or predictability. From 40 articles retrieved from the initial search and sorting, 20 studies were included in the final review per inclusion criteria. Three studies included all the pharmacy programs, 15 were single-institution studies, two were multi-institution studies, and four had been published as posters. Among 30 different variables identified as potential predictors of success on the NAPLEX, the most examined variables were student age at matriculation, having a prior degree, Pharmacy College Admission Test (PCAT) scores, cumulative pharmacy school grade point average (GPA), overall Pharmacy Curriculum Outcomes Assessment (PCOA) scores, and PCOA content areas scores. Positively correlated factors included PCAT scores, not having a prior degree, prepharmacy and pharmacy school GPA, institutional characteristics, and PCOA scores. Negatively correlated factors included older age at admission.Summary. Cumulative pharmacy school GPA and PCOA scores were predictors of NAPLEX success consistently in the studies. The effects of preadmission student characteristics on NAPLEX success varied and were not consistently correlated or predictive.

Challenges to Implementation of the Co-Curriculum in Accredited Pharmacy Programs.

Objective. To determine areas of concern, and challenges to implementing and assessing the co-curriculum in accredited Doctor of Pharmacy programs, along with how confident programs are in their ability to meet the co-curriculum requirement as mandated by the Accreditation Council for Pharmacy Education (ACPE).Methods. A survey was administered to all ACPE-accredited pharmacy programs to collect information regarding areas of concern, challenges, and confidence in their ability to meet the co-curriculum requirement. The frequency of responses to items are presented along with comparisons based on characteristics, including institution type, cohort size, most recent ACPE accreditation review, and supporting offices.Results. The most common concerns centered on the documentation and assessment process. The most commonly reported challenges were lack of enthusiasm or buy-in from faculty, staff, and students; lack of a clear definition of co-curriculum; and faculty time and insufficient staff. Overall, programs had a high level of confidence in their ability to meet the requirements for co-curriculum. The only differences found were related to supporting offices and cohort size.Conclusion. The results suggest that having supporting offices may reduce the co-curriculum burden. Similarly, student cohort size may have an impact on the challenges for some programs, particularly those with moderate-sized cohorts reporting challenges related to faculty and staff. Further research is needed to determine how programs address these critical issues, and to explore whether programs report differently on these areas after completing an accreditation review. The study results may be useful to members of the Academy when evaluating co-curriculum.

Administration and Evaluation of the American Association of Colleges of Pharmacy Curriculum Quality Surveys in Pharmacy Schools.

Objective. To evaluate how pharmacy programs administer and evaluate American Association of Colleges of Pharmacy (AACP) curriculum quality perception surveys for continuous quality improvement, and to compare usage across the academy to the Principles of Good Use: AACP Quality Perception Surveys document.Methods. A 27-item survey instrument examining how schools used the curriculum quality survey was created and administered between March and June 2019 to assessment contacts of accredited schools and colleges of pharmacy. Descriptive statistics were performed for each survey item.Results. Of the 140 programs invited to participate, 88 (62.8%) responded. Curriculum quality survey data were triangulated with additional existing data (39.8%) or additional data sources were collected for triangulation with the survey data (54.5%). Programs reported on modifications made in the following areas: curriculum (85.2%), communication (75.0%), student services (68.2%), policy and process (61.4%), and professional development (53.4%). Most programs reported the assessment lead was responsible for oversight of the curriculum quality survey.Conclusion. Of respondents, 66% were familiar with the AACP Principles of Good Use document, and results indicate that institutions are generally following the recommendations. Survey analysis revealed that a significant number of programs are utilizing curriculum quality survey data for making meaningful programmatic improvements. Future work should center on further development of best practices for schools and colleges of pharmacy to effectively use the CQS data for continuous quality improvement.

Co-Curriculum Implementation and Assessment in Accredited Doctor of Pharmacy Programs.

Objective. To determine how accredited Doctor of Pharmacy programs implement and evaluate the co-curriculum requirement as mandated by the Accreditation Council for Pharmacy Education (ACPE). Methods. A survey was administered to all ACPE-accredited pharmacy programs to collect information regarding how co-curriculum models were being implemented, including types of activities, structure, learning outcomes, oversight, and assessment. The frequency of responses to items were presented to describe the general features of co-curriculum models. Results. The types of co-curricular activities reported by programs were generally consistent, with the majority of programs categorizing these activities and allowing students to choose which they would engage in. Most respondents reported that the program mapped co-curricular activities to learning outcomes, primarily ACPE Standards 1-4. The structural oversight of the co-curriculum typically included a co-curriculum committee, subcommittee, or task force, and supporting offices. The most common offices/departments involved in the co-curriculum were assessment, student affairs/services, experiential education, and academic/curricular affairs. The most common assessments were reflections, self-assessment surveys, and checklists. Conclusion. In most programs, implementation of the co-curriculum was a joint effort among various individuals, committees, and offices. Given the developing nature of programs, descriptive studies should be repeated to identify how programs develop and enhance co-curriculum models. The study results may be useful to members of the Academy when evaluating the current state of co-curriculum implementation and potential areas for program development.

National Trends in the Adoption of Pharmacy Curriculum Outcomes Assessment for Student Assessment and Remediation.

Objective. To determine and describe the current uses of the Pharmacy Curriculum Outcomes Assessment (PCOA) by US schools and colleges of pharmacy. Methods. Assessment professionals from 135 US schools and colleges of pharmacy were invited to complete a 38-item electronic survey. Survey items were designed to investigate common uses of the PCOA, cut points, and "stakes" assigned to the PCOA, identification of at-risk students, and remediation approaches. Results. The school response rate was 68%. The most common uses of the PCOA included curricular assessment (76%), individual student performance assessment (74%), and cohort performance assessment (71%). The PCOA was most frequently administered to third-year pharmacy (P3) students. The approach for assigning "stakes" to PCOA performance varied among programs depending on the student's professional year in the curriculum. Programs used a variety of approaches to establish the benchmark (or cut point) for PCOA performance. Remediation for at risk students was required by less than 25% of programs. Remediation was most commonly required for P3 students (22%). Conclusion. Survey results indicate wide variability between programs regarding PCOA cut points (benchmarks), stakes, and remediation approaches. In the future, it will be important for pharmacy educators to identify and study best practices for use of PCOA within student assessment and remediation plans.

Consideration of Aggressive and Strategic Approaches to Address Declining Enrollment in US Pharmacy Schools.

Members from Cohort 13 of the Academic Leadership Fellows Program (ALFP) 2016-2017 were challenged to present a debate on the topic: "In Turbulent Times, Pharmacy Education Leaders Must Take Aggressive Action to Prevent Further Declines in Enrollment" at the American Association of Colleges of Pharmacy INfluence 2017 meeting in Rio Grande, Puerto Rico. This paper is the result of thoughtful insights emerging from this debate. We present a discussion of the question of whether pharmacy education leaders must take aggressive action or strategic approaches to prevent further declines in enrollment. There are many thoughts regarding current declines in enrollment. Some educators contend that a more aggressive approach is needed while others argue that, while aggressive actions might lead to short-term gains, a more viable approach involves strategic actions targeting the underlying causes for decreasing enrollment. This paper explores themes of enrollment challenges, current and future workforce needs, and financial issues for both pharmacy programs and students. In summation, both aggressive actions and a strategic, sustainable approach are urgently needed to address declining enrollment.

Surveying the current landscape of assessment structures and resources in US schools and colleges of pharmacy.

INTRODUCTION: Expectations for assessment in higher education have increased in recent decades, prompting institutions to invest additional resources in this area. This study aimed to determine the resources, structure, and perception of assessment resources in United States schools and colleges of pharmacy (S/COPs). METHODS: Assessment personnel in S/COPs were surveyed electronically. Information collected included S/COP demographics, composition of assessment positions, experience and training of assessment personnel, and structure and responsibilities of committees engaged in assessment. Respondents' perception of their S/COPs having sufficient assessment personnel, recent changes in assessment, and the factors that prompted assessment changes were also surveyed. RESULTS: Respondents included individuals from 113 S/COPs (84% response rate). Most S/COPs had 1-2 assessment positions and 1-2 assessment-related committees. The most common assessment position titles were assistant/associate dean, director, coordinator/specialist, and administrative assistant. Dean-level administrators typically had worked in assessment the longest, whereas directors were more likely to have formal assessment training. Most respondents (75%) agreed they had sufficient assessment personnel to meet the 2007 Accreditation Council for Pharmacy Education Standards. Nearly two-thirds of respondents agreed they had sufficient personnel to meet the 2016 Standards and support their current assessment plan/process. Most S/COPs had a formal assessment committee (93%) and an average of two committees overseeing assessment. CONCLUSION: Most S/COPs reported having sufficient resources to support assessment activities. Although there were some consistent themes, there does not appear to be a single model for structuring assessment resources or committees. Effectiveness of various assessment structures represents an area for future research.

Factors associated with cultures of assessment at US schools and colleges of pharmacy.

INTRODUCTION: Pharmacy education has standards for assessment that must be met, prompting many programs to develop cultures of assessment. This study assesses the extent to which assessment personnel perceive that a culture of assessment has been established within their schools/colleges of pharmacy (S/COPs) and determines whether a relationship exists between assessment structure and the perceived strength of the culture of assessment. METHODS: An electronic survey was administered to assessment personnel at 134 S/COPs. Respondents were asked about changes within the past five years pertaining to assessment personnel/committees and factors prompting these changes. They were also asked to rate items related to support for institutional assessment (SIA scale) and faculty engagement with assessment (FEA scale). The reliability of each scale was estimated using Cronbach alpha and bivariate correlations were conducted to estimate the relationships among the two scales and other survey items. RESULTS: The S/COP response rate was 84% (113/134). Most (58%) noted the formation or re-structuring of the assessment office, with 26% creating one or more new assessment positions. The most common source of change was new accreditation standards (55%) followed by a new leadership position created in assessment (40%). Cronbach alpha for the SIA scale was 0.79 and 0.84 for the FEA scale. Program size was strongly negatively correlated with school age (rs = -0.58) and the FEA scale (r = -0.21). CONCLUSIONS: There was a strong relationship between SIA and FEA. Future studies are warranted to determine the interdependency of support for assessment and faculty engagement with assessment.