Assessing pharmacogenomic literacy in China through validation of the Chinese version of the Minnesota Assessment of Pharmacogenomic Literacy.

Pharmacogenomics (PGx) implementation into clinical care is rapidly increasing in China. However, the extent to which the public understands PGx testing and important knowledge domains requiring patient education or counseling remains unclear. To address this, we created and validated the Chinese version of the Minnesota Assessment of Pharmacogenomic Literacy (MAPL-CTM ). The MAPL-C was developed by translating the English MAPL to Chinese following cross-cultural translation guidelines. An online survey validated the MAPL-C and assessed Chinese individuals' PGx literacy. Validation analyses were performed and associations of PGx literacy with participants' characteristics were quantified. Of 959 high-quality responses, the majority of respondents were Han Chinese (96.3%), men (54.5%), aged 18-29 years (70.9%), residing in China (97.3%), and had received college or higher education (95.0%). Out of 15 starting items developed to query specific predefined knowledge domains, two uninformative items were excluded, resulting in a 13-item MAPL-C. Chinese participants' MAPL-C performance was best explained by a three-factor model, encompassing PGx concepts and function, testing limitations, and privacy. Higher MAPL-C performance was associated with younger age, higher education, and previous genetic testing experience. Correct response rates for questions related to testing limitations were lower than those in other domains. The creation and validation of the MAPL-C fills a gap in determining PGx knowledge among Chinese speakers, quantifying PGx literacy within a Chinese cohort, and identifying response patterns and knowledge gaps. The MAPL-C can be useful in clinical practice to guide patient counseling, assess PGx education interventions, and quantify PGx knowledge in relation to outcomes in research studies involving Chinese participants.

Medical Residents' Informal Learning from Pharmacists in the Clinical Workplace.

Workplace-based interactions between residents and pharmacists, though relatively underexplored, might contribute substantially to learning. This international study sought to investigate the affordances residents use for informal learning about medications, their interactions with pharmacists and patterns of resident-pharmacist engagement, as well as residents' perceived impact of these interactions on their learning. Contextual differences between US and Dutch residency training and electronic health record (EHR) may impact informal learning about medications. We conducted a cross-sectional, online, 25-item survey study, including closed-format and open-response questions among current resident physicians (post-graduate years 1-6, from a variety of residency programs n = 803) from the University of California San Francisco, the University of Minnesota, and the University Medical Center Utrecht. Responses from 173 residents in both countries revealed that these physician trainees were afforded opportunities to engage in a wide variety of pharmacotherapy-related activities but engaged differently with social and environmental resources for support. Residents from the United States (US) utilized pharmacists and Up-To-Date, whereas Dutch residents preferentially utilized the online Dutch medication information site and EHR-embedded medication resources. US residents interacted with pharmacists significantly more frequently than Dutch residents. Pharmacists provided residents with a wide range of useful information, much of which is integrated into the medication resources in the Dutch EHR-based decision-support system. While US residents reported overwhelmingly that informal interactions with pharmacists contribute to their learning about medications, Dutch residents' responses did not confirm this. Intentionally designing residents' training to include opportunities for interactions with pharmacists could potentially positively impact residents' informal workplace learning. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-023-01784-1.

Development and validation of a tool to assess the readiness of a clinical teaching site for interprofessional education (InSITE).

Interprofessional education within clinical teaching sites is a key part of training for pre-professional students. However, the necessary characteristics of these interprofessional clinical teaching sites is unclear. We developed a tool, the Interprofessional Education Site Readiness, or InSITE, tool, for individuals at a site to use as a self-assessment of the site's current readiness for providing interprofessional education. The tool progressed through six stages of development, collecting evidence for validity and reliability, resulting in a final tool with 23 questions distributed across five domains. Data from 94 respondents from a variety of national sites were used for the item analysis showing acceptable item-to-total correlations. Internal reliability testing gave a Cronbach's coefficient alpha of more than 0.70 for each group level comparison. Known groups validity testing provides strong evidence for its responsiveness in detecting differences in sites where IPE is implemented. The results of the testing lead us to conclude that the InSITE tool has acceptable psychometric properties. Additionally, we discovered that the process in which the InSITE tool was used demonstrated that it can facilitate learning in practice for the health professionals and can help make implicit, informal workplace learning and the hidden curriculum explicit.

Preliminary evidence for a Tool to Observe the Construction of Knowledge in Interprofessional teams (TOCK-IP).

Collaborative knowledge construction (KC) is an important process in interprofessional learning and a logical assessment target. A tool supporting the formative evaluation of KC behaviors ideally would be: 1) applicable to interprofessional teams of learners in clinical contexts; 2) informed by contemporary learning frameworks; 3) feasible and useful. No existing assessment tool meets these criteria. This paper describes the development and preliminary validity evidence for a Tool for Observing Construction of Knowledge in Interprofessional teams (TOCK-IP). Following literature review and needs assessment, the TOCK-IP was drafted based upon Gunawardena's five-phase KC model. Educational expert review established content validity. Response process and internal structure validity, feasibility, and utility were assessed through step-wise evaluation. Faculty raters applied the tool to four videos of simulated interactions between health professions learners. Faculty ratings were compared to expert consensus ratings. Thematic analysis of post-rating survey and debrief allowed assessment of feasibility and utility. Across videos, faculty raters' agreement was fair (n = 25; Fleiss' kappa = 0.40, <0.001). Excellent agreement (95%) was found for raters' scores compared to consensus rating. Faculty supported tool feasibility and utility. The TOCK-IP meets the three criteria for evaluating team-level KC and offers a progression roadmap to help learners move toward collaborative learning.

Undergraduate Student Perceptions of Pharmacy Entrustable Professional Activities.

Introduction: With declining applicant numbers, pharmacy education seeks to effectively communicate the value of the pharmacist on the health care team to attract learners to pharmacy. The American Association of Colleges of Pharmacy (AACP) published entrustable professional activities (EPAs) to outline the expected roles and responsibilities of new pharmacy graduates. However, it is unknown whether these statements resonate with the general public and could potentially attract learners to pharmacy. Description of Innovation: Students in PHAR1001: Orientation to Pharmacy, and online undergraduate course were surveyed using EPA statements. The survey was administered to all students before and after the completion of an undergraduate overview of pharmacy course. Participants were to respond "yes" or "no" to statements assessing the relevance of each EPA to pharmacy practice and the expectation of the activity to occur in all pharmacy practice settings. Findings: A total of 283 students participated in the pre-course survey, with 258 students completing the post-course survey. Pre-course, 11 of the 15 EPA statements had a high level of agreement (>80%) for relevance to practice. The high level of agreement continued in the post-course survey with five EPA statements exhibiting a significant increase in both the relevance and expectation of the role in pharmacy practice. Conclusions: Most students, regardless of prior knowledge of pharmacy, found the EPAs illustrative of pharmacist activities even prior to completing the pharmacy course. Entrustable professional activity statements may be a reasonable means to communicate the value of the pharmacist to the general public.

Development and Validation of the Minnesota Assessment of Pharmacogenomic Literacy (MAPL).

Ensuring that patients have an adequate understanding of pharmacogenomic (PGx) test results is a critical component of implementing precision medicine into clinical care. However, no PGx-specific validated literacy assessment has yet been developed. To address this need, we developed and validated the Minnesota Assessment of Pharmacogenomic Literacy (MAPLTM). Foundational work included a scoping review of patient and general public attitudes and experiences with pharmacogenomic testing, three focus groups, readability assessments, and review by experts and members of the general public. This resulted in a 15-item assessment designed to assess knowledge in four domains: underlying concepts, limitations, benefits, and privacy. For validation, 646 participants completed the MAPL as a part of a larger survey about pharmacogenomic research and statewide PGx implementation. Two items were deemed to be "too easy" and dropped. The remaining 13 items were retained in the final MAPL with good internal reliability (Cronbach's alpha = 0.75). Confirmatory factor analysis validated the four-domain construct of MAPL and suggested good model performance and high internal validity. The estimated coefficient loadings across 13 questions on the corresponding domains are all positive and statistically significant (p < 0.05). The MAPL covers multiple knowledge domains of specific relevance to PGx and is a useful tool for clinical and research settings where quantitative assessment of PGx literacy is of value.

A Scoping Review of Attitudes and Experiences with Pharmacogenomic Testing among Patients and the General Public: Implications for Patient Counseling.

The use of pharmacogenomic (PGx) tests is increasing, but there are not standard approaches to counseling patients on their implications or results. To inform approaches for patient counseling, we conducted a scoping review of published literature on patient experiences with PGx testing and performed a thematic analysis of qualitative and quantitative reports. A structured scoping review was conducted using Joanna Briggs Institute guidance. The search identified 37 articles (involving n = 6252 participants) published between 2010 and 2021 from a diverse range of populations and using a variety of study methodologies. Thematic analysis identified five themes (reasons for testing/perceived benefit, understanding of results, psychological response, impact of testing on patient/provider relationship, concerns about testing/perceived harm) and 22 subthemes. These results provide valuable context and potential areas of focus during patient counseling on PGx. Many of the knowledge gaps, misunderstandings, and concerns that participants identified could be mitigated by pre- and post-test counseling. More research is needed on patients' PGx literacy needs, along with the development of a standardized, open-source patient education curriculum and the development of validated PGx literacy assessment tools.

Addressing practical challenges with entrustable professional activities in ambulatory care experiential education.

BACKGROUND AND PURPOSE: Entrustable professional activities (EPAs) have been adopted and endorsed by the American Association of Colleges of Pharmacy as an assessment strategy. Application of EPAs in ambulatory care advanced pharmacy practice experiences (APPEs) has demonstrated potential internal grading consistency challenges and warrants further guidance. EDUCATIONAL ACTIVITY AND SETTING: Within a required ambulatory care APPE, a rubric was developed using purposeful and convenience sampling of preceptors, faculty, and students to concretely delineate expected EPA level progression from week one to week four based on setting specific activities. FINDINGS: The rubric was perceived to be accurate and useful as a tool for improving expectations of progression in the respective roles on ambulatory care APPEs. SUMMARY: Specifying performance of common clinical activities by weekly EPA level progression within an ambulatory care APPE was perceived to address some of the challenges in APPE grading consistency. Research for determining appropriate progression within an APPE and progression context within graduate and post-graduate training is needed. As more pharmacy training programs implement EPAs, collaboration in applying core EPAs into experiential education will be needed. This example of a setting specific progression rubric may serve as a useful tool for preceptors and students during their APPEs.

Beginning Early: Reflective Practice Development in a Pre-health Course on Health Literacy and Health Disparities.

Background: Learning reflective practices and understanding the complexity of health literacy and health disparities need to start early in health professions training. The primary objective of this inquiry was to evaluate the feasibility and effectiveness of using reflection categorization for assessing learner progression on reflective practice development. The secondary objective was to evaluate student reflection as a strategy for introducing and advancing pre-professional learners' understanding of health literacy and health disparities. Case Description: Within an online undergraduate health literacy course, two written reflection assignments were coded using Kember's four categories: habitual action, understanding, reflection, and critical reflection. Students received feedback based on this reflection categorization to promote development of reflective practices. However, reflections were not graded using the reflection categorization. Case Themes: Most (78%) students were at the level of understanding for the first reflection. For the second reflection, 29% of students were at the reflection level, demonstrating health literacy application and describing the important contributing role of personal context to health outcomes. Sixteen (33%) students progressed in their level of reflection. Within the reflections, students discussed knowledge gained and plans for future application. Conclusion: Using a structured reflection activity allowed pre-health students to begin developing reflection practices. Through reflection, students were able to describe and apply health literacy and health disparities knowledge.

Pharmacy Student Perceptions of the Entrustable Professional Activities.

Objective. To determine pharmacy students' perceptions and self-reported preparedness to perform the 15 core entrustable professional activities (EPA) established by the American Association of Colleges of Pharmacy for new pharmacy graduates. Methods. A random sample of Doctor of Pharmacy (PharmD) students from all four professional years at four universities were asked to indicate whether each of the 15 EPA statements was relevant to the practice of pharmacy and whether pharmacists were expected to perform the activity in multiple practice settings. Participants rated their self-perceived level of entrustability for each activity and indicated which three EPAs they felt most and least prepared to perform. Results. Four hundred twenty-three usable responses were received. The majority (≥85%) of students rated all of the EPA statements as relevant and ≥67% indicated that pharmacists were expected to perform them in multiple practice settings with a high percentage of agreement. Students' perceived need for supervision decreased from the P1 to P4 years. These data suggest that students' confidence to perform some activities grew as they gained experience and knowledge. However, in some cases, the self-perceived need for supervision regressed as students better understood the complexity of the activity. The EPA statements students felt most and least prepared to perform varied by year in the program. Conclusion. The core EPA statements were consistently rated by pharmacy students as relevant to pharmacy practice and as an expectation in multiple settings. Students perceived that they require less supervision when performing EPA activities as they progressed through the curriculum.

Partnering for a Sustainable Interprofessional Psychiatric Mental Health Nurse Practitioner Education Curriculum.

BACKGROUND: The World Health Organization recommends that health care educators create a collaborative and practice-ready workforce. Focused interprofessional education (IPE) promotes collaborative practice, yet few examples of how to develop sustained IPE and clinical partnerships exist. Mental health care professionals competent in their specialty and prepared for interprofessional collaboration are needed to treat complex mental health needs of patients. METHOD: Doctor of Nursing Practice Psychiatric Mental Health Nurse Practitioner (PMHNP) faculty partnered with College of Pharmacy faculty to create didactic, clinical, and simulation coursework and IPE competencies within PMHNP courses. Students developed skills about providing interprofessional mental health care. RESULTS: Recommendations for faculties include: embrace the value of interprofessional faculty partnerships; plan for time, money, motivation, and recognition needed for sustainable IPE; and design courses that become part of the fabric of the curricula. CONCLUSION: Embedding IPE into PMHNP curricula creates increased faculty satisfaction and positive feedback from students and clinical sites. [J Nurs Educ. 2019;58(12):723-727.].

Drug Therapy Problems in NCAA Division I-A Minnesota Gopher Student-Athletes.

As with all competitive sports, in NCAA Division I-A athletics it is important to maximize controllable factors in order for each athlete to compete at his or her best. One important and controllable factor that has the potential for improvement is the athlete's medication experience. When medications are used to treat the correct condition, administered at the right time, and with the correct dose, they have the potential to improve outcomes and enhance athletic performance. As an example, it is essential that a soccer player who has asthma is using the correct inhaler, at the correct time, with proper technique, and with the correct number of puffs in order to support breathing and improve oxygen transportation during a game. Ineffective and unsafe use of medications can lead to serious diseaserelated events and also prevent an athlete from achieving their performance goals when they are not being used correctly. The goal of this project was to explore the presence and extent of drug therapy problems (DTPs) among a college athlete population. Within the 10 student-athletes who were interviewed, there were a total of 36 drug therapy problems identified, many of which were tied to a lack of understanding for their own medication regimen. This project suggests that medication-related needs may not be appropriately addressed among the college athlete population. The addition of a pharmacist on the athletic healthcare team would ensure appropriate medication use and optimization for health outcomes and overall athletic performance.

Innovative Partnership between a Rural Mental Health Center and Community Pharmacy: Integration of a Mental Health Pharmacist.

PURPOSE: The purpose of this article is to describe how an innovative partnership between a rural community mental health center, community independent pharmacy and College of Pharmacy and integration of a mental health pharmacist lead to identification of medication therapy problems (MTP's) and interprofessional team partnerships with center mental health professionals. METHODS: A contractual arrangement was initiated between Northern Pines Mental Health Center (NPMHC), GuidePoint Pharmacy Services GPS) and the University of Minnesota College of Pharmacy (UMN CoP) to place a PGY1 resident at NPMHC. The resident was assigned to work closely with the Chief Medical Officer and provide initial comprehensive medication management (CMM) services to individuals who were enrolled in Assertive Community Treatment (ACT). A retrospective chart review was conducted to evaluate the impact of services provided. Patient inclusion criteria included ACT enrollees 18 years or older, a diagnosis of SPMI, taking at least one psychotropic medication, and participation in at least one resident-led CMM visit. Additional findings included the relationship between the pharmacist, the psychiatric physician, and other members of the ACT team. Descriptive statistics were used to document the findings. FINDINGS: N = 30 met the inclusion criteria: 18 males and 12 females, age ranged from 24 - 69 with average of 44 years old. 110 MTPs were identified ranging from no MTPs to 10 MTPs per patient, with a mean of 4 MTPs/patient. There was an uneven distribution of MTPs between psychiatric and medical conditions, with a disproportionately high occurrence of "Needs Additional Drug Therapy" in medical conditions and "Adverse Drug Reaction" in psychiatric conditions. In addition, the services were valued by members on the ACT team. CONCLUSION: Rural residents with SPMI in intensive community treatment have complex medication needs that require the training and skills of a clinical pharmacist. Despite the inclusion of a medication list as part of the ACT fidelity standards MTPs may go unrecognized and unresolved without the services of a clinical pharmacist conducting CMM. The pharmacist and psychiatric physician formed a collaborative partnership to address medication issues. We conclude that there is a need for integrating clinical pharmacist services into rural mental health centers.

Closing the Classroom Theory to Practice Gap by Simulating a Psychiatric Pharmacy Practice Experience.

Objective. To examine the feasibility and effectiveness of combining whole-task and guided reflection educational design principles with cloud-based learning technologies to simulate the clinical psychiatric advanced pharmacy practice experience (APPE) in the classroom to begin to close the theory to practice gap. Methods. Components of the typical student experience while completing an APPE were integrated into the course experience, ie, patient case work-ups, facilitated sessions with a preceptor, personal statement of goals and progress with feedback, and intentional interaction with peer-learners. Multiple sources of quantitative and qualitative data were collected and analyzed. Results. Twelve third-year pharmacy students from two campuses participated in and successfully completed this one-credit elective advanced psychiatric pharmacotherapy course. Eleven board-certified psychiatric pharmacists (BCPP) served as visiting experts, some participating for multiple weeks, and provided preceptor-like feedback to the case presentations in spring 2017. All BCPP pharmacists plus an additional geriatric pharmacist specialist participated in the course in spring 2018. Results of the quantitative and qualitative analyses demonstrated that students progressed in their readiness for APPEs and gained additional psychiatric pharmacy knowledge and evidence-based medicine decision making skills. Conclusion. Pharmacy programs are challenged to find additional ways to improve student readiness for APPEs and expand psychiatric learning opportunities to meet the increasing mental health needs across clinical settings. This example provides a feasible and effective strategy to do both without the requirement to create extensive new learning materials or add significant faculty workload.

Interprofessional Education Within a Pilot HIV-Care Elective Course.

Objective. To create an IPE course that improved knowledge related to HIV history, prevention, and therapy, in health professions students and improved their interest and confidence in becoming interprofessional collaborative clinicians, specifically involved in the care of people living with HIV. Methods. A motivational design framework was used to create an interprofessional course that incorporated whole-task complex scenarios, team-based application, and experiential components. Multiple sources of quantitative and qualitative data, including the AIDS Education and Training Center evaluation tool and Interprofessional Collaborative Competency Attainment Scale instrument, as well as assignments and course evaluations, were collected and analyzed. Results. Fifteen students from medicine, nursing, and pharmacy participated in 2017, and 21 students from medicine, nursing, pharmacy, and social work participated in 2018. In both offerings, students rated the course experience positively and self-reported increases in confidence related to interprofessional competencies. Ninety-three percent and 68% of the students in 2017 and 2018, respectively, stated they planned to be involved in HIV care to some degree in the future. Students demonstrated high levels of knowledge of the AIDS Training & Education Center National HIV Curriculum at the end of the 2018 course offering. Conclusion. This educational course design provided an effective interprofessional learning experience and establishes a sustainable interprofessional format for teaching health professions students about HIV.

Preceptor and resident perceptions of entrustable professional activities for postgraduate pharmacy training.

BACKGROUND: Following the development and implementation of entrustable professional activities (EPAs) as a collective graduation standard for pharmacy students, the University of Minnesota's multi-site post-graduate year one pharmacy residency program made a similar transition. EDUCATIONAL ACTIVITY AND SETTING: An electronic survey was distributed to program preceptors and residents to describe the perceptions of using EPAs to assess performance. FINDINGS: Residents (66.7%) and preceptors (78.3%) found the EPA framework to be more helpful than the traditional progression ratings model, but both groups also described EPAs as less beneficial for some objectives. Sixteen (69.6%) preceptors felt the residency graduation standard for attainment should be an entrustment level 4 or 5. Preceptor's found the EPA scale to be more descriptive, more objective, and easier to understand; however, it did not always apply to all skills and was more challenging to provide residents information regarding what they needed to improve upon. Residents commented that the EPA scale was more descriptive, had better clarity, and gave them a better idea of where their progression was at on a continuum. Residents also commented that regardless of the rating scale, specific, day-to-day feedback is more helpful. SUMMARY: Residents and preceptors perceive the EPA framework to be more helpful than traditional rating scales when evaluating learning objectives. Preceptors were less certain of what the EPAs entrustment level should be to consider a learning objective or skill achieved for residency. Both found some learning objectives to be more challenging for EPA use.

Personalized Learning in an Online Drugs and US Health Care System Controversies Course.

Objective. To examine if personalized learning objectives influenced student engagement and if achievement of objectives could be measured from course assignments. Methods. Learners created personalized learning objectives that correlated with their own goals within the context of the course. Using a mixed-methods analysis approach, the influence of these objectives on engagement and evidence of achievement of objectives were examined. Results. Students reported a positive influence of personalized learning objectives on engagement. Additionally, measurement of student progression or achievement of objectives was possible from analysis of the course assignments. Conclusion. Personalized learning is an important educational design for future pharmacists and health care professionals. Creating personalized learning objectives that build on centralized course objectives and connect to a broader context is one way to achieve the goal of an engaged and expanded learning experience.

Validation of the entrustable professional activities for new pharmacy graduates.

PURPOSE: The face validity of the core entrustable professional activities (EPAs) for new pharmacy graduates published by the American Association of Colleges of Pharmacy (AACP) in 2017 was evaluated. METHODS: A 28-item questionnaire was sent to experienced pharmacy practitioners affiliated with 4 schools of pharmacy. In addition to demographic information about education, training, credentials, and practice setting, participants were asked whether each EPA statement was pertinent to pharmacy practice and an expected activity that all pharmacists should be able to perform. Questions regarding the secondary attributes of the EPA statements examined whether each activity is observable, is measurable, is transferable to multiple practice settings, and integrates multiple competencies. RESULTS: The questionnaire was distributed to 137 eligible participants, and 71 usable survey responses were received. Participants consistently agreed (≥75% agreement) that the 15 EPA statements for new pharmacy graduates describe activities that are pertinent to pharmacy practice and that pharmacists are expected to perform. A consistent level of agreement was observed regardless of the preceptor's employment with a college or school, board certification status, or completion of postgraduate training, and no statistical differences in level of agreement were found based on these attributes. There was consistent agreement (≥60%) across geographic regions. No statistical differences in agreement were found between acute care practitioners and ambulatory care practitioners. CONCLUSION: A survey suggested that the core EPAs developed and vetted by AACP have face validity and are believed by experienced pharmacy preceptor-practitioners to be pertinent to pharmacy practice and to describe activities that all pharmacists should be able to competently perform.

An Interprofessional Education and Collaborative Practice Model for Dentistry and Pharmacy.

The aims of this study were to evaluate the feasibility of an interprofessional education and collaborative practice model (IECPM) developed by the School of Dentistry (SOD) and College of Pharmacy (COP) for the University of Minnesota dental clinics and to report results of the needs assessment using specific primary care metrics and medication histories gathered through use of the model in 2015-16. Planning focused on establishing a workflow to implement the IECPM by the SOD and COP. The interprofessional team that provided patient care for the study consisted of 50 dental students, ten dental therapy students, one pharmacy student, one pharmacy resident, one faculty pharmacist, one dental assistant, one faculty dental hygienist, and two dentists. The team selected 190 patients in the SOD clinic for the study based on the inclusion criteria: patients with two or more chronic medical conditions who were taking medications. The 190 patients received a comprehensive dental exam, review of social and medical history, and medication therapy management assessment by the interprofessional team. Specific core primary care metrics (blood pressure, pulse, tobacco use, and diabetes status) and identification of drug therapy problems (DTPs) were monitored and/or screened for during the dental visit. The results showed that the IECPM helped identify that this cohort of patients presented with chronic conditions: 64% had hypertension, 34% had diabetes, and 10.5% reported smoking cigarettes. Several DTPs were identified, of which "needs additional drug therapy" was the most common. This cohort was taking multiple medications (2-34 per patient) to address a variety of medical conditions. The study concluded that the IECPM with the SOD and COP helped address a primary care need that often goes unmet in dental clinics.

The impact of proctored online exams on the educational experience.

PURPOSE: This study explored new ways to maintain academic integrity for large enrollment, completely online courses. We examined the use of ProctorU as our proctoring strategy with the objectives to identify any implementation challenges and understand the impact of using an online proctor on the student experience. METHODS: In fall 2013, students were surveyed after each exam. Based on these preliminary findings, ProctorU-related questions were included in the course evaluation administered in spring 2014. A mixed-methods analysis plan was used to examine the results, including quantitative analysis and qualitative analysis of open-ended survey questions. RESULTS: In the fall 2013 surveys, 88.95% of students reported being satisfied with their experience using ProctorU. Of those who were unsatisfied, following three emerging themes were identified: took too long to setup, technical difficulties, and personnel issues with proctors. In the spring 2014 course evaluation data, the majority of students rated the experience "good" (57.53%), but a large number of those same students also commented on issues they encountered. Over half of the students indicated that the use of ProctorU would influence their future decision to take another online course, either negatively or positively. CONCLUSIONS: The question of how to maintain academic integrity with online courses is still an ongoing question, but this project demonstrates that online proctoring does influence the educational experience in ways that must be considered when determining the risk and benefit balance of proctored and unproctored assessments.