Development and Assessment of a Horizontally Integrated Biological Sciences Course Sequence for Pharmacy Education.

OBJECTIVE: To design and assess a horizontally integrated biological sciences course sequence and to determine its effectiveness in imparting the foundational science knowledge necessary to successfully progress through the pharmacy school curriculum and produce competent pharmacy school graduates. DESIGN: A 2-semester course sequence integrated principles from several basic science disciplines: biochemistry, molecular biology, cellular biology, anatomy, physiology, and pathophysiology. Each is a 5-credit course taught 5 days per week, with 50-minute class periods. ASSESSMENT: Achievement of outcomes was determined with course examinations, student lecture, and an annual skills mastery assessment. The North American Pharmacist Licensure Examination (NAPLEX) results were used as an indicator of competency to practice pharmacy. CONCLUSION: Students achieved course objectives and program level outcomes. The biological sciences integrated course sequence was successful in providing students with foundational basic science knowledge required to progress through the pharmacy program and to pass the NAPLEX. The percentage of the school's students who passed the NAPLEX was not statistically different from the national percentage.

A Methodology for Assessing Skill-Based Educational Outcomes in a Pharmacy Course.

Objective. To develop a methodology for assessing skill development in a course while providing objective evidence of success and actionable data to improve instructional effectiveness. Design. Course objectives were recast as skills to be demonstrated. Confidence in these skills was surveyed before and after the course. Student skills were demonstrated using 4 work products and a multiple-choice examination. Assessment. The change from precourse survey to postcourse survey was analyzed with a paired t test. Quality of the student work product was assessed using scoring guides. All students demonstrated skill mastery by scoring 70% or above on the work product, and 87/88 demonstrated individual progress on the surveyed skills during the 15-week course. Conclusion. This assessment strategy is based on sound design principles and provides robust multi-modal evidence of student achievement in skill development, which is not currently available using traditional student course evaluation surveys.

Reliability of a minimal competency score for an annual skills mastery assessment.

OBJECTIVE: To determine whether the modified Angoff process can be used to calculate a reliable minimal competency ("cut") score for the Annual Skills Mastery Assessment (ASMA). METHODS: Three panels of pharmacy faculty members used a modified Angoff method to create a minimal competency score for 60 previously used test items. The panels did not know which items were included. Data were analyzed to determine differences between rating sessions, faculty type, item difficulty, and rater scoring bias. RESULTS: The cut score generated was not significantly different by session or faculty type. The range of cut scores varied by less than 3% per examination. Faculty panelists correctly predicted student performance on items grouped as easy, medium, and hard. CONCLUSION: A properly constructed faculty panel can determine a reliable cut score and accurately rank relative test item difficulty using the modified Angoff process.

Prospective measurement of a problem-based learning course sequence.

OBJECTIVE: To measure the effect, over time, of a 2-year problem-based learning (PBL) sequence on the skills, knowledge, and abilities it was designed to develop and enhance. DESIGN: At the start of each PBL semester, students were provided a "work sample" case with a main medical issue not previously covered in the curriculum. A standardized form containing 6 sections (hypotheses, learning issues to investigate, how hypotheses ruled in/out, primary-problem identification, plan, and goals of plan) was completed for each case. To rate student performance, investigators used a standardized form with 5-point Likert scale. ASSESSMENT: Sixty-seven students who completed 4 assessments were included in data analyses. Scores significantly improved for each semester compared with baseline. Minimal significant differences were observed among semesters 2, 3, and 4. CONCLUSION: The 2-year PBL sequence improved students' performance compared with baseline, but the performance ceiling observed in our study requires further investigation.

Development of a reliable, valid annual skills mastery assessment examination.

OBJECTIVE: To develop a methodology for a reliable, valid annual skills mastery assessment examination to provide formative student feedback, inform curricular review, and comply with the Accreditation Council for Pharmacy Education (ACPE) Standards 2007. DESIGN: A sample of program-level ability-based outcomes skills were chosen for the examination. Test items were written, underwent quality control, and were scored for level of difficulty. Versions of the examination for first-, second-, third-, and fourth-year pharmacy students were developed and administered, the results were analyzed, reliability and validity were evaluated, and reports were generated. Item-writing guidelines, quality control procedures, and examination production steps were codified to create a criterion-referenced examination. Students and faculty advisors received detailed score reports and results were used to guide student performance and stimulate a review of curricular outcomes. ASSESSMENT: Content, criterion, and construct validity were analyzed as defined in the literature for the intended use of this assessment tool. Data suggest the Annual Skills Mastery Assessment (ASMA) examination is both reliable and valid. Students and faculty members were surveyed regarding the usefulness of the examination. Results indicate general satisfaction with the assessment program. CONCLUSION: A reasonably reliable, reasonably valid multiple-choice annual skills mastery assessment for selected outcomes statements providing formative feedback and informed curricular review was developed.