Evaluation and refinement of Self-Directed Learning Readiness Scale for medical students.

PURPOSE: This study evaluated the underlying subdomain structure of the Self-Directed Learning Readiness Scale (SDLRS) for medical students and refined the instrument to measure the subdomains to provide evidence for construct validity. Developing self-directed learners is a well-recognized goal amongst medical educators. The SDLRS has been frequently used, however, lack of construct validity makes it difficult to interpret results. METHODS: To identify the valid subdomains of the SDLRS, items were calibrated with the graded response model (GRM) and results were used to construct a 30-item short form. Short-form validity was evaluated by examining the correspondence between the total scores from the short form and the original instrument for individual students. RESULTS: A five-subdomain model explained the SDLRS item response data reasonably well. These included: (1) initiative and independence in learning, (2) self-concept as an effective learner, (3) openness to learning opportunity, (4) love of learning, and (5) acceptance for one's own learning. The unidimensional GRM for each subdomain fits the data better than multi-dimensional models. The total scores from the refined short form and the original form were correlated at 0.98 and the mean difference was 1.33, providing evidence for validation. Nearly 91% of 179 respondents were accurately classified within the low, average, and high readiness groups. CONCLUSION: Sufficient evidence was obtained for the validity and reliability of the refined 30-item short-form targeting five subdomains to measure medical students' readiness to engage in self-directed learning.

Supporting Self-Directed Learning: Development of a Faculty Evaluation Scale.

Construct: We sought to assess medical education faculty members' ability to support students in the development of self-regulated learning skills. Background: It is broadly agreed that medical students should become self-directed lifelong learners to succeed as physicians. To that end, many instruments have been developed that measure student attributes of self-directed learning (SDL). By contrast, no such analogous valid instrument is available to assess faculty familiarity in pedagogical strategies to promote SDL among students. Approach: An item bank with 45 items was created and its content-related validity evaluated by a panel of twenty experts. The items were selected and validated in the framework of Item Response Theory (IRT). The unidimensionality of all items within four constructs was assessed by using modified parallel analysis and 2 parameter IRT model calibration. The final version of the Recognition of Learning-Oriented Teaching Strategies (RoLOTS) instrument was validated by using the IRT marginal reliability as well as 2PL model calibrations. Different Item Functioning (DIF) in student-educators and gender was examined. Findings: The final version of RoLOTS included 20 items, with five in each of four subdomains: building content knowledge; the emotional and motivational aspects of learning; leveraging the social nature of the learning process; and metacognitive processes that promote student regulation of the learning process. Construct validity and reliability of each of the four domains were well supported by the results. Significant DIF was not detected in student-educator and gender. Conclusions: The RoLOTS successfully evaluated whether a faculty member is familiar with pedagogical tools to promote medical students' self-directed learning, which can be used as a first step in needs-based professional development. Further research is needed to provide more validity and reliability evidence among groups with diverse SDL experience.

Supporting Self-Directed Learning: A National Needs Analysis.

INTRODUCTION: It is broadly agreed that physicians must be self-directed learners, and many studies measuring medical student readiness and capacity for self-directed learning (SDL) have been reported. However, less attention has been paid to faculty tasked with guiding students to develop SDL skills. In an effort to determine whether faculty feel equipped to foster SDL, a needs assessment was developed. METHODS: The faculty needs assessment for fostering SDL (FNA-fSDL) was constructed and distributed nationally to medical educators. Faculty perceptions of the value of SDL, administration support, and faculty development were queried. Items also measured faculty recognition of cognitive, affective, and metacognitive strategies shown to cultivate SDL. Descriptive statistical analysis and frequency comparison tests to identify potential group differences related to work responsibility, classroom/clinical setting, and level of learner served were performed. RESULTS: Among the 359 respondents representing diverse areas of medical education, there was overwhelming agreement that physicians in training should be self-directed learners and that faculty require training in educational approaches promoting SDL. Faculty were more likely to recognize strategies supporting development of metacognitive skills and least likely to identify those related to cognitive aspects of learning. Fewer than half of faculty had received training in fostering SDL at their home institutions or at conferences. DISCUSSION: While there is strong support for the development of SDL among learners, these results suggest most faculty do not feel sufficiently trained to accomplish this goal. The FNA-fSDL can be used by institutions to assess their own faculty.

Development of Food Literacy Assessment Tool Targeting Adults With Low Income.

OBJECTIVE: Develop and test validity and reliability of the Food Literacy Assessment Tool (FLitT) in adults with low income. DESIGN: Face validity was tested using cognitive interviews, content validity using expert review, and internal consistency reliability and test-retest reliability based on 2 administrations of the survey. SETTING: Urban choice food pantry in Cincinnati, OH. PARTICIPANTS: There were 10 and 98 adults with low income for the cognitive interview and survey, respectively VARIABLES MEASURED: Knowledge, self-efficacy, and behavior required to plan and manage, select, prepare, and eat in relation to food. ANALYSIS: Cronbach α and Kuder-Richardson Formula 20 for internal consistency reliability and intraclass correlation coefficient (ICC) for test-retest reliability. RESULTS: Cognitive interviews and expert feedback suggested modifications to improve the clarity of FLitT and offer more response options. Testing shows acceptable internal consistency in self-efficacy (Cronbach α = 0.92) and behavior (Cronbach α = 0.90) but not in knowledge (Kuder-Richardson Formula 20 = 0.51). The FLitT shows acceptable test-retest reliability for knowledge (ICC = 0.84), self-efficacy (ICC = 0.70), and behavior (ICC = 0.93). CONCLUSIONS AND IMPLICATIONS: The FLitT was developed and tested for face and content validity and internal and test-retest reliability in adults with low income. Additional research is needed to conduct a second round of face validity and test construct validity using factor analysis with a larger size.

Interventions to improve medication adherence in adolescents with HIV: a systematic review and meta-analysis.

As of 2017, 1.8 million people living with HIV (PLWH) were adolescents between ages 10 and 19, accounting for 5% of all PLWH and 590,000 people between the ages 15 and 24 were newly infected with HIV. Between 2004 and 2011, AIDS-related deaths increased 50% among adolescents, and optimal adolescent adherence to antiretroviral treatment (ART) is estimated at only 62% of adolescents worldwide. While there have been great strides toward achieving the UNAIDS 90-90-90 goals, adolescents remain a group lacking appropriate resources and research to achieve these. This review analyzes current interventions aimed toward increasing adolescent ART adherence. Systematic searches of EMBASE, PubMed and PsycINFO were performed using the keywords 'adolescent HIV medication adherence interventions'. The Gain Score effect size was calculated for studies reporting the Cohen's d and variance to include both prestudy and poststudy values. A random-effects model analyzed intervention significance. Authors were contacted to obtain additional data values and study clarification. Twelve studies met inclusion criteria for meta-analysis. There were no significant differences seen between control and intervention groups in medication adherence (z=-1.4714, p<0.1412), viral load (z=-0.1946, p<0.8547) or CD4+ lymphocyte count (z=0.2650, p<0.7910). There was no significant difference between studies in increasing medication adherence. Results indicate that interventions did not improve medication adherence in adolescents with HIV. However, the paucity of quantitative research available speaks to a need for more quantitative intervention studies and standardization of measures of intervention efficacy.

How to Use Cognitively Diagnostic Assessments of Student Performance as a Method for Monitoring and Managing the Instructional Quality in Undergraduate Medical Education.

PURPOSE: Course evaluations by students are a standard tool that U.S. universities use to monitor the quality of their product. Here, the authors examine an alternative method of monitoring instructional quality that differs from traditional approaches in that it does not rely on students' ratings. The authors sought to glean relevant diagnostic information about course effectiveness from in-class exams used to assess students' learning progress (i.e., cognitively diagnostic assessments that explicitly target instructional content). METHOD: The authors used data from an end-of-course, cumulative exam given in 2015 and in 2016 to 200 first-year medical students. They mapped the exam questions to 4 attributes and analyzed the students' overall mastery of the content tested and the percentage of students mastering each attribute. RESULTS: Analyses of the cognitively diagnostic assessment data revealed the percentage of the cohort who achieved/failed to achieve mastery of each of the attributes, discreet mastery profiles that distinguish among students with similar scores, and the percentage of the cohort within each of the 16 attribute mastery profiles. Analysis allowed the authors to evaluate how well the course content was delivered. CONCLUSIONS: Cognitively diagnostic assessments enable in-class tests to appraise which skills specified in the curriculum have/have not been mastered by the students and how many students have mastered/failed to master which particular skills. Hence, if the learning goals have been well defined at the beginning of a course, then cognitively diagnostic assessments can show to what degree the instructional objectives have actually been accomplished.

Utilization of high-fidelity simulation to address challenges with the basic science immunology education of preclinical medical students.

BACKGROUND: Immune function and dysfunction are highly complex basic science concepts introduced in the preclinical medical school curriculum. A challenge for early learners is connecting the intricate details and concepts in immunology with clinical manifestations. This impedes relevance and applicability. The impetus in medical education reform is promoting consolidation of basic science and clinical medicine during the first two years of medical school. Simulation is an innovation now widely employed in medical schools to enhance clinical learning. Its use in basic science curriculums is largely deficient. The authors piloted simulation as a novel curricular approach to enhance fundamental immunology knowledge and clinical integration. METHODS: The authors introduced a Primary Immunodeficiency Disease (PIDD) simulation during a basic science immunology course for second-year medical students at the Zucker School of Medicine at Hofstra/Northwell. The simulation tasked small groups of students with evaluating, diagnosing and managing an infant with previously undiagnosed immunodeficiency. Joint facilitation by clinical and science faculty during terminal debriefings engaged students in Socratic discussion. Debriefing aimed to immerse basic science content in the context of the clinical case. Students completed a post-simulation Likert survey, assessing utility in reinforcing clinical reasoning, integration of basic science and clinical immunology, enhanced knowledge and understanding of immunodeficiency, and enhanced learning. A summative Immunodeficiency Objective Structured Clinical Examination (OSCE) question was created by faculty to assess students' recognition of a PIDD and clinical reasoning. RESULTS: The simulation was well received by students with > 90% endorsing each of the objectives on the post-simulation survey. The authors also determined a statistically significant score variance on the summative OSCE question. Higher scores were achieved by the cohort of students completing the OSCE post-simulation versus the cohort completing the OSCE pre-simulation. CONCLUSIONS: The innovative use of simulation in a highly complex basic science immunology course provides relevance and consolidation for preclinical learners. Additional data will be collected to continuously assess application of concepts and proficiency stemming from this novel curricular intervention. The authors advocate the initiation and/or expansion of simulation in non-clinical basic science courses such as immunology to bridge the gap between theory and practice.

Students' Perception of Formative Assessment as an Instructional Tool in Medical Education.

In self-directed learning environments, students must monitor and assess their learning progress themselves. Formative assessments play a central role as instructional tools to train students in these self-assessment skills. Key to the effectiveness of a formative assessment as a training instrument is the degree of student involvement in the design and implementation of the formative assessment used to train them. Five characteristics are essential to secure a sufficient level of student involvement: (1) congruence (i.e., the tasks of a formative assessment must reflect the instructional content), (2) authenticity (i.e., the tasks of a formative assessment should be related to students' background and study context), (3) consultation (i.e., students should have a say in how their answers are evaluated based on which criteria), (4) transparency (i.e., no "mystery items"-the wording of the items must clearly addresses the targeted content), and (5) accommodation (i.e., all students should have the same chance to complete the items of a formative assessment). These five characteristics of student involvement were used to evaluate the quality of formative assessments that are employed to train undergraduates in a medical school in self-assessment skills. Different from extant approaches that typically rely on instructors' judgments to infer the degree of student involvement, this study uses a paradigm in higher educational research asking students directly about their perceptions of an instructional tool. Thus, here, students had to rate their level of involvement in the design and implementation of the formative assessments used to train them. A total of 140 medical students participated in the study. Multivariate statistical methods were used to analyze the data. The majority of medical students appreciate the use of formative assessments to teach them self-directed learning skills; but the results also show that there is still room for increasing student involvement-especially, for minority students.

Assessment of medical students' leadership traits in a problem/case-based learning program.

BACKGROUND: Healthcare delivery is shifting to team-based care and physicians are increasingly relied upon to lead and participate in healthcare teams. Educational programs to foster the development of leadership qualities in medical students are needed to prepare future physicians for these roles. OBJECTIVE: Evaluate the development of leadership attributes in medical students during their first 2 years of medical school while participating in leadership training integrated into a problem/case-based learning program utilizing the Leadership Traits Questionnaire assessment tool. DESIGN: Ninety-eight students enrolled at Zucker School of Medicine participated in Patient-Centered Explorations in Active Reasoning, Learning and Synthesis (PEARLS), a hybrid problem/case-based learning program, during the first and second years of medical school. The Leadership Traits Questionnaire, designed to measure 14 distinct leadership traits, was utilized. It was administered to students, peers in students' PEARLS groups and their faculty facilitators. Participants completed questionnaires at three-time points during the study. Likert scale data obtained from the questionnaire was analyzed using a two-level Hierarchal Linear Model. RESULTS: Complete data sets were available for 84 students. Four traits, including self-assured, persistent, determined, and outgoing, significantly increased over time by measurements of both peer and facilitator-rated assessments. Six additional traits significantly increased over time by measurement of facilitator-rated assessment. By contrast, a majority of student self-rated assessments trended downward during the study. CONCLUSIONS: Medical students demonstrated development of several important leadership traits during the first 2 years of medical school. This was accomplished while participating in the PEARLS program and without the addition of curricular time. Future work will examine the impact of third year clerkships on leadership traits.

Modeling integration: co-teaching basic and clinical sciences medicine in the classroom.

PURPOSE: Calls for revision in undergraduate medical education frequently cite the importance of integrating basic and clinical sciences and the use of active pedagogies. One under-appreciated approach to accomplishing both is interactive co-teaching, defined as two instructors with complementary expertise engaging students and each other instead of lecturing. This study sought to determine if interactive co-teaching helped students integrate and learn basic and clinical sciences, as well as to explore potential advantages and barriers to co-teaching. METHODS: The comparative success of solo- and co-teaching in a microbiology/infectious disease course was determined by surveying student perceptions at the end of the course and examination scores for questions based on either solo- or co-taught content. The advantages and barriers to co-teaching were explored by thematic analysis of student responses to open-ended survey questions. RESULTS: Results suggest that co-teaching supported content integration as a significant majority of students (92%, n=112) reported they understood the connection between basic and clinical sciences better when content was co-taught. In addition, a plurality of students indicated that co-teaching provided a better overall learning experience (81%, n=99), was more engaging (74%, n=90), and made it easier to apply content (74%, n=90). These positive perceptions were reflected in better exam outcomes for materials covered in co-taught over solo-taught sessions. CONCLUSION: Results suggest students value co-teaching as a means to integrate basic and clinical sciences. However, interactive co-teaching pedagogies require careful planning and collaboration among faculty. Co-teaching requires the commitment of both faculty members to this pedagogy.

Nonparametric Calibration of Item-by-Attribute Matrix in Cognitive Diagnosis.

A nonparametric technique based on the Hamming distance is proposed in this research by recognizing that once the attribute vector is known, or correctly estimated with high probability, one can determine the item-by-attribute vectors for new items undergoing calibration. We consider the setting where Q is known for a large item bank, and the q-vectors of additional items are estimated. The method is studied in simulation under a wide variety of conditions, and is illustrated with the Tatsuoka fraction subtraction data. A consistency theorem is developed giving conditions under which nonparametric Q calibration can be expected to work.

Efficient Models for Cognitive Diagnosis With Continuous and Mixed-Type Latent Variables.

The issue of latent trait granularity in diagnostic models is considered, comparing and contrasting latent trait and latent class models used for diagnosis. Relationships between conjunctive cognitive diagnosis models (CDMs) with binary attributes and noncompensatory multidimensional item response models are explored, leading to a continuous generalization of the Noisy Input, Deterministic "And" Gate (NIDA) model. A model that combines continuous and discrete latent variables is proposed that includes a noncompensatory item response theory (IRT) term and a term following the discrete attribute Deterministic Input, Noisy "And" Gate (DINA) model in cognitive diagnosis. The Tatsuoka fraction subtraction data are analyzed with the proposed models as well as with the DINA model, and classification results are compared. The applicability of the continuous latent trait model and the combined IRT and CDM is discussed, and arguments are given for development of simple models for complex cognitive structures.