A narrative review of adaptive testing and its application to medical education.

Adaptive testing has a long but largely unrecognized history. The advent of computer-based testing has created new opportunities to incorporate adaptive testing into conventional programmes of study. Relatively recently software has been developed that can automate the delivery of summative assessments that adapt by difficulty or content. Both types of adaptive testing require a large item bank that has been suitably quality assured. Adaptive testing by difficulty enables more reliable evaluation of individual candidate performance, although at the expense of transparency in decision making, and requiring unidirectional navigation. Adaptive testing by content enables reduction in compensation and targeted individual support to enable assurance of performance in all the required outcomes, although at the expense of discovery learning. With both types of adaptive testing, candidates are presented a different set of items to each other, and there is the potential for that to be perceived as unfair. However, when candidates of different abilities receive the same items, they may receive too many they can answer with ease, or too many that are too difficult to answer. Both situations may be considered unfair as neither provides the opportunity to demonstrate what they know. Adapting by difficulty addresses this. Similarly, when everyone is presented with the same items, but answer different items incorrectly, not providing individualized support and opportunity to demonstrate performance in all the required outcomes by revisiting content previously answered incorrectly could also be considered unfair; a point addressed when adapting by content. We review the educational rationale behind the evolution of adaptive testing and consider its inherent strengths and limitations. We explore the continuous pursuit of improvement of examination methodology and how software can facilitate personalized assessment. We highlight how this can serve as a catalyst for learning and refinement of curricula; fostering engagement of learner and educator alike.

Timing is key to providing modified assessments for students with specific learning difficulties.

INTRODUCTION: Medical students who are diagnosed with a specific learning difficulty (SpLD) will typically receive a reasonable adjustment within examinations in the form of modified assessment provision (MAP). This study investigated whether the timing of SpLD diagnosis and subsequent implementation of MAP has an impact on performance in applied medical knowledge multiple choice question (MCQ) assessments. METHOD: The MCQ performance of 108 students diagnosed with SpLD who received a MAP was monitored and compared with 1960 students who received an unmodified assessment, over 5 years of a medical program. Students who received a SpLD diagnosis in the latter years of the program were identified as not receiving a MAP in assessments prior to diagnosis. RESULTS: Differences were found between declaration and diagnosis, with 44.4% of students who declared and 48.1% who did not declare subsequently receiving a diagnosis. Students with SpLD who receive a MAP increase their applied medical knowledge assessment performance, although there is a delay of up to a year for this impact to reach significance. CONCLUSION: Early diagnosis of SpLD is necessary to ensure the intended benefit is received from MAP.

Analysis of question text properties for equality monitoring.

INTRODUCTION: Ongoing monitoring of cohort demographic variation is an essential part of quality assurance in medical education assessments, yet the methods employed to explore possible underlying causes of demographic variation in performance are limited. Focussing on properties of the vignette text in single-best-answer multiple-choice questions (MCQs), we explore here the viability of conducting analyses of text properties and their relationship to candidate performance. We suggest that such analyses could become routine parts of assessment evaluation and provide an additional, equality-based measure of an assessment's quality and fairness. METHODS: We describe how a corpus of vignettes can be compiled, followed by examples of using Microsoft Word's native readability statistics calculator and the koRpus text analysis package for the R statistical analysis environment for estimating the following properties of the question text: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (Grade), word count, sentence count, and average words per sentence (WpS). We then provide examples of how these properties can be combined with equality and diversity variables, and the process automated to provide ongoing monitoring. CONCLUSIONS: Given the monitoring of demographic differences in assessment for assurance of equality, the ability to easily include textual analysis of question vignettes provides a useful tool for exploring possible causes of demographic variations in performance where they occur. It also provides another means of evaluating assessment quality and fairness with respect to demographic characteristics. Microsoft Word provides data comparable to the specialized koRpus package, suggesting routine use of word processing software for writing items and assessing their properties is viable with minimal burden, but that automation for ongoing monitoring also provides an additional means of standardizing MCQ assessment items, and eliminating or controlling textual variables as a possible contributor to differential attainment between subgroups.

Modifying Hofstee standard setting for assessments that vary in difficulty, and to determine boundaries for different levels of achievement.

BACKGROUND: Fixed mark grade boundaries for non-linear assessment scales fail to account for variations in assessment difficulty. Where assessment difficulty varies more than ability of successive cohorts or the quality of the teaching, anchoring grade boundaries to median cohort performance should provide an effective method for setting standards. METHODS: This study investigated the use of a modified Hofstee (MH) method for setting unsatisfactory/satisfactory and satisfactory/excellent grade boundaries for multiple choice question-style assessments, adjusted using the cohort median to obviate the effect of subjective judgements and provision of grade quotas. RESULTS: Outcomes for the MH method were compared with formula scoring/correction for guessing (FS/CFG) for 11 assessments, indicating that there were no significant differences between MH and FS/CFG in either the effective unsatisfactory/satisfactory grade boundary or the proportion of unsatisfactory graded candidates (p > 0.05). However the boundary for excellent performance was significantly higher for MH (p < 0.01), and the proportion of candidates returned as excellent was significantly lower (p < 0.01). MH also generated performance profiles and pass marks that were not significantly different from those given by the Ebel method of criterion-referenced standard setting. CONCLUSIONS: This supports MH as an objective model for calculating variable grade boundaries, adjusted for test difficulty. Furthermore, it easily creates boundaries for unsatisfactory/satisfactory and satisfactory/excellent performance that are protected against grade inflation. It could be implemented as a stand-alone method of standard setting, or as part of the post-examination analysis of results for assessments for which pre-examination criterion-referenced standard setting is employed.

Promoting equality by assessing discrimination in clinical practice.

How well do you understand your impact on the opportunities of others, and their impact on your opportunities? Do you reflect on the equality of your values and whether your behaviour could be improved? This article gives an overview of these issues for hospital doctors.

Towards a better understanding of clinical leadership.

The effectiveness of leadership is important to everyone, yet leadership is often misunderstood and represented as undesirable. There are many theoretical models of the virtues which make a good leader, but comparatively little practical advice. How well do you understand your ability to influence others? How could you improve?

Group in-course assessment promotes cooperative learning and increases performance.

The authors describe and evaluate a method to motivate medical students to maximize the effectiveness of dissection opportunities by using In-Course-Assessments (ICAs) to encourage teamwork. A student's final mark was derived by combining the group dissection mark, group mark for questions, and their individual question mark. An analysis of the impact of the ICA was performed by comparing end of module practical summative marks in student cohorts who had, or had not, participated in the ICAs. Summative marks were compared by two-way ANOVA followed by Dunnets test, or by repeated measures ANOVA, as appropriate. A cohort of medical students was selected that had experienced both practical classes without (year one) and with the new ICA structure (year two). Comparison of summative year one and year two marks illustrated an increased improvement in year two performance in this cohort. A significant increase was also noted when comparing this cohort with five preceding year two cohorts who had not experienced the ICAs (P <0.0001). To ensure that variation in the practical summative examination was not impacting on the data, a comparison was made between three cohorts who had performed the same summative examination. Results show that students who had undertook weekly ICAs showed significantly improved summative marks, compared with those who did not (P <0.0001). This approach to ICA promotes engagement with learning resources in an active, team-based, cooperative learning environment.

Delivery and use of individualised feedback in large class medical teaching.

BACKGROUND: Formative feedback that encourages self-directed learning in large class medical teaching is difficult to deliver. This study describes a new method, blueprinted feedback, and explores learner's responses to assess its appropriate use within medical science teaching. METHODS: Mapping summative assessment items to their relevant learning objectives creates a blueprint which can be used on completion of the assessment to automatically create a list of objectives ranked by the attainment of the individual student. Two surveys targeted medical students in years 1, 2 and 3. The behaviour-based survey was released online several times, with 215 and 22 responses from year 2, and 187, 180 and 21 responses from year 3. The attitude-based survey was interviewer-administered and released once, with 22 responses from year 2 and 3, and 20 responses from year 1. RESULTS: 88-96% of learners viewed the blueprinted feedback report, whilst 39% used the learning objectives to guide further learning. Females were significantly more likely to revisit learning objectives than males (p = 0.012). The most common reason for not continuing learning was a 'hurdle mentality' of focusing learning elsewhere once a module had been assessed. CONCLUSIONS: Blueprinted feedback contains the key characteristics required for effective feedback so that with further education and support concerning its use, it could become a highly useful tool for the individual and teacher.

Cellular assessment of the extract of bambangan (Mangifera pajang) as a potential cytoprotective agent for the human hepatocellular HepG2 cell line.

This study was conducted to investigate the potential of bambangan (Mangifera pajang) fruit extracts in the protection against oxidative damage caused by tert-butyl hydroperoxide in the human hepatocellular HepG2 cell line. Proteins which might be involved in the cytoprotective mechanism were investigated using western blotting technique. Quercetin was used as a positive control. The results showed that only the kernel extract of M. pajang and quercetin displayed cytoprotective activity in HepG2 cells, with EC(50) values of 1.2 and 5.3µg/ml, respectively. Expression of quinone reductase, glutathione reductase and methionine sulfoxide reductase A proteins were significantly up-regulated by quercetin, suggesting their involvement in the cytoprotective activity of quercetin. However, expressions of only glutathione reductase and methionine sulfoxide reductase A proteins were significantly up-regulated by the kernel extract, again suggesting their involvement in the cytoprotective activity of bambangan kernel extract. Future study is needed to investigate the involvement of other cytoprotective proteins in the cytoprotection mechanism.

A double-blind atropine trial for active learning of autonomic function.

Here, we describe a human physiology laboratory class measuring changes in autonomic function over time in response to atropine. Students use themselves as subjects, generating ownership and self-interest in the learning as well as directly experiencing the active link between physiology and pharmacology in people. The class is designed to concomitantly convey the importance of bias in experimentation by adopting a double-blind placebo-controlled approach. We have used this class effectively in various forms with ∼600 students receiving atropine over the last 16 yr. This class has received favorable feedback from staff and students of medicine, pharmacy, and neuroscience, and we recommend it for such undergraduates. The learning objectives that students are expected to achieve are to be able to 1) know the ethical, safety, and hygiene requirements for using human volunteers as subjects; 2) implement and explain a double-blind placebo-controlled trial; 3) design, agree, and execute a protocol for making (and accurately recording) precise reproducible measurements of pulse rate, pupil diameter, and salivary flow; 4) evaluate the importance of predose periods and measurement consistency to detect effects (including any reversibility) after an intervention; 5) experience direct cause-and-effect relationships integrating physiology with pharmacology in people; 6) calculate appropriate summary statistics to describe the data and determine the data's statistical significance; 7) recognize normal variability both within and between subjects in baseline physiological parameters and also recognize normal variability in response to pharmacological treatment; 8) infer the distribution and role of muscarinic receptors in the autonomic nervous system with respect to the heart, eye, and mouth; 9) identify and explain the clinical significance of differences in effect due to the route and formulation of atropine; 10) produce and deliver a concise oral presentation of experimental findings; and 11) produce a written report in the form of a short scientific research article. The results of a typical study are presented, which demonstrate that the administration of atropine by a subcutaneous injection elicited a significant increase in pulse rate and pupil diameter and a significant decrease in salivary flow, whereas administration of atropine in an oral liquid elicited significant effects on pulse rate and salivary flow, and an oral solid format elicited a significant alteration in salivary flow alone. More detailed analysis of the salivary flow data demonstrated clear differences between the routes of administration and formulation in the onset and magnitude of action of atropine.

Cytotoxicity, cell cycle arrest, and apoptosis in breast cancer cell lines exposed to an extract of the seed kernel of Mangifera pajang (bambangan).

An extract of Mangifera pajang kernel has been previously found to contain a high content of antioxidant phytochemicals. The present research was conducted to investigate the anticancer potential of this kernel extract. The results showed that the kernel crude extract induced cytotoxicity in MCF-7 (hormone-dependent breast cancer) cells and MDA-MB-231 (non-hormone dependent breast cancer) cells with IC50 values of 23 and 30.5 microg/ml, respectively. The kernel extract induced cell cycle arrest in MCF-7 cells at the sub-G1 (apoptosis) phase of the cell cycle in a time-dependent manner. For MDA-MB-231 cells, the kernel extract induced strong G2-M arrest in cell cycle progression at 24h, resulting in substantial sub-G1 (apoptosis) arrest after 48 and 72 h of incubation. Staining with Annexin V-FITC and propidium iodide revealed that this apoptosis occurred early in both cell types, 36 h for MCF-7 cells and 24 h for MDA-MB-231 cells, with 14.0% and 16.5% of the cells respectively undergoing apoptosis at these times. This apoptosis appeared to be dependent on caspase-2 and -3 in MCF-7 cells, and on caspase-2, -3 and -9 in MDA-MB-231 cells. These findings suggest that M. pajang kernel extract has potential as a potent cytotoxic agent against breast cancer cell lines.

The effects of combined exposure to the pyrethroids deltamethrin and S-bioallethrin on hippocampal inhibition and skeletal muscle hyperexcitability in rats.

The default assumption that different pyrethroid insecticides, sharing a common mode of action, will show additivity of toxicity has not always been supported by in vitro measures, some of which have indicated antagonism. Our intention was to see whether the antagonism between pyrethroids of different classes seen in vitro could be reproduced in vivo. We therefore investigated the effects of single and combined exposures to two commonly used pyrethroids, deltamethrin (type II) and S-bioallethrin (type I) given intravenously to anaesthetised rats. We used two quantitative measures that are responsive to pyrethroids: the duration of prolongation of hippocampal dentate granule cell inhibition and the amplitude of the abnormal electromyogram discharge. At equi-toxic doses, S-bioallethrin extended the inter-stimulus interval evoking 50% inhibition in the hippocampus by 30+/-2.2 ms, and deltamethrin extended it by 199+/-21 ms. Combined administration of the same doses of deltamethrin and S-bioallethrin extended hippocampal inhibition by 164+/-14 ms, which did not differ significantly from the effect of deltamethrin alone. S-bioallethrin was without any effect on the electromyogram, and produced no significant change in the amplitude of the abnormal muscle discharges evoked by deltamethrin. The increase in arterial blood pressure evoked by the combination was significantly less than that evoked by either pyrethroid alone (p<0.001). In summary, although our electrophysiological indices provide no support for functional antagonism between these two pyrethroids, they also fail to indicate any summation of effect.

Structure-activity and interaction effects of 14 different pyrethroids on voltage-gated chloride ion channels.

We have proposed that since the type II pyrethroids deltamethrin and cypermethrin, but not the type I pyrethroid cismethrin act on chloride channels, this could contribute to the bimodal nature of pyrethroid poisoning syndromes. We now examine a wider range of pyrethroid structures on the activity of these calcium-independent voltage-gated maxi-chloride channels. Excised inside-out membrane patches from differentiated mouse neuroblastoma cells were used, and mean channel open probabilities calculated. For single dosing at 10 microM, bioallethrin, beta-cyfluthrin, cypermethrin, deltamethrin, and fenpropathrin were all found to significantly decrease open channel probability (p < 0.05). Bifenthrin, bioresmethrin, cispermethrin, cisresmethrin, cyfluthrin isomers 2 and 4, lambda-cyhalothrin, esfenvalerate, and tefluthrin, did not significantly alter open channel probability (p > 0.05). Since the type II pyrethroids, esfenvalerate, and lambda-cyhalothrin were ineffective, we must conclude that actions at the chloride ion channel target cannot in themselves account for the differences between the two types of poisoning syndrome. Sequential dosing with type II pyrethroids caused no further chloride ion channel closure. The type I pyrethroid cisresmethrin did however prevent a subsequent effect by the mixed type pyrethroid fenpropathrin. In contrast, the type I pyrethroid cispermethrin did not prevent a subsequent effect due to the type II pyrethroid deltamethrin. The difference in effect may be the result of differences in potency, as deltamethrin had a greater effect than fenpropathrin. It therefore appears clear that in some combinations the type I and type II pyrethroids can compete and may bind to the same chloride channel target site.