Does student confidence on multiple-choice question assessments provide useful information?

CONTEXT: Feedback from multiple-choice question (MCQ) assessments is typically limited to a percentage correct score, from which estimates of student competence are inferred. The students' confidence in their answers and the potential impact of incorrect answers on clinical care are seldom recorded. Our purpose was to evaluate student confidence in incorrect responses and to establish how confidence was influenced by the potential clinical impact of answers, question type and gender. METHODS: This was an exploratory, cross-sectional study conducted using a convenience sample of 104 Year 3 dental students completing 20 MCQs on implant dentistry. Students were asked to select the most correct response and to indicate their confidence in it for each question. Identifying both correctness and confidence allowed the designation of uninformed (incorrect and not confident) or misinformed (incorrect but confident) responses. In addition to recording correct/incorrect responses and student confidence, faculty staff designated incorrect responses as benign, inappropriate or potentially harmful if applied to clinical care. Question type was identified as factual or complex. Logistic regression was used to evaluate relationships between student confidence, and question type and gender. RESULTS: Students were misinformed more often than uninformed (22% versus 8%), and misinformed responses were more common with complex than factual questions (p < 0.05). Students were significantly more likely to be confident of correct than incorrect benign, incorrect inappropriate or incorrect harmful answers (p < 0.001), but, contrary to expectations, confidence did not decrease as answers became more harmful. CONCLUSIONS: Recording student confidence was helpful in identifying uninformed versus misinformed responses, which may allow for targeted remediation strategies. Making errors of calibration (confidence and accuracy) more visible may be relevant in feedback for professional development.

Identifying student misconceptions in biomedical course assessments in dental education.

Dental student performance on examinations has traditionally been estimated by calculating the percentage of correct responses rather than by identifying student misconceptions. Although misconceptions can impede student learning and are refractory to change, they are seldom measured in biomedical courses in dental schools. Our purpose was to determine if scaling student confidence and the clinical impact of incorrect answers could be used on multiple-choice questions (MCQs) to identify potential student misconceptions. To provide a measure of student misconception, faculty members indicated the correct answer on twenty clinically relevant MCQs and noted whether the three distracters represented potentially benign, inappropriate, or harmful application of student knowledge to patient treatment. A group of 105 third-year dental students selected what they believed was the most appropriate answer and their level of sureness (1 to 4 representing very unsure, unsure, sure, and very sure) about their answer. Misconceptions were defined as sure or very sure incorrect responses that could result in inappropriate or harmful clinical treatment. In the results, 5.2 percent of the answers represented student misconceptions, and 74 percent of the misconceptions were from four case-based interpretation questions. The mean student sureness was 3.6 on a 4.0 scale. The students' sureness was higher with correct than with incorrect answers (p<0.001), yet there was no difference in sureness levels among their incorrect (benign, inappropriate, or harmful) responses (p>0.05). This study found that scaling student confidence and clinical impact of incorrect answers provided helpful insights into student thinking in multiple-choice assessment.

Dental students' self-assessment of preclinical examinations.

Accurate self-assessment is an important attribute for practicing dentists and, therefore, an important skill to develop in dental students. Our purpose was to examine the relationship between faculty and student assessments of preclinical prosthodontic procedures. Seventy-six second-year students completed two consecutive examinations and two self-assessments. The examinations involved setting maxillary denture teeth on a model to simulate the clinical procedure of a complete maxillary denture. Results indicated no significant increases in examination or student self-assessment mean scores; however, regression analysis indicated changes in student self-assessment scores explained 16.3 percent of the variation in examination scores. In essence, improvement in student self-assessment predicted improvement in examination scores among dental students completing a preclinical dental procedure.

Correlation of admissions criteria with academic performance in dental students.

Our purpose was to compare admissions criteria as predictors of dental school performance in underachieving and normally tracking dental students. Underachieving dental students were identified by selecting ten students with the lowest class grade point average following the first year of dental school from five classes, resulting in a pool of fifty students. Normally tracking students served as a control and were randomly selected from students who had completed their first year of dental school not in the underachieving group. Admission measures of college grade point average (GPA), science grade point average (SGPA), academic average (AA), Perceptual Ability Test (PAT), college rigor, and academic load in college were evaluated with descriptive statistics, correlation, and regression analysis with first-year and graduating GPA as the dependent variables. Admissions criteria were generally weak predictors of first-year and graduating GPA. However, first-year dental school GPA was a strong predictor (R(2)=0.77) of graduating GPA for normally tracking students and a moderate predictor (R(2)=0.58) for underachieving students. Students who completed the first year of dental school having a low GPA tended to graduate with a low GPA. Therefore, remediation and monitoring would be important during the dental school experience for these students.

The correlation of student performance in preclinical and clinical prosthodontic assessments.

Tracking student performance in preclinical and clinical courses can be helpful in developing and refining a curriculum. Our objective was to correlate student performance on three fixed prosthodontic examinations taken by eighty junior dental students. Examinations included a knowledge-based objective structured clinical examination (OSCE), a manual skills exercise completed on a typodont (Typodont), and a competency casting exam (Casting CE) on a patient. Multiple regression analysis indicated that the OSCE and Typodont exam scores, as independent variables, were not statistically significant predictors (P=0.07; P=0.87, respectively) of Casting CE exam performance, which was the dependent variable. Correlations were weak for the OSCE (r=0.21) and nearly nonexistent for the Typodont exam(r=0.03) when compared to the Casting CE. Our results indicate a weak correlation between an OSCE-based knowledge exam measuring students' knowledge of critical errors in preparations and castings and a competency exam involving the preparation of a full veneer crown. Results also indicate virtually no correlation between a typodont preparation examination designed to provide a measure of students' clinical skill and a clinical competency exam involving the preparation of a full crown.