Development of a bench station objective structured assessment of technical skills.

OBJECTIVE: We have previously shown that objective structured assessment of technical skills performed in an animal model was an innovative, reliable, and valid method of assessing surgical skills. Our goal was to develop a less costly bench station objective structured assessment of technical skills and to evaluate the feasibility, reliability, and validity of this exam. METHODS: A seven-station examination was administered to 24 residents. The tests included laparoscopic procedures (salpingostomy, intracorporeal knot tying, closure of port sites) and open abdominal procedures (subcuticular closure, bladder neck suspension, repair of enterotomy, abdominal wall closure). All tasks were performed using life-like surgical models. Residents were timed and assessed at each station using three methods of scoring: a task-specific checklist, a global rating scale, and a pass/fail grade. RESULTS: Assessment of construct validity, the ability of the test to discriminate among residency levels, found significant differences on the checklist, global rating scale, time for procedures, and pass/fail grade by level of training. Reliability indices calculated with Cronbach's alpha were 0.77 for the checklists and 0.94 for the global rating scale. Overall interrater reliability indices were 0.91 for the global rating scale and 0.92 for the checklists. Total cost for replaceable parts and facilities was $1900. CONCLUSION: The less costly and more portable bench station objective structured assessment of technical skills can reliably and validly assess the surgical skills of gynecology residents. This type of examination can be a useful tool to identify residents who need additional surgical instruction, provide remediation, and may become a mechanism to certify surgical skill competence.

Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability.

OBJECTIVE: Resident surgical skills are acquired mainly through observing and later performing procedures in the operating room. Evaluation of surgical skills has traditionally been done through subjective faculty evaluation, a technique that has poor reliability and unknown validity. Our goal was to develop specific surgical tasks, both laparoscopic and open abdominal, that could be objectively and reliably evaluated in a bench laboratory setting. STUDY DESIGN: The prospective development of a reliable and valid resident surgical skills test in a bench laboratory setting was our goal. A written test of surgical knowledge and 12 skills tests were administered to 36 residents. Laparoscopic bench tasks were simulated with the use of a box and camera with a video display. Six laparoscopic tasks were assessed, including placing pegs on a board, running the bowel simulation, and other tasks that involve hand-eye coordination and manual dexterity. Open abdominal skills simulated incision closure, suturing a vaginal cuff, knot tying, and using a tie on a passer. Residents were timed at each given station and were given a rating score by 2 examiners. RESULTS: Knowledge scores showed a significant improvement by residency level. Assessment of construct validity (the ability to discriminate among residency levels) demonstrated significant differences on the rating of overall performance and individual tasks by level (determined by 1-way analysis of variance). Interrater reliability (agreement between 2 raters) with the use of intraclass correlation was 0.79 for the total score. The cost to administer the bench laboratory test was less than $50 and required 30 hours of faculty time. CONCLUSION: The results of this study suggest that surgical bench laboratory tasks can assess residents' surgical skills with good reliability and validity on most tasks. Our previous study, which used an animal laboratory, was expensive, and the bench laboratory model may provide an alternative means to assess surgical skills.

Development of an objective structured assessment of technical skills for obstetric and gynecology residents.

OBJECTIVE: To develop an objective structured assessment for evaluating surgical skills of obstetrics and gynecology residents and to evaluate the reliability and validity of the assessment. METHODS: A seven-station, objective, structured assessment of technical skills was administered to 24 residents. The test included laparoscopic procedures (port placement, salpingostomy, suturing, vessel ligation) and open abdominal procedures (hypogastric ligation, repair of enterotomy, salpingo-oophorectomy.) All surgical tasks were done on pigs. Residents were timed and assessed at each station using three methods of scoring, a task-specific checklist, global rating scale, and pass-fail grade. RESULTS: Assessment of construct validity (the ability of the test to discriminate among residency levels) found significant differences on the checklist and the global rating scale by residency level. Reliability indices calculated with Cronbach's alpha were 0.89 for the global rating scale and 0.89-0.95 for the individual skills checklists. Interrater reliability was 0.87 for the global rating scale and 0.78-0.98 for the checklists. CONCLUSION: Objective, structured assessment of technical skills can assess residents' surgical skills with high reliability and validity. These assessments have possible application for identifying residents who need additional training and might provide a mechanism to ensure competence of surgical skills.

Information and informatics literacy: skills, timing, and estimates of competence.

BACKGROUND: Computing and biomedical informatics technologies are providing almost instantaneous access to vast amounts of possibly relevant information. Although students are entering medical school with increasingly sophisticated basic technological skills, medical educators must determine what curricular enhancements are needed to prepare learners for the world of electronic information. PURPOSE: The purpose was to examine opinions of academic affairs and informatics administrators, curriculum deans and recently matriculated medical students about prematriculation competence and medical education learning expectations. METHODS: Two surveys were administered: an Information Literacy Survey for curriculum/informatics deans and a Computing Skills Survey for entering medical students. RESULTS: Results highlight differences of opinion about entering competencies. They also indicate that medical school administrators believe that most basic information skills fall within the domain of undergraduate medical education. CONCLUSIONS: Further investigations are needed to determine precise entry-level skills and whether information literacy will increase as a result of rising levels of technical competence.

Formal teaching of surgical skills in an obstetric-gynecologic residency.

OBJECTIVE: To describe a formal teaching program of basic surgical skills in an obstetric-gynecologic residency program and evaluate its effectiveness. METHODS: A surgical skills program was developed for all residents. Using bench and animal laboratory sessions, residents were given instruction and performed both laparoscopic and open abdominal procedures. All were given a pretest and were tested again 6 months later. Residents also evaluated their experiences. RESULTS: To date, the formal teaching sessions have been given to 24 residents, all of whom believed their confidence and technical skills improved as a result of the sessions. On a scale of 1 to 5, the median rating of the bench laboratory experience was 5 (range 4-5), and the pig laboratory was 5 (range 4-5). All residents believed the surgery sessions should be continued and the number of sessions increased. Preliminary evaluation indicated that time to suture a 10-inch incision decreased by 28%, from an average of 225 (standard deviation [SD] 51) seconds to 171 (SD 43) seconds (P < .001), and evaluation of surgical technique significantly improved at the second pretest (P = .013). Laparoscopic placement of pegs on a board in 2 minutes increased from an average of 5 (SD 2.5) to 7.3 (SD 2.6; P = .001). The cost of the bench laboratory sessions was minimal. Each pig was approximately $100, and the facility charge for each 4-hour laboratory session was $1500. CONCLUSION: When formal surgical training was given to obstetric-gynecologic residents, their surgical skills improved subjectively and objectively.

Comparison of incoming dental school patients with and without disabilities.

A survey of incoming dental school patients compared 64 adult patients (DECOD) and 73 patients without disability (ND), regarding past dental experience, current needs, and basis for selecting the school's clinics. The responses indicated that, for DECOD patients, clinic selection was based largely on Medicaid acceptance, staff experience, and inability of other dentists to manage their disability; for ND patients, selection was based on lower fee structure. Both groups expressed high treatment need, but the rate was lower for DECOD than for ND patients. More DECOD patients reported severe dental anxiety and adverse effects of dental problems on general health. Chart records revealed that clinical findings exceeded perceived need for both DECOD and ND patients. While both groups had high periodontal disease rates (91%), DECOD patients had significantly poorer oral hygiene and less restorative need than ND patients. The findings suggest differences between persons with disabilities and other patient groups in difficulty of access to dental services in the community, reasons for entering the dental school system, and in presenting treatment need and/or treatment planning.

Evaluating students' substance abuse assessment skills.

Graduating nursing students' abilities to assess clients for substance use were evaluated using two standardized patient (SP) cases. In the comprehensive health history case (unprompted), 55% of the study participants assessed alcohol use and 20% assessed drug use. When prompted to conduct a substance use history, at least 90% of the subjects asked about the quantity and frequency of alcohol use and 80% asked about drug use. However, in the prompted case, fewer than 10% of the subjects linked current alcohol and drug (AOD) use with risks and consequences, nor did they ask the SP if he had considered decreasing substance use. The mean score for communication skills used in the two assessment interviews were 3.65 and 3.56 on a 1 to 6 Likert scale. These data point out the need for additional emphasis on instructing students to include AOD questions as part of routine assessments, what questions to ask in a substance use assessment, and how to act on the information received. The use of SPs is one way to obtain valid information about students' assessment competencies.

A comparison of oral health in spinal cord injury and other disability groups.

A controlled pilot study determined oral health in persons with quadriplegia due to spinal cord injury, and compared dental disease rates in spinal cord injury and other disability groups. Seventeen adults with spinal cord injury and 17 controls were assessed for dental/medical/social history; manual function; head, neck, and oral lesions; salivary flow; DMFS; and gingivitis, periodontal pockets, plaque, and calculus. Findings were compared with those from prior studies according to the same protocol, for groups of similar age with mental retardation, cerebral palsy, traumatic brain injury, and chronic mental illness. No significant differences between spinal cord injury and control subjects were noted, except that fewer spinal cord injury subjects brushed daily or flossed (p < 0.05); dependent subjects tended to have more plaque and gingivitis than those brushing independently. Subjects with spinal cord injury and mental illness had less gingivitis than those with mental retardation and cerebral palsy (p < 0.001); on calculus, subjects with spinal cord injury ranked lower than subjects with mental illness (p < 0.05). On DFS, mentally ill subjects and those with traumatic brain injury ranked higher than mentally retarded and cerebral palsy groups, with spinal cord injury subjects intermediate. Mentally retarded and traumatic-brain-injured subjects had fewer teeth than other groups (p < 0.05). The findings suggest differences in oral health status and oral care for various disabled populations.

Efficacy of chlorhexidine swabbing in oral health care for people with severe disabilities.

Chlorhexidine is effective when used as an oral rinse, but many disabled people cannot use such a protocol. A double-blind cross-over study tested the efficacy of applying chlorhexidine with a sponge-swab, in a sample of 76 severely disabled adults, drawn from diverse rehabilitation settings. Two randomly assigned groups applied 10 mL 0.12% chlorhexidine gluconate (Peridex, Procter & Gamble) or 10 mL placebo, using a "Toothette" (Halbrand) once daily, 5 times per week for 10 weeks. All subjects received 10 mL 0.05% NaF, applied similarly but separately from the test/placebo agent. Pre- and post-trial measures included perceived level of function and oral status, that is, DMFS, plaque, calculus, pocket depth, and tooth stain. The protocol received high levels of compliance and acceptance. Compared with placebo, swabbing with chlorhexidine resulted in consistent, and, in part, significant improvements in plaque, gingivitis, and periodontal pocket depth. Side effects of chlorhexidine, that is, tooth stain and calculus, were relatively minor. Perceived improvements in dental health were associated with improved physical health, appearance, and mouth odor. The results indicate that chlorhexidine swabbing is a useful oral disease preventive protocol for persons with disability.

A comparison of the oral health of persons with and without chronic mental illness in community settings.

Severe dental disease has been reported for patients receiving psychiatric treatment. This study compared the oral status of noninstitutionalized adults with chronic mental illness with a similar group without such history, and evaluated relative risk factors, for example, xerostomia, diet, hygiene, and poverty. A sample of 37 subjects with chronic mental illness (CMI) and 29 control subjects without mental illness were assessed for dental, medical and social history; head, neck, and oral soft tissue pathology; salivary flow; DMFS, gingivitis, loss of periodontal attachment, plaque, and calculus. The groups were equivalent in socio-economic level, education, dental history, and home care. All subjects with CMI received psychotropic medications (mean of 3.8 drugs for 10.3 years). The CMI group had significantly higher incidence in the following variables: self-reported dry mouth; consumption of carbonated beverages (P less than .001); mucosal, lip, and tongue lesions (P less than .01); coronal smooth surface caries (P less than .001); severity of plaque (P less than .001) and calculus (P less than .01); and salivary flow (P less than .05). No significant differences were evident in the M and F components of DMFS, in gingivitis or loss of attachment. The results indicate significant increases in risk factors and increased oral pathosis in persons with mental illness who live in community settings compared with a control group that showed dental neglect.