Assessing performance, productivity, and staffing needs in pathology groups: observations from the College of American Pathologists PathFocus pathology practice activity and staffing program.

CONTEXT: The PathFocus program affords the opportunity for participating pathology practices to be compared with other practices that have similar characteristics. OBJECTIVES: To demonstrate variability in workload among different pathology practice settings and to determine practice characteristics that influence staffing levels. DESIGN: Among 228 group practices in the PathFocus database, group practice settings were analyzed. The practice characteristics that were highly correlated with staffing levels are presented. RESULTS: Activities that showed significant variation include surgical pathology (P = .003), cytopathology (P = .006), miscellaneous (P = .006), and professional development (P = .003). Group practices report up to 4% of hours devoted to clinical pathology consultation, on average, and from 20% to 25% to administration and management. There are strong positive associations with staffing levels for lower-complexity Current Procedural Terminology code volumes (P < .001) and higher-complexity Current Procedural Terminology code volumes (P = .006). CONCLUSION: The settings of pathology practices carry specific commitments of time that are different and not equally distributed among all practice settings and strongly influence staffing requirements.

Altered recognition of reparative changes in ThinPrep specimens in the College of American Pathologists Gynecologic Cytology Program.

CONTEXT: Previous studies have shown that the diagnosis of reparative changes in conventional smears in the College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology is one of the least reproducible diagnoses. Indeed, the diagnosis of reparative changes consistently yields the highest false-positive rate of any negative for intraepithelial lesions and malignancy (NILM) cytodiagnostic category. It is unknown whether cytologists recognize reparative changes in ThinPrep specimens as well, or less often, as in conventional smears. OBJECTIVE: To assess and compare the ability of cytologists to recognize reparative changes in conventional and ThinPrep preparations. DESIGN: We compiled performance data from the College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology from the 2000-2003 program years. More than 400 slides with a reference diagnosis of reparative changes met our study criteria, representing a total of 11 200 individual responses for conventional cases and 1155 individual responses for ThinPrep specimens. We evaluated the results of both individual and laboratory participants using 2 performance criteria: the false-positive discordancy rate and the exact match error rate (any response that does not exactly match the reference diagnosis of 120 [reparative changes]). RESULTS: Cases with a reference diagnosis of reparative changes made up 1.2% of all ThinPrep slides and 3.7% of all conventional slides in circulation. The false-positive discordancy rate of individual responses on educational slides for conventional smears was significantly higher than the corresponding false-positive discordancy rate for ThinPrep specimens (15.7% for conventional vs 7.1% for ThinPrep specimens, P < .001). Laboratory responses on educational conventional smears and ThinPrep slides showed a similar trend (14.2% for conventional smears vs 2.4% for ThinPrep slides, P = .002). The exact match error rate on educational conventional slides was 41.4% for individual responses, while on educational ThinPrep slides, the overall error rate was 57.5% (P < .001). For laboratory responses, the exact match error rate was 40.5% for educational conventional smears versus 58.9% for educational ThinPrep smears (P < .001). Characteristic features of reparative changes were identified in ThinPrep specimens. CONCLUSIONS: In the College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology, ThinPrep slides with a reference diagnosis of reparative changes have a lower false-positive discordancy rate than conventional slides. Responses to ThinPrep cases with a reference diagnosis of reparative change show a higher exact match error rate than conventional smears. Since reparative changes in gynecologic cytology are recognized as indicating an increased risk of significant lesions, the clinical significance of these altered patterns of recognition of reparative changes in ThinPrep specimens warrants further investigation.

Detection and correction of systematic laboratory problems by analysis of clustered proficiency testing failures.

CONTEXT: The Laboratory Accreditation Program of the College of American Pathologists monitors the performance of its subscribers in proficiency testing (PT). Failure to perform as expected prompts the program to query the laboratory. OBJECTIVE: To determine whether laboratories are correcting apparent problems when contacted by the program about repeatedly unacceptable performance in a diagnostic test. DESIGN: Retrospective analysis of 1 year's records (2002-2003) from the College's Proficiency Testing Exception Summary correspondence, which identifies clusters of PT failures. The analysis focused on those laboratories in which the Proficiency Testing Exception Summary algorithm identified repeated failures over 3 or 4 testing events; PT performance is monitored as a condition of accreditation. During 1 survey year, approximately 6300 accredited laboratories collectively tested approximately 1,205,000 analytes and submitted results to their PT providers on more than 3,500,000 PT challenges. During the period of observation, 14,085 Proficiency Testing Exception Summary reports were mailed to participants. Educational materials were included to help laboratories identify and correct each PT failure. RESULTS: There were only 1304 cases of repeated PT failures after the initial correspondence from the accreditation program (9.3%). Of these, there were only 119 cases of unsatisfactory results on the subsequent PT event (9.1%). All systematic problems were resolved by the conclusion of the third round of correspondence. CONCLUSIONS: Accredited laboratories generally perform well in proficiency testing. Identification of clusters of PT failures by the accreditation provider can help those laboratories having analytic difficulties to investigate and correct the problems.

Bethesda 2001 implementation and reporting rates: 2003 practices of participants in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology.

CONTEXT: The 2001 Bethesda System for the reporting of cervical cytology specimens and the use of new liquid-based and human papillomavirus testing technologies have led to changes in cervical cytology reporting practices. OBJECTIVES: To analyze current laboratory reporting practices using Bethesda 2001 terminology and to compare results with previous survey data from 1996. DESIGN: Questionnaire survey mailed to 1751 laboratories in mid-2003. PARTICIPANTS: Laboratories enrolled in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology. RESULTS: Of the 759 responding laboratories, most (85.5%) had implemented Bethesda 2001 terminology, and the majority had adopted major changes, such as elimination of the benign cellular changes category and the satisfactory but limited category. The median reporting rate for low-grade squamous intraepithelial lesion was 2.1%, compared to a 1996 median rate of 1.6%, but the increase was confined to liquid-based preparations. Reporting rates for high-grade squamous intraepithelial lesion (median, 0.5%) and atypical squamous cells (ASC) had changed little. Most ASC cases were subclassified as "undetermined significance" (median, 3.9%) with ASC, cannot exclude high-grade SIL accounting for a minority (median, 0.2%). The median ratio of ASC to squamous intraepithelial lesions and carcinomas (SIL+) was 1.4 and was lower than the 1996 median ratio of 2.0. Median reporting rates for squamous abnormalities for 2002 were noted to be significantly higher for liquid-based preparations than for conventional smears, while median ASC/SIL+ ratios were lower. Most laboratories offer human papillomavirus testing, but almost half (47%) of laboratories do not know the percentage of positive testing results in patients with ASC. CONCLUSIONS: Most laboratories have implemented Bethesda 2001 terminology. New criteria and liquid-based methods have led to an increase in low-grade squamous intraepithelial lesion reporting rates and a decrease in ASC/ SIL+ ratios when compared with 1996 data. Liquid-based preparations have higher median squamous intraepithelial lesion rates and lower ASC/SIL+ ratios than conventional smears.

Calibration verification performance relates to proficiency testing performance.

CONTEXT: Since 1988, the College of American Pathologists has been offering materials for calibration verification coupled with the surveys for linearity, called the linearity (LN) surveys. OBJECTIVE: To determine whether successful completion of the College of American Pathologists LN surveys provides a benefit in terms of improved proficiency testing (PT) performance. DESIGN: In this study, we used information from LN surveys LN1/2, LN3, and LN5 and from the PT surveys C, Z, and K administered and analyzed in the year 2000. For the PT data, we calculated 4 measures of performance: passing PT, results exceeding 2 SDs, sum of absolute SD intervals, and the absolute sum of SD intervals. For the LN data, we classified laboratories as participants versus nonparticipants in LN surveys and by whether or not LN survey performance was successful. RESULTS: LN enrollees had fewer unacceptable PT results than did nonenrollees. Additionally, for many analytes there was a significant positive association between LN performance and PT performance. CONCLUSIONS: For most analytes studied, there was strong evidence linking performance on PT surveys with performance on LN surveys. Eight of 13 analyses (62%) demonstrated improved performance with successful calibration verification.

Comparison of performance of conventional and ThinPrep gynecologic preparations in the College of American Pathologists Gynecologic Cytology Program.

CONTEXT: Results of clinical trials suggest that interpretation of liquid-based cytology preparations is more accurate and is associated with less screening error than interpretation of conventional preparations. OBJECTIVE: In this study, the performance of participants in interpreting ThinPrep (TP) preparations was compared with participants' performance on conventional Papanicolaou tests in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology (PAP). DESIGN: The results of the PAP from the year 2002 were reviewed, and the discordancies to series and exact-match error rates for the 2 cytologic methods were compared. RESULTS: For this study, a total of 89 815 interpretations from conventional smears and 20 886 interpretations from TP samples were analyzed. Overall, interpretations of TP preparations had both significantly fewer false-positive (1.6%) and false-negative (1.3%) rates than those of conventional smears (P =.001 and P =.02, respectively) for validated or validated-equivalent slides, as assessed by concordance with the correct diagnostic series. In this assessment of concordance to series, interpretations of educational TP and conventional preparations were similar, except for high-grade squamous intraepithelial lesion, in which the performance was significantly worse for educational TP preparations (false-negative rate of 8.1% vs 4.1% for conventional smears, P <.001). When interpretations were matched to the exact diagnosis, validated-equivalent TP preparations were generally more accurate for diagnoses in the 100 series and 200 series than were conventional smears. Notably, for the reference diagnosis of squamous cell carcinoma, the exact-match error rate on validated equivalent TP slides was significantly greater than that of conventional slides (44.5% vs 23.1%, P <.001). Interpretations of educational TP preparations also had a significantly higher error rate in matching to the exact reference diagnosis for squamous cell carcinoma (33.7% vs 22.8%, P =.007). CONCLUSIONS: Overall, TP preparations in this program were associated with significantly lower error rates than conventional smears for both validated and educational cases. However, unlike the negative for intraepithelial lesion and malignancy, not otherwise specified, low-grade squamous intraepithelial lesion, and adenocarcinoma cytodiagnostic challenges, participants' responses indicated some difficulty in recognizing high-grade squamous intraepithelial lesion and squamous cell carcinoma.

Precision in gynecologic cytologic interpretation: a study from the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology.

CONTEXT: Numerous studies address the accuracy or positive predictive value of cytologic interpretations for defined histopathologic entities. The reproducibility (precision) of cytologic interpretation is less well defined. OBJECTIVE: To establish and compare the reproducibility (precision) of cytologic interpretation in gynecologic cytopathology, as reflected in the educational program of the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology (PAP). METHODS: The pathologists' interpretations for both validated (25 745 responses) and educational conventional (14 353 responses) slides in the PAP program for 2001 were analyzed. The frequency of exact matches between the reference and pathologists' interpretation for each of the cytologic interpretative categories was identified, and the cumulative distributions of exact match rates were derived. chi2 Tests by reference interpretations were used for cytodiagnostic categories, least and most reproducible groupings, and high-grade (HSIL) versus low-grade (LSIL) squamous intraepithelial lesions. RESULTS: Pathologists' interpretations of negative, Candida, Trichomonas, herpes, and LSIL were characterized by a high degree of exact matching, while interpretations of repair, HSIL, adenocarcinoma, and squamous cell carcinomas were characterized by a lesser degree of exact matching (reproducibility). Pathologists' cytologic interpretations of HSIL were significantly less reproducible than those of LSIL. CONCLUSION: The cytologic interpretations of the most significant categories (HSIL, squamous cell carcinoma, and adenocarcinoma) are less precise than those of specific infection (Candida, Trichomonas, and yeast), negative, and LSIL categories. Cytologic interpretations of LSIL are made with greater precision than those of HSIL and may represent a more appropriate endpoint to measure the precision performance of gynecologic cytology laboratories.