Development of an international glossary for clinical guidelines collaboration.

OBJECTIVES: Clinical practice guidelines (CPGs) are often created through collaboration among organizations. The use of inconsistent terminology may cause poor communication and delays. This study aimed to develop a glossary of terms related to collaboration in guideline development. STUDY DESIGN AND SETTING: A literature review of collaborative guidelines was performed to develop an initial list of terms related to guideline collaboration. The list of terms was presented to the members of the Guideline International Network Guidelines Collaboration Working Group, who provided presumptive definitions for each term and proposed additional terms to be included. The revised list was subsequently reviewed by an international, multidisciplinary panel of expert stakeholders. Recommendations received during this pre-Delphi review were implemented to augment an initial draft glossary. The glossary was then critically evaluated and refined through two rounds of Delphi surveys and a virtual consensus meeting with all panel members as Delphi participants. RESULTS: Forty-nine experts participated in the pre-Delphi survey, and 44 participated in the two-round Delphi process. Consensus was reached for 37 terms and definitions. CONCLUSION: Uptake and utilization of this guideline collaboration glossary by key organizations and stakeholder groups may facilitate collaboration among guideline-producing organizations by improving communication, minimizing conflicts, and increasing guideline development efficiency.

An International Needs Assessment Survey of Guideline Developers Demonstrates Variability in Resources and Challenges to Collaboration between Organizations.

BACKGROUND: The development of rigorous, high-quality clinical guidelines increases the need for resources and skilled personnel within guideline-producing organizations. While collaboration between organizations provides a unique opportunity to pool resources and save time and effort, the collaboration presents its own unique challenges. OBJECTIVE: To assess the perceived needs and current challenges of guideline producers worldwide related to guideline development and collaboration efforts. DESIGN: Survey questions were developed by the Guidelines International Network and the US GRADE Network, pilot-tested among attendees of a guideline development workshop, and disseminated electronically using convenience and snowball sampling methods. PARTICIPANTS: A total of 171 respondents representing 30 countries and more than 112 unique organizations were included in this analysis. MAIN MEASURES: The survey included free-response, multiple-choice, and seven-point Likert-scale questions. Questions assessed respondents' perceived value of guidelines, resource availability and needs, guideline development processes, and collaboration efforts of their organization. KEY RESULTS: Time required to develop high-quality systematic reviews and guidelines was the most relevant need (median=7; IQR=5.5-7). In-house resources to conduct literature searches (median=4; IQR=3-6) and the resources to develop rigorous guidelines rapidly (median=4; IQR=2-5) were perceived as the least available resources. Difficulties reconciling differences in guideline methodology (median=6; IQR=4-7) and the time required to establish collaborative agreements (median=6; IQR=5-6) were the most relevant barriers to collaboration between organizations. Results also indicated a general need for improvement in conflict of interest (COI) disclosure policies. CONCLUSION: The survey identified organizational challenges in supporting rigorous guideline development, including the time, resources, and personnel required. Connecting guideline developers to existing databases of high-quality systematic reviews and the use of freely available online platforms may facilitate guideline development. Guideline-producing organizations may also consider allocating resources to hiring or training personnel with expertise in systematic review methodologies or utilizing resources more effectively by establishing collaborations with other organizations.

A memorandum of understanding has facilitated guideline development involving collaborating groups.

OBJECTIVE: Collaboration between groups can facilitate the development of high-quality guidelines. While collaboration is often desirable, misunderstandings can occur. One method to minimize misunderstandings is the pre-specification of terms of engagement in a memorandum of understanding (MOU). This study considered when an MOU may be most helpful, and which key elements should be included. STUDY DESIGN AND SETTING: An international panel of representatives from guideline groups was convened. A literature review to identify publications and other documents relevant to the establishment of MOUs between two or more guideline groups, supplemented by available source documents, was used to inform development of a draft MOU resource. This was iteratively refined until consensus was achieved. RESULTS: The level of detail in an MOU may vary based on institutional preferences and the particular collaboration. Elements within an MOU include those pertaining to: (1) scope and purpose; (2) leadership and team; (3) methods and commitment; (4) review and endorsement; and (5) publication and dissemination. CONCLUSION: Since groups may have different expectations regarding how a collaboration will unfold, an MOU may mitigate preventable misunderstandings. The result may be a higher likelihood of producing a guideline without disruption and delay.

Evaluating the Adoption of Laboratory Practice Guidelines.

CONTEXT.­: To date, the College of American Pathologists (CAP) has developed 17 laboratory practice guidelines (LPGs) including updates. In 2013, the CAP was awarded a 5-year cooperative agreement grant from the United States Centers for Disease Control and Prevention to increase the effectiveness of LPGs. OBJECTIVE.­: To assess the awareness and adoption of 2 CAP LPGs: immunohistochemical (IHC) assay validation and initial workup of acute leukemia. DESIGN.­: Baseline surveys for each LPG were conducted in 2010 and 2015, respectively. To measure the adoption of guideline recommendations and inform future updates, a follow-up study consisting of surveys, telephone interviews, and focus group sessions was conducted in laboratories that indicated they perform IHC testing. A follow-up study for the acute leukemia LPG is planned. RESULTS.­: For the IHC Validation LPG, a total of 1624 survey responses, 40 telephone interviews, and discussions with 5 focus group participants were analyzed. The response rate for the aforementioned 3 modalities was 46%, 13%, and 3%, respectively. All modalities indicated most respondents were aware of the LPG and had adopted most or all of its recommendations. Respondents expressed needs for continued communication, increased specificity, and more prescriptive recommendations when the guideline is updated. CONCLUSIONS.­: While data-driven development of evidence-based LPGs requires significant resources, active data collection to identify gaps and assess adoption contributes to improved laboratory testing practices in support of patient care. The CAP identified sustainable modalities to track metrics and developed multiple tools that should improve guideline development, adoption, and implementation. Of these modalities, written or electronic surveys were the most logistically feasible and had the highest response rate.

Analytic Validation of Immunohistochemistry Assays: New Benchmark Data From a Survey of 1085 Laboratories.

CONTEXT: - A cooperative agreement between the College of American Pathologists (CAP) and the United States Centers for Disease Control and Prevention was undertaken to measure laboratories' awareness and implementation of an evidence-based laboratory practice guideline (LPG) on immunohistochemical (IHC) validation practices published in 2014. OBJECTIVE: - To establish new benchmark data on IHC laboratory practices. DESIGN: - A 2015 survey on IHC assay validation practices was sent to laboratories subscribed to specific CAP proficiency testing programs and to additional nonsubscribing laboratories that perform IHC testing. Specific questions were designed to capture laboratory practices not addressed in a 2010 survey. RESULTS: - The analysis was based on responses from 1085 laboratories that perform IHC staining. Ninety-six percent (809 of 844) always documented validation of IHC assays. Sixty percent (648 of 1078) had separate procedures for predictive and nonpredictive markers, 42.7% (220 of 515) had procedures for laboratory-developed tests, 50% (349 of 697) had procedures for testing cytologic specimens, and 46.2% (363 of 785) had procedures for testing decalcified specimens. Minimum case numbers were specified by 85.9% (720 of 838) of laboratories for nonpredictive markers and 76% (584 of 768) for predictive markers. Median concordance requirements were 95% for both types. For initial validation, 75.4% (538 of 714) of laboratories adopted the 20-case minimum for nonpredictive markers and 45.9% (266 of 579) adopted the 40-case minimum for predictive markers as outlined in the 2014 LPG. The most common method for validation was correlation with morphology and expected results. Laboratories also reported which assay changes necessitated revalidation and their minimum case requirements. CONCLUSIONS: - Benchmark data on current IHC validation practices and procedures may help laboratories understand the issues and influence further refinement of LPG recommendations.

Analytic Validation of Immunohistochemical Assays: A Comparison of Laboratory Practices Before and After Introduction of an Evidence-Based Guideline.

CONTEXT: - Laboratories must demonstrate analytic validity before any test can be used clinically, but studies have shown inconsistent practices in immunohistochemical assay validation. OBJECTIVE: - To assess changes in immunohistochemistry analytic validation practices after publication of an evidence-based laboratory practice guideline. DESIGN: - A survey on current immunohistochemistry assay validation practices and on the awareness and adoption of a recently published guideline was sent to subscribers enrolled in one of 3 relevant College of American Pathologists proficiency testing programs and to additional nonsubscribing laboratories that perform immunohistochemical testing. The results were compared with an earlier survey of validation practices. RESULTS: - Analysis was based on responses from 1085 laboratories that perform immunohistochemical staining. Of 1057 responses, 65.4% (691) were aware of the guideline recommendations before this survey was sent and 79.9% (550 of 688) of those have already adopted some or all of the recommendations. Compared with the 2010 survey, a significant number of laboratories now have written validation procedures for both predictive and nonpredictive marker assays and specifications for the minimum numbers of cases needed for validation. There was also significant improvement in compliance with validation requirements, with 99% (100 of 102) having validated their most recently introduced predictive marker assay, compared with 74.9% (326 of 435) in 2010. The difficulty in finding validation cases for rare antigens and resource limitations were cited as the biggest challenges in implementing the guideline. CONCLUSIONS: - Dissemination of the 2014 evidence-based guideline validation practices had a positive impact on laboratory performance; some or all of the recommendations have been adopted by nearly 80% of respondents.

Evaluation of Testing of Acute Leukemia Samples: Survey Result From the College of American Pathologists.

CONTEXT: - The classification and prognosis determination in acute leukemia (AL) are complex and it is unclear what testing is being performed in practice. OBJECTIVE: - To survey physicians describing their current practice of test ordering in the diagnosis of AL. DESIGN: - In anticipation of a guideline by the College of American Pathologists (CAP) and the American Society for Hematology on laboratory testing needed for the initial workup of AL, a baseline survey was designed by an expert panel from CAP. Members of professional societies were asked to describe their current practice of test ordering. RESULTS: - Two hundred ninety-four responses were received with 258 respondents analyzed after the first qualifying survey question regarding initial diagnosis of AL. One hundred seventy-six of 249 respondents (70.7%) were board-certified hematopathologists. Flow cytometry and karyotype analysis were routinely performed for acute myeloid leukemia (AML) (99.1% [232 of 234] and 96.2% [225 of 234], respectively) and acute lymphoblastic leukemia (ALL) (98.3% [229 of 233] and 96.6% [225 of 233], respectively). In addition, fluorescence in situ hybridization studies were routinely performed by 81.2% (190 of 234) of respondents for AML and 85.0% (198 of 233) of respondents for ALL; other molecular studies were performed by 78.2% (183) for AML and 54.9% (128) for ALL; immunohistochemistry by 44.9% (105) for AML and 47.6% (111) for ALL; and cytochemistry by 24.8% (58) for AML and 14.2% (33) for ALL. CONCLUSIONS: - While flow cytometry and karyotyping are routinely reported as being performed for the diagnosis of AL, there is marked variation in the reporting of testing patterns for other genetic studies, immunohistochemistry, and cytochemistry.

Interpretive Diagnostic Error Reduction in Surgical Pathology and Cytology: Guideline From the College of American Pathologists Pathology and Laboratory Quality Center and the Association of Directors of Anatomic and Surgical Pathology.

CONTEXT: Additional reviews of diagnostic surgical and cytology cases have been shown to detect diagnostic discrepancies. OBJECTIVE: To develop, through a systematic review of the literature, recommendations for the review of pathology cases to detect or prevent interpretive diagnostic errors. DESIGN: The College of American Pathologists Pathology and Laboratory Quality Center in association with the Association of Directors of Anatomic and Surgical Pathology convened an expert panel to develop an evidence-based guideline to help define the role of case reviews in surgical pathology and cytology. A literature search was conducted to gather data on the review of cases in surgical pathology and cytology. RESULTS: The panel drafted 5 recommendations, with strong agreement from open comment period participants ranging from 87% to 93%. The recommendations are: (1) anatomic pathologists should develop procedures for the review of selected pathology cases to detect disagreements and potential interpretive errors; (2) anatomic pathologists should perform case reviews in a timely manner to avoid having a negative impact on patient care; (3) anatomic pathologists should have documented case review procedures that are relevant to their practice setting; (4) anatomic pathologists should continuously monitor and document the results of case reviews; and (5) if pathology case reviews show poor agreement within a defined case type, anatomic pathologists should take steps to improve agreement. CONCLUSIONS: Evidence exists that case reviews detect errors; therefore, the expert panel recommends that anatomic pathologists develop procedures for the review of pathology cases to detect disagreements and potential interpretive errors, in order to improve the quality of patient care.

Immunohistochemistry practices of cytopathology laboratories: a survey of participants in the College of American Pathologists Nongynecologic Cytopathology Education Program.

CONTEXT: Immunohistochemistry (IHC) is important for cytology but poses special challenges because preanalytic conditions may differ from the conditions of IHC-positive controls. OBJECTIVE: To broadly survey cytology laboratories to quantify preanalytic platforms for cytology IHC and identify problems with particular platforms or antigens. To discover how validation guidelines for HER2 testing have affected cytology. DESIGN: A voluntary survey of cytology IHC practices was sent to 1899 cytology laboratories participating in the College of American Pathologists Nongynecologic Cytopathology Education Program in the fall of 2009. RESULTS: A total of 818 laboratories (43%) responded to the survey by April 2010. Three hundred fourty-five of 791 respondents (44%) performed IHC on cytology specimens. Seventeen different fixation and processing platforms prior to antibody reaction were reported. A total of 59.2% of laboratories reported differences between the platforms for cytology specimens and positive controls, but most (155 of 184; 84%) did not alter antibody dilutions or antigen retrieval for cytology IHC. When asked to name 2 antibodies for which staining conditions differed between cytology and surgical samples, there were 18 responses listing 14 antibodies. A total of 30.6% of laboratories performing IHC offered HER2 testing before publication of the 2007 College of American Pathologists/American Society of Clinical Oncologists guidelines, compared with 33.6% afterward, with increased performance of testing by reference laboratories. Three laboratories validated a nonformalin HER2 platform. CONCLUSIONS: The platforms for cytology IHC and positive controls differ for most laboratories, yet conditions are uncommonly adjusted for cytology specimens. Except for the unsuitability of air-dried smears for HER2 testing, the survey did not reveal evidence of systematic problems with any antibody or platform.

Principles of analytic validation of immunohistochemical assays: Guideline from the College of American Pathologists Pathology and Laboratory Quality Center.

CONTEXT: Laboratories must validate all assays before they can be used to test patient specimens, but currently there are no evidence-based guidelines regarding validation of immunohistochemical assays. OBJECTIVE: To develop recommendations for initial analytic validation and revalidation of immunohistochemical assays. DESIGN: The College of American Pathologists Pathology and Laboratory Quality Center convened a panel of pathologists and histotechnologists with expertise in immunohistochemistry to develop validation recommendations. A systematic evidence review was conducted to address key questions. Electronic searches identified 1463 publications, of which 126 met inclusion criteria and were extracted. Individual publications were graded for quality, and the key question findings for strength of evidence. Recommendations were derived from strength of evidence, open comment feedback, and expert panel consensus. RESULTS: Fourteen guideline statements were established to help pathology laboratories comply with validation and revalidation requirements for immunohistochemical assays. CONCLUSIONS: Laboratories must document successful analytic validation of all immunohistochemical tests before applying to patient specimens. The parameters for cases included in validation sets, including number, expression levels, fixative and processing methods, should take into account intended use and should be sufficient to ensure that the test accurately measures the analyte of interest in specimens tested in that laboratory. Recommendations are also provided for confirming assay performance when there are changes in test methods, reagents, or equipment.

Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center.

CONTEXT: There is increasing interest in using whole slide imaging (WSI) for diagnostic purposes (primary and/or consultation). An important consideration is whether WSI can safely replace conventional light microscopy as the method by which pathologists review histologic sections, cytology slides, and/or hematology slides to render diagnoses. Validation of WSI is crucial to ensure that diagnostic performance based on digitized slides is at least equivalent to that of glass slides and light microscopy. Currently, there are no standard guidelines regarding validation of WSI for diagnostic use. OBJECTIVE: To recommend validation requirements for WSI systems to be used for diagnostic purposes. DESIGN: The College of American Pathologists Pathology and Laboratory Quality Center convened a nonvendor panel from North America with expertise in digital pathology to develop these validation recommendations. A literature review was performed in which 767 international publications that met search term requirements were identified. Studies outside the scope of this effort and those related solely to technical elements, education, and image analysis were excluded. A total of 27 publications were graded and underwent data extraction for evidence evaluation. Recommendations were derived from the strength of evidence determined from 23 of these published studies, open comment feedback, and expert panel consensus. RESULTS: Twelve guideline statements were established to help pathology laboratories validate their own WSI systems intended for clinical use. Validation of the entire WSI system, involving pathologists trained to use the system, should be performed in a manner that emulates the laboratory's actual clinical environment. It is recommended that such a validation study include at least 60 routine cases per application, comparing intraobserver diagnostic concordance between digitized and glass slides viewed at least 2 weeks apart. It is important that the validation process confirm that all material present on a glass slide to be scanned is included in the digital image. CONCLUSIONS: Validation should demonstrate that the WSI system under review produces acceptable digital slides for diagnostic interpretation. The intention of validating WSI systems is to permit the clinical use of this technology in a manner that does not compromise patient care.

Consensus statement on effective communication of urgent diagnoses and significant, unexpected diagnoses in surgical pathology and cytopathology from the College of American Pathologists and Association of Directors of Anatomic and Surgical Pathology.

CONTEXT: Recognizing the difficulty in applying the concept of critical values to anatomic pathology diagnoses, the College of American Pathologists and the Association of Directors of Anatomic and Surgical Pathology have chosen to reevaluate the concept of critical diagnoses. OBJECTIVE: To promote effective communication of urgent and significant, unexpected diagnoses in surgical pathology and cytology. DESIGN: A comprehensive literature search was conducted and reviewed by an expert panel. RESULTS: A policy of effective communication of important results in surgical pathology and cytology is desirable to enhance patient safety and to address multiple regulatory requirements. CONCLUSIONS: Each institution should create its own policy regarding urgent diagnoses and significant, unexpected diagnoses in anatomic pathology. This policy should be separate from critical results or panic-value policies in clinical pathology, with the expectation of a different time frame for communication. Urgent diagnosis is defined as a medical condition that, in most cases, should be addressed as soon as possible. Significant, unexpected diagnosis is defined as a medical condition that is clinically unusual or unforeseen and should be addressed at some point in the patient's course. Further details of this statement are provided.

Gynecologic cytology proficiency testing failures: what have we learned?: observations from the college of american pathologists gynecologic cytology proficiency testing program.

CONTEXT: In 2006, the first gynecologic cytology proficiency tests were offered by the College of American Pathologists. Four years of data are now available using field-validated slides, including conventional and liquid-based Papanicolaou tests. OBJECTIVE: To characterize the pattern of error types that resulted in initial proficiency-test failure for cytotechnologists, primary screening pathologists, and secondary pathologists (those whose slides are prescreened by cytotechnologists). DESIGN: The results of 37 029 initial College of American Pathologists Papanicolaou proficiency tests were reviewed from 4 slide-set modules: conventional, ThinPrep, SurePath, or a module containing all 3 slide types. RESULTS: During this 4-year period, cytotechnologists were least likely to fail the initial test (3.4%; 614 of 18 264), followed by secondary pathologists (ie, those reviewing slides already screened by a cytotechnologist) with a failure rate of 4.2% (728 of 17 346), and primary pathologists (ie, those screening their own slides) having the highest level of failure (13.7%; 194 of 1419). Failure rates have fallen for all 3 groups over time. Pathologists are graded more stringently on proficiency tests, and more primary pathologists would have passed if they had been graded as cytotechnologists. There were no significant differences among performances using different types of slide sets. False-positive errors were common for both primary (63.9%; 124 of 194 errors) and secondary (55.6%; 405 of 728 errors) pathologists, whereas automatic failures were most common for cytotechnologists (75.7%; 465 of 614 errors). CONCLUSIONS: The failure rate is decreasing for all participants. The failures for primary pathologist screeners are due to false-positive responses. Primary screening cytotechnologists and secondary pathologists have automatic failures more often than do primary screening pathologists.

Granulomatous inflammation-an underestimated cause of false-positive diagnoses in lung fine-needle aspirates: observations from the college of american pathologists nongynecologic cytopathology interlaboratory comparison program.

CONTEXT: The false-positive rate for fine-needle aspirates of the lung has been cited as less than 1% for granulomatous inflammation, comprising one of the known causes of false-positive diagnoses. OBJECTIVE: To determine the rate of false-positive diagnoses of granulomatous inflammation for lung fine-needle aspirates by assessing the false-positive response rate in the context of the College of American Pathologists Nongynecologic Cytopathology Interlaboratory Comparison Program. DESIGN: We performed a retrospective review of 1092 participant responses for lung fine-needle aspirate challenges with the reference diagnosis of specific infections/granulomatous inflammation from 1998 to 2008 from the College of American Pathologists Nongynecologic Cytopathology Interlaboratory Comparison Program. False-positive rates by participant type (pathologist versus cytotechnologist), general diagnosis category, reference diagnosis, and preparation type were analyzed for the pathologists' responses. RESULTS: Of the 502 general category responses for pathologists, 428 (85.3%) were benign, 55 (11%) were malignant, and 19 (3.8%) were suspicious. There was no difference in the false-positive rate between preparations (P  =  .76) or participants (P  =  .39). Of those responses by pathologists that were benign, only 68.7% (292 of 425) were an exact match to granulomatous inflammation. Non-small cell carcinoma, adenocarcinoma, and squamous carcinoma represented 64% of false-positive/suspicious responses, while small cell carcinoma and carcinoid comprised 13%. CONCLUSION: In an interlaboratory comparison program, granulomatous inflammation represents an important cause of false-positive/suspicious responses in lung fine-needle aspirates (14.8%) and is much higher than false-positive rates reported historically in clinical studies. These results highlight the importance of granulomatous inflammation as a mimic of carcinoma.

Practices of participants in the college of american pathologists interlaboratory comparison program in cervicovaginal cytology, 2006.

CONTEXT: Liquid-based preparations (LBPs) and human papillomavirus testing have led to changes in cervical cytology practices. The College of American Pathologists attempts to track practice patterns using a supplemental questionnaire, which allows laboratories to report diagnostic practices. OBJECTIVE: To analyze the 2006 reporting practices and to compare the results with the 2003 survey data. DESIGN: Questionnaire was mailed to 1621 laboratories. Participants included laboratories enrolled in the 2006 College of American Pathologists Gynecologic Proficiency Testing Program or the educational Interlaboratory Comparison Program in Gynecologic Cytology. RESULTS: Of the 679 responding laboratories (response rate, 42%), most (97.8%; n = 664) had implemented the Bethesda 2001 terminology. The median rate for all preparations with low-grade squamous intraepithelial lesions was 2.5% (2.9% for LBPs) compared with a 2003 median rate of 2.1%; the increase was confined to LBPs. Rates for high-grade squamous intraepithelial lesions (median, 0.5%) and atypical squamous cells have changed little. High-grade squamous intraepithelial lesions and unsatisfactory rates varied at statistically significant levels between types of LBPs. Most atypical squamous cell cases were subclassified as undetermined significance (median, 4.3%). The median ratio of atypical squamous cells to squamous intraepithelial lesions and carcinomas for all specimen types combined was 1.5, similar to the 2003 median ratio of 1.4. The median rates for findings of squamous cell abnormalities for 2006 were significantly higher for LBPs than for conventional smears. CONCLUSIONS: Most responding laboratories have implemented the Bethesda 2001 terminology. There is an increase in LBP low-grade squamous intraepithelial lesion rates when compared with 2003 data. Liquid-based preparations have higher median squamous intraepithelial lesion and atypical squamous cell rates.

Unsatisfactory reporting rates: 2006 practices of participants in the college of american pathologists interlaboratory comparison program in gynecologic cytology.

CONTEXT: -Minimum cellular criteria for satisfactory Papanicolaou tests were established with the Bethesda System in 2001, and unsatisfactory rates are used as a quality-reporting measure. OBJECTIVE: -To evaluate practices and unsatisfactory rates from laboratories responding to the 2007 College of American Pathologists supplemental questionnaire survey. DESIGN: -In 2007, a supplemental questionnaire was mailed to 1621 laboratories enrolled in the 2006 College of American Pathologists Interlaboratory Comparison Program in Gynecologic Cytology (PAP Education), requesting data from the 2006 calendar year. Unsatisfactory rates, reasons for unsatisfactory specimens, laboratory size, and specimen preparation type were analyzed. RESULTS: -A total of 42% of the laboratories responded to the survey. Most of those laboratories (637 of 674; 94.5%) used the Bethesda System minimum cellularity criteria. Of those laboratories responding, 79% (527 of 667) used the Bethesda System criteria for atrophic or postirradiation specimens. Unsatisfactory rates have increased since 1996. SurePath preparations were associated with the lowest unsatisfactory rate (50th percentile, 0.30; 95th percentile, 1.3), conventional Papanicolaou tests had the highest 95th percentile rates (50th percentile, 1.0; 95th percentile, 5.90), and ThinPrep specimens had the highest median percentile (50th percentile, 1.1; 95th percentile, 3.4). The most-common reason for unsatisfactory Papanicolaou tests was too few squamous cells. Air-drying artifact was the least-common reason for unsatisfactory reporting for liquid-based preparations. CONCLUSIONS: -Use of the Bethesda System criteria for unsatisfactory specimens is widespread. Unsatisfactory rates have increased since 1996; however, the median rates are 1.1% or less for all preparations. Results from the College of American Pathologists PAP Education supplemental questionnaire continue to provide valuable benchmarking data for cytologic quality-improvement programs in laboratories.