The dysfunctional federally mandated proficiency test in cytopathology: a statistical analysis.

Proficiency testing in cytopathology and in other disciplines should be based on firm statistical and scientific foundations, because test theory in general is a heavily statistical subject. Statistical considerations have demonstrated that the design of "short" proficiency tests in cytopathology, including the current federally mandated test, fundamentally is unsound because of the lack of sufficient validity and reliability. Examinees too frequently are misclassified by such short-format tests: Competent examinees fail the test in surprisingly high numbers, whereas most of the examinees who have insufficient cytologic skills eventually pass the test after the allowed retakes. Only dichotomous tests are suitable for accurate computation of the effects of test design on reliability, but the statistical conclusions also are generalizable to nondichotomous tests. In conclusion, the current federally mandated proficiency test cannot reliably measure the level of expertise of cytologists and, thus, cannot assure that only adequately skilled individuals evaluate Papanicolaou test samples. To render the test suitable for its intended purpose, the authors believe that complete redesign of the test, with the participation of experts in modern test theory, would be advisable.

Sir Francis Galton and proficiency testing in cytopathology.

Data from the National Cytology Proficiency Testing Update show that as of January 31, 2006, 9% of 12,786 examinees failed the test on the first attempt. For the second attempt, the failure rate among those who had initially failed remained surprisingly similar, 10%, although common sense would dictate that it should be much higher among those who have already failed the test once and should have lower professional skills. What is the reason for this remarkable improvement in performance? There is a simple explanation: this is a statistical phenomenon, known as "regression toward the mean." Two groups of examinees earn failing scores during proficiency testing: those whose skills are really insufficient and those who are competent but who achieved lower scores due to random variation in the test results. The latter, "misclassified" examinees subsequently "regress" toward the mean during the second test; that is, their test results become more commensurate with their genuine skills. Since the failure rates of all participants during the first and second attempts were similar, we must assume that the majority of the examinees who failed on the first attempt fall into the second, misclassified group, and only a minority have truly insufficient skills.

Review of Penner's criteria for proficiency testing in cytopathology: implications for the redesign of the federally mandated proficiency testing in cytopathology.

Thirty-three years ago Penner advocated six criteria for the performance of proficiency testing in cytopathology (PTC). Since that time, several further requirements have been added by other authors. The present article critically evaluates and modifies the original criteria and adds two more principles, validity and reliability, that we recognize as crucially important in the performance of PTC. The revised criteria should be taken into consideration in the planned technical redesign of the nationwide PTC.

Cytopathology proficiency testing: where do we go from here?

With the recent introduction of nationwide proficiency testing in cytopathology (PTC), reconsideration of several aspects of this controversial quality assurance method becomes justified. This paper discusses various merits and demerits of the PTC system currently prescribed by federal regulations, points out perceived deficiencies, and suggests methods for improvement.

The Pathology of Human Bladder Cancer.

The pathologist plays a key role in the detection and diagnosis of bladder cancer, as well as in the development of strategies for the clinical management of this disease by the urologist. In order to make appropriate decisions, the urologist needs help from the pathologist in determining: 1) whether or not a bladder tumor is present; 2) if present, its histologic type, grade, depth of invasion and evidence of lymphatic or blood vessel invasion; 3) whether or not there are associated neoplastic lesions elsewhere in the bladder -or elsewhere in the lower urinary tract; and 4) if some therapeutic modality has been directed toward the neoplastic tissue, the completeness of tumor removal or destruction. It is recommended that the WHO classification of bladder tumors be used by pathologists in their reports to facilitate communication among pathologists, urologists, radiation therapists, medical oncologists, and others involved in the management of bladder cancer patients. In addition, the pathologist should make a definite statement regarding the presence or absence of muscle tissue in every bladder biopsy specimen containing tumor. After one or more superficial tumors are identified by the urologist and treated by transurethral means, management of the patient will depend not only on the microscopic evaluation of the resected tumor but also on an accurate assessment of the neoplastic potential of the remaining epithelium. The cytopathologic examination of urine samples or bladder washings aids this assessment.