Interaction in the segmentation of medical images: a survey.

Segmentation of the object of interest is a difficult step in the analysis of digital images. Fully automatic methods sometimes fail, producing incorrect results and requiring the intervention of a human operator. This is often true in medical applications, where image segmentation is particularly difficult due to restrictions imposed by image acquisition, pathology and biological variation. In this paper we present an early review of the largely unknown territory of human-computer interaction in image segmentation. The purpose is to identify patterns in the use of interaction and to develop qualitative criteria to evaluate interactive segmentation methods. We discuss existing interactive methods with respect to the following aspects: the type of information provided by the user, how this information affects the computational part, and the purpose of interaction in the segmentation process. The discussion is based on the potential impact of each strategy on the accuracy, repeatability and interaction efficiency. Among others, these are important aspects to characterise and understand the implications of interaction to the results generated by an interactive segmentation method. This survey is focused on medical imaging, however similar patterns are expected to hold for other applications as well.

Velocity estimation of spots in three-dimensional confocal image sequences of living cells.

BACKGROUND: The analysis of three-dimensional (3D) motion is becoming increasingly important in life cell imaging. A simple description of sometimes complex patterns of movement in living cells gives insight in the underlying mechanisms governing these movements. METHODS: We evaluate a velocity estimation method based on intensity derivatives in spatial and temporal domain from 3D confocal images of living cells. Cells of the sample contain intense spots throughout the cell nucleus. In simulations, we model these spots as Gaussian intensity profiles which are constant in intensity and shape. To quantify the quality of the estimated velocity, we introduce a reliability measure. RESULTS: For constant linear velocity, the velocity estimation is unbiased. For accelerated motion paths or when a neighboring spot disturbs the intensity profile, the method results are biased. The influence of the point-spread function on the velocity estimation can be compensated for by introducing anisotropic derivative kernels. The insight gained in the simulations is confirmed by the results of the method applied on an image sequence of a living cell with fluorescently labeled chromatin. CONCLUSIONS: With the velocity estimation method, a tool for estimating 3D velocity fields is described which is successfully applied to a living cell sequence. With the estimated velocity fields, motion patterns can be observed, which are a useful starting point for the analysis of dynamic processes in living cells.

Chromosome no. 1 of Crepis capillaris shows defined 3D-shapes in mitotic prophase.

The shape of mitotic prophase chromosomes has been studied in root tip nuclei by confocal microscopy and 3D-image analysis. Crepis capillaris chromosome no. 1 was used as a test object. Chromosome conformation was studied in early, mid- and in late prophase. In mid- and late prophase, individual chromosomes could be distinguished on the basis of their length. Early prophase chromosomes could not be distinguished as individuals. The central axes of prophase chromosomes were traced with an automated computer procedure and then represented as a string of 3D coordinates. This representation facilitated measurement along the chromosome axis of shape parameters such as curvature (amount of bending), torsion (helical winding) and torsion sign (helical handedness). Stretches of early prophase chromosomes showed full helical turns, which could be left- or right-handed. In the later prophase stages curvature and torsion were statistically analysed. Our data on 40 midprophase chromosomes no. 1 show that they are still highly curved, but full helical turns were no longer found. Instead, an overall meandering pattern was observed. In late prophase, one central loop persisted, flanked by two preferential regions of high curvature.

Robust autofocusing in microscopy.

BACKGROUND: A critical step in automatic microscopy is focusing. This report describes a robust and fast autofocus approach useful for a wide range of microscopic modalities and preparations. METHODS: The focus curve is measured over the complete focal range, reducing the chance that the best focus position is determined by dust or optical artifacts. Convolution with the derivative of a Gaussian smoothing function reduces the effect of noise on the focus curve. The influence of mechanical tolerance is accounted for. RESULTS: The method is shown to be robust in fluorescence, bright-field and phase contrast microscopy, in fixed and living cells, as well as in fixed tissue. The algorithm was able to focus accurately within 2 or 3 s, even under extremely noisy and low contrast imaging conditions. CONCLUSIONS: The proposed method is generally applicable in light microscopy, whenever the image information content is sufficient. The reliability of the autofocus method allows for unattended operation on a large scale.

PicToSeek: combining color and shape invariant features for image retrieval.

We aim at combining color and shape invariants for indexing and retrieving images. To this end, color models are proposed independent of the object geometry, object pose, and illumination. From these color models, color invariant edges are derived from which shape invariant features are computed. Computational methods are described to combine the color and shape invariants into a unified high-dimensional invariant feature set for discriminatory object retrieval. Experiments have been conducted on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. From the theoretical and experimental results it is concluded that object retrieval based on composite color and shape invariant features provides excellent retrieval accuracy. Object retrieval based on color invariants provides very high retrieval accuracy whereas object retrieval based entirely on shape invariants yields poor discriminative power. Furthermore, the image retrieval scheme is highly robust to partial occlusion, object clutter and a change in the object's pose. Finally, the image retrieval scheme is integrated into the PicToSeek system on-line at http://www.wins.uva.nl/research/isis/PicToSeek/ for searching images on the World Wide Web.

Segmentation of tissue architecture by distance graph matching.

BACKGROUND: Characterization of tissues can be based on the topographical relationship between the cells. Such characterization should be insensitive to distortions intrinsic to the acquisition of biological preparation. In this paper, a method for the robust segmentation of tissues based on the spatial distribution of cells is proposed. MATERIALS AND METHODS: The neighborhood of each cell in the tissue is modeled by the distances to the surrounding cells. Comparison with an example or prototype neighborhood reveals topographical similarity between tissue and prototype. Processing of all cells in the tissue extracts the regions with tissue architecture similar to the given example. RESULTS: Comparison with other topographical-segmentation methods shows that the proposed method is better suited for partitioning tissue architecture. As an example, the quantification of the structural integrity in rat hippocampi after ischemia is demonstrated. In contrast to other methods, the algorithm correlates well with expert evaluation. CONCLUSIONS: The present method reduces the nonbiological variation in the analysis of tissue sections and thus improves confidence in the result. The method can be applied to any field where regular patterns have to be detected, as long as the directional distribution of neighbors may be neglected.

Randomness of spatial distributions of two proteins in the cell nucleus involved in mRNA synthesis and their relationship.

Randomness and relation between point positions play an important role in archeology, cosmology, geography, and biology. An often-discarded effect is the edge effect, the effect that points are bound to a certain region. Without an appropriate correction, the outcome will be wrong. We studied the problem of randomness by comparing the distribution of interpoint distances with what can be expected for randomly distributed points (pair correlation function in statistics), and applied this to two sets of nuclear proteins inside the cell nucleus. The technique comprised labelling the proteins with a fluorescent dye, recording the fluorescent distribution with a 3D confocal microscope, and detecting the positions of the individual fluorescent spots. Results showed that, apart from studying randomness, the method is well equipped to quantitatively analyze a spot detection procedure, as the resolving power and the subvoxel accuracy were clearly visible. Given the results of assessing the randomness of the general transcription factor BRG1 and the RNA synthesizing protein RNA polymerase II (polII) in the cell nucleus, we concluded that the high intensity spots of the BRG1 protein are regularly spaced. The low intensity spots of the BRG1 protein and the low- and high-intensity spots of the polII protein showed more random behavior. The BRG1 and polII proteins showed correlation; unexpectedly, the relation was also found for the low-intensity spots, which were expected to have a more random behavior.

A 3-D model for chromatin organisation of G1 and G2 populations from quantitative confocal image analysis.

A study on the chromatin organisation of synchronised G1 and G2 populations of maize root cell nuclei is reported using 3-D images acquired with a confocal fluorescence microscope. The analysis is based on the concept of accessibility. Accessibility of a position x is the effort to arrive at x, when choosing the minimum effort path to arrive at x from the nuclear border. The effort is then taken to be proportional to the amount of all mass encountered on the path, and computed by a technique called the grey valued distance transform. The approach relies heavily on quantitative analysis of the intensity information. Hence, considerable attention was paid to the quantitative modification of the confocal intensity values by diffraction, absorption and scatter corrections. Three texture features are extracted from the accessibility maps: the global object inaccessibility, the relative object accessibility, and the object homogeneity. On the basis of individual texture features, no distinction between the G1 and G2 populations could be established. However, the three features combined did show a clear difference with a high significance.

Method for counting mitoses by image processing in Feulgen stained breast cancer sections.

This study describes an image processing method for the assessment of the mitotic count in Feulgen-stained breast cancer sections. The segmentation procedure was optimized to eliminate 95-98% of the nonmitoses, whereas 11% of the mitoses did not survive the segmentation procedure. Contour features and optical density measurements of the remaining objects were computed to allow for classification. Twelve specimens were analyzed, nine used to serve as a training set, and three put aside for later use as independent test set. The fully automatic image processing method correctly classified 81% of the mitoses at the specimen level while inserting 30% false positives. The automatic procedure strongly correlated with the interactive counting procedure (r = 0.98). Although the fully automatic method provided satisfactory results, it is not yet suited for clinical practice. The automated method with an interactive evaluation step gave an accurate reflection of the mitotic count showing an almost perfect correlation with the results of the interactive morphometry (r = 0.998). Therefore this semiautomated method may be useful as prescreening device.

The homing cursor: a tool for three-dimensional chromosome analysis.

When studying the three-dimensional shape of prophase chromosomes (or any other tubular structure), it is useful to represent these structures as a string of three-dimensional Cartesian coordinates along the medial axis. This procedure was automated in order to limit the number of human interactions and to improve reproducibility. In this paper the design, implementation, and validation of the automated method is presented. From the data presented it can be concluded that the cursor algorithm provides an objective and therefore reproducible method to trace the medial axes of prophase chromosomes automatically. This method could allow a more extensive understanding of the (changes in) chromosome organisation throughout the cell cycle, its relation to cell function, and the complex process of chromosome condensation.

Automated selection of the most epithelium-rich areas in gynecologic tumor sections.

The paper describes an image analysis technique for automated selection of the epithelium-rich areas in standard paraffin tissue sections of ovarian and endometrial premalignancies and malignancies. Two staining procedures were evaluated, Feulgen (pararosanilin) and CAM 5.2, demonstrating the presence of cytokeratin 8 and 18; both were counterstained with naphthol yellow. The technique is based on the corresponding image processing method of automated estimation of the percentage of epithelium in interactively selected microscope fields. With the technique, one image is recorded with a filter to demonstrate where epithelium and stroma lie. This filter is chosen according to the type of staining: it is yellow (lambda = 552 nm) for Feulgen and blue (lambda = 470 nm) for anticytokeratin CAM 5.2. When stroma cannot be distinguished from lumina with the green filter or from epithelium with the blue filter, a second image is recorded from the same microscope field, with a blue filter (lambda = 420 nm) for Feulgen and a yellow filter (lambda = 576 nm) for anticytokeratin CAM 5.2. Discrimination between epithelium and stroma is based on the image contrast range and the packing of nuclei in the yellow image and on the automated classification of the gray value histogram peaks in the blue image. For Feulgen stain the method was evaluated on 30 ovarian tumors of the common epithelial types (8 borderline tumors and 22 carcinomas with various degrees of differentiation) and 30 endometrial carcinomas of different grades.(ABSTRACT TRUNCATED AT 250 WORDS)

DNA measurement errors with a scanning microdensitometer in cytologic and histologic samples of breast cancers.

The influence of various DNA measurement errors using a commercially available scanning microdensitometer was evaluated on Feulgen-stained cytologic and histologic samples prepared from paraffin blocks containing invasive ductal breast cancers. The overall average total measurement error was 5.5% for the cytologic specimens and 10.9% for the 4 micron histologic sections. Components of the error included microscopic adjustment variation and focussing errors (3.5% and 1.1%, respectively, for both cytologic and histologic samples) and background intensity estimation errors (3.0% for the cytologic samples and 10.0% for the histologic samples). Measurements of the integrated optical density had a minimal error of 0.5% and an average error of 1.0%. Limitations due to the histologic architecture and/or heterogeneous cell population gave rise to large differences in the selection of nuclei when differently sized scanning masks were used. To improve the reproducibility, masks used should be based on the individual cell size, and background intensity values should be carefully estimated in the vicinity of the selected cells. Overall, the cytologic tumor samples were preferable to the histologic samples for static DNA measurements. It was easier to select cells suitable for measurement in the cytologic samples, and the cytologic measurements were less time consuming and produced a smaller measurement error.

Reasoning in uncertainties. An analysis of five strategies and their suitability in pathology.

In reasoning systems, uncertainty plays a crucial part, especially for those fields in which judgements are essential, as in pathology. Uncertainty has several aspects, such as prevalence of diseases, occurrence of findings and the sensitivity and predictive value of findings. For the functioning of a reasoning system, two aspects are crucial: (1) the internal representation of the uncertainty and (2) the way in which the uncertainty is propagated in the reasoning process when combining formal statements. Five well-known reasoning strategies (Bayes' probability theory, MYCIN's certainty factor model, fuzzy set theory, the theory of Dempster-Shafer and Pathfinder's scoring mechanism) are compared, with particular attention to: (1) Under what conditions will the model function? In particular, what information is to be specified a priori to the system? (2) Can the different aspects of uncertainty be dealt with as separate entities? (3) How are unknown uncertainties dealt with? (4) How is evidence in favor of a hypothesis combined with evidence against it? (5) How does the model treat the simultaneous occurrence of more than one disorder, that is, how does the model support reasoning with compound hypotheses? It is preliminarily concluded that the different aspects of uncertainty are expressed as separate entities only in Pathfinder and probability theory. Hence, the other models do not accurately represent uncertain knowledge. Also, such theoretically attractive models as the Bayes, MYCIN and Dempster-Shafer theory can only function properly under the tight condition of mutual exclusiveness of hypotheses, which is not always suited for broader areas of pathology. They may, however, be suited for smaller areas, with a limited number of defined diseases and a limited number of features. All models but the Bayes model lack a predictable performance since there is no (or only a partial) underlying theory to guarantee minimization of the overall error.

Automated estimation of epithelial volume in breast cancer sections. A comparison with the image processing steps applied to gynecologic tumors.

The paper describes an image analysis technique for automated estimation of the epithelial percentage in standard paraffin tissue sections of invasive ductal breast cancers. Two staining procedures are evaluated: Feulgen (pararosanilin) and CAM 5.2-demonstrating the presence of cytokeratin 8 and 18-, both counterstained with naphthol yellow. In the technique, one image is recorded with a filter to visualize where the epithelium lies. This filter is chosen corresponding to the type of staining: it is yellow for Feulgen and blue for anti-cytokeratin CAM 5.2. To visualize where the stroma lies, the same image can be used for anti-cytokeratin CAM 5.2, whereas for Feulgen, a second image has to be recorded from the same microscope field with a blue filter. The image processing steps to determine the total tissue area comprise correction for shading, segmentation of the tissue area, and restoration of the segmented image by removal of small artefacts and closure of small tears in the tissue. The method for determination of the epithelial area consists of the following steps: correction for shading, gaussian blurring, segmentation of nuclei or epithelial cells, and editing of the segmented image by removal of small objects and closure of small spaces between the epithelial nuclei or cells. These image processing steps are compared to those for quantification of the epithelial percentage in gynecologic tumors of epithelial origin. For the Feulgen stain, the method is evaluated on 30 breast cancers of the ductal type (4 grade I, 12 grade II, and 14 grade III).(ABSTRACT TRUNCATED AT 250 WORDS)

Evaluation of a Diagnostic Encyclopedia Workstation for ovarian pathology.

The Diagnostic Encyclopedia Workstation (DEW) is a computer system that provides completely integrated pictorial and textual information as reference knowledge in the field of ovarian pathology. The textual component comprises information per diagnosis such as descriptions of macroscopic and microscopic images, clinical signs, and prognosis. In addition, the system offers lists of differential diagnoses and criteria to differentiate among lists of differential diagnoses and criteria to differentiate among them. The present study evaluates to what extent the system influences the diagnostic process in efficiency and outcome. Therefore, two groups of six pathologists each, covering a wide spectrum of experience in ovarian pathology, participated in the evaluation of the DEW. The quality of the resulting diagnoses was statistically analyzed with the Wilcoxon rank sum test with respect to five different viewpoints: classification, morphology, clinical consequences, duration of diagnostic process, and consensus among the participants. The results are discussed and it is concluded that classification and morphology showed better results when books were used. The evaluation experiment was, however, very rigid and negatively biased with respect to the DEW system. Positive aspects of the encyclopedia are the easy access to diagnostic and differential diagnostic information and the large set of illustrations. Insight is acquired with respect to existing bottlenecks and how they may be overcome.

A method for the acquisition of formalized knowledge in pathology.

A tool is introduced for the acquisition of pathology knowledge in a formalized form, directly by the expert. Formalization of the knowledge is intended to make descriptive pathology knowledge more suitable for computerized diagnostic support since a formal representation of knowledge allows more extensive indexing, hence more flexible access. The knowledge acquisition (KA) tool also provides a useful research instrument to investigate to what extent pathology knowledge can be made explicit, to what degree ambiguity is present, in what way experts differ when formalizing knowledge, and whether it is feasible to incrementally acquire decision criteria on the basis of the formalized descriptive knowledge. Crucial in the design of the KA tool is the incorporated meta-knowledge, which is reflected by the knowledge-base structure and is used to elicit knowledge from the expert. Knowledge is acquired from the expert via a menu-driven user interface, which follows the general steps of the pathologist when describing a case. The paper discusses the considerations underlying the design, the implementation of the KA tool, and the research goals.

Design of the diagnostic encyclopedia workstation (DEW).

The Diagnostic Encyclopedia Workstation (DEW) contains reference knowledge for diagnostic support in pathology. Illustrations are accessible via a video disc device. DEW can hold more knowledge, pictures and case histories than books, and its information is accessible via several entries. Software for data entry has been written in MUMPS with use of the relational database toolkit AIDA, which is particularly suited for manipulation of free text. The graphical mouse-driven user interface is written in C using MetaWindows. The DEW contains 85 diagnoses in ovarian pathology, covering all frequent cases and many rarities, illustrated by approximately 3000 pictures, divided among 158 cases.

A comparative study in morphometric grading of transitional cell carcinoma of the urinary bladder.

To overcome the considerable observer inconsistency in the histologic grading of transitional cell carcinomas, the value of four different morphometric grading methods was investigated in 61 tumors of the bladder. Only two methods showed satisfactory reproducibility. Both methods, one based on random nuclear sampling and the other on selective nuclear sampling, showed an increase in the mean and standard deviation of the nuclear area with higher tumor grades (P less than .00001). Morphometric classification of the learning set (44 cases) was in agreement with the unequivocally assessed histologic grade in 35 cases (79.5%) using random sampling and in 38 cases (86.4%) using selective sampling. By reducing the grading classes to "low" (grades 1 and 2) and "high" (grade 3) and by introducing a classification probability threshold (0.80), an accurate morphometric classification was achieved in 38 cases (86.4%) using random sampling and in 41 cases (93.2%) using selective sampling. Of the 17 cases with histologic grading discrepancies, all 10 low-grade tumors (with discrepancies of grade 1 versus grade 2) were correctly classified as low-grade carcinomas by both of the morphometric methods; in the remaining 7 cases, with low-versus-high discrepancies (grade 2 versus grade 3), the selective method yielded better correlation with the tumor stage and clinical follow-up. It is concluded that morphometric classification is an acceptable alternative for histologic grading by pathologists, provided that the reproducibility of the method is confirmed. Although both random and selective sampling yielded satisfactory classifications, the selective method gave more reliable results as confirmed by the clinical behavior.

Construction of the diagnostic encyclopedia workstation. Computerizing pathology for the pathologist.

The combination of a personal computer and a laser-vision disc player is an adroit tool for storing and retrieving textual information in combination with pictorial information. This paper describes such a system (a "diagnostic encyclopedia workstation"), which provides information to the pathologist engaged in daily diagnostic practice. The system contains a considerable amount of descriptive textual information that might be useful in making diagnoses in histopathology, along with many illustrations; the text is presented in a natural language (English). The descriptive information is divided into 18 categories, including such topics as histology, macroscopy, immunopathology and clinical data; each topic has a separate display in the system. Also present are two types of "decision rules" for making diagnoses (confirmative criteria for a diagnosis under consideration and exclusive criteria for many other diagnoses) and a classifying structure. The present version of this system contains information on about 100 diagnoses in ovarian pathology, illustrated with about 3,000 color slides from about 140 cases. The information and pictures are immediately available. Further characteristics of this system are its flexibility, accessibility and user friendliness; it has been structured so that components of artificial intelligence may be added later.

A computer based handbook and atlas of pathology.

The Diagnostic Encyclopaedia Workstation (DEW) is a computerized handbook of pathology intended for use in diagnostic practice. It consists of a combination of a personal computer (PC), a video disc player (VDP), for which a specially developed disc is used, two monitors, a mouse and software. The hard disc of the computer contains textual information on diagnoses in categories such as macroscopy, common histology, immunopathology, clinical observations and prognosis and case histories. This information is frequently illustrated by pictures on a video disc which is automatically addressed by the computer software. All pictures, at present some 3000, pertain to case histories which are included in the system. Also integrated are classification aids in two categories: diagnostic criteria and differential diagnosis. Advantages of DEW over the use of conventional manuals are 1) the extensive volume of text, 2) the large number of high quality illustrations, 3) the immediate access to cross references and 4) the potential for continuous revision.