Overlapping Complex Concepts Have More Commission Errors, Especially in Intensive Terminology Auditing.

SNOMED CT is a large, complex and widely-used terminology. Auditing is part of the life cycle of terminologies. A review of terminologies' content can identify two error categories: commission errors, such as an incorrect parent or attribute relationship, indicating errors in a concept's modeling, and omission errors, such as missing a parent or attribute relationship, representing incomplete modeling of a concept. According to our experience, terminology curators are mostly interested in commission errors. In recent years, a long-term remodeling project has addressed modeling issues in SNOMED CT's Infectious disease and Congenital disease subhierarchies. In this longitudinal study, we investigated a posteriori the efficacy of complex concepts, called overlapping concepts, to identify commission errors during intensive auditing periods and during maintenance periods over several releases. The algorithmic implication is that when auditing resources are scarce, a methodology of auditing first, or only, the overlapping concepts will obtain a higher auditing yield.

Mining non-lattice subgraphs for detecting missing hierarchical relations and concepts in SNOMED CT.

OBJECTIVE: Quality assurance of large ontological systems such as SNOMED CT is an indispensable part of the terminology management lifecycle. We introduce a hybrid structural-lexical method for scalable and systematic discovery of missing hierarchical relations and concepts in SNOMED CT. MATERIAL AND METHODS: All non-lattice subgraphs (the structural part) in SNOMED CT are exhaustively extracted using a scalable MapReduce algorithm. Four lexical patterns (the lexical part) are identified among the extracted non-lattice subgraphs. Non-lattice subgraphs exhibiting such lexical patterns are often indicative of missing hierarchical relations or concepts. Each lexical pattern is associated with a potential specific type of error. RESULTS: Applying the structural-lexical method to SNOMED CT (September 2015 US edition), we found 6801 non-lattice subgraphs that matched these lexical patterns, of which 2046 were amenable to visual inspection. We evaluated a random sample of 100 small subgraphs, of which 59 were reviewed in detail by domain experts. All the subgraphs reviewed contained errors confirmed by the experts. The most frequent type of error was missing is-a relations due to incomplete or inconsistent modeling of the concepts. CONCLUSIONS: Our hybrid structural-lexical method is innovative and proved effective not only in detecting errors in SNOMED CT, but also in suggesting remediation for these errors.

Analyzing structural changes in SNOMED CT's Bacterial infectious diseases using a visual semantic delta.

Thousands of changes are applied to SNOMED CT's concepts during each release cycle. These changes are the result of efforts to improve or expand the coverage of health domains in the terminology. Understanding which concepts changed, how they changed, and the overall impact of a set of changes is important for editors and end users. Each SNOMED CT release comes with delta files, which identify all of the individual additions and removals of concepts and relationships. These files typically contain tens of thousands of individual entries, overwhelming users. They also do not identify the editorial processes that were applied to individual concepts and they do not capture the overall impact of a set of changes on a subhierarchy of concepts. In this paper we introduce a methodology and accompanying software tool called a SNOMED CT Visual Semantic Delta ("semantic delta" for short) to enable a comprehensive review of changes in SNOMED CT. The semantic delta displays a graphical list of editing operations that provides semantics and context to the additions and removals in the delta files. However, there may still be thousands of editing operations applied to a set of concepts. To address this issue, a semantic delta includes a visual summary of changes that affected sets of structurally and semantically similar concepts. The software tool for creating semantic deltas offers views of various granularities, allowing a user to control how much change information they view. In this tool a user can select a set of structurally and semantically similar concepts and review the editing operations that affected their modeling. The semantic delta methodology is demonstrated on SNOMED CT's Bacterial infectious disease subhierarchy, which has undergone a significant remodeling effort over the last two years.

Tracking the Remodeling of SNOMED CT's Bacterial Infectious Diseases.

SNOMED CT's content undergoes many changes from one release to the next. Over the last year SNOMED CT's Bacterial infectious disease subhierarchy has undergone significant editing to bring consistent modeling to its concepts. In this paper we analyze the stated and inferred structural modifications that affected the Bacterial infectious disease subhierarchy between the Jan 2015 and Jan 2016 SNOMED CT releases using a two-phased approach. First, we introduce a methodology for creating a human readable list of changes. Next, we utilize partial-area taxonomies, which are compact summaries of SNOMED CT's content and structure, to identify the "big picture" changes that occurred in the subhierarchy. We illustrate how partial-area taxonomies can be used to help identify groups of concepts that were affected by these editing operations and the nature of these changes. Modeling issues identified using our two-phase methodology are discussed.

Extending the coverage of phenotypes in SNOMED CT through post-coordination.

OBJECTIVES: To extend the coverage of phenotypes in SNOMED CT through post-coordination. METHODS: We identify frequent modifiers in terms from the Human Phenotype Ontology (HPO), which we associate with templates for post-coordinated expressions in SNOMED CT. RESULTS: We identified 176 modifiers, created 12 templates, and generated 1,617 post-coordinated expressions. CONCLUSIONS: Through this novel approach, we can increase the current number of mappings by 50%.

A tribal abstraction network for SNOMED CT target hierarchies without attribute relationships.

OBJECTIVE: Large and complex terminologies, such as Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT), are prone to errors and inconsistencies. Abstraction networks are compact summarizations of the content and structure of a terminology. Abstraction networks have been shown to support terminology quality assurance. In this paper, we introduce an abstraction network derivation methodology which can be applied to SNOMED CT target hierarchies whose classes are defined using only hierarchical relationships (ie, without attribute relationships) and similar description-logic-based terminologies. METHODS: We introduce the tribal abstraction network (TAN), based on the notion of a tribe-a subhierarchy rooted at a child of a hierarchy root, assuming only the existence of concepts with multiple parents. The TAN summarizes a hierarchy that does not have attribute relationships using sets of concepts, called tribal units that belong to exactly the same multiple tribes. Tribal units are further divided into refined tribal units which contain closely related concepts. A quality assurance methodology that utilizes TAN summarizations is introduced. RESULTS: A TAN is derived for the Observable entity hierarchy of SNOMED CT, summarizing its content. A TAN-based quality assurance review of the concepts of the hierarchy is performed, and erroneous concepts are shown to appear more frequently in large refined tribal units than in small refined tribal units. Furthermore, more erroneous concepts appear in large refined tribal units of more tribes than of fewer tribes. CONCLUSIONS: In this paper we introduce the TAN for summarizing SNOMED CT target hierarchies. A TAN was derived for the Observable entity hierarchy of SNOMED CT. A quality assurance methodology utilizing the TAN was introduced and demonstrated.

Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies.

OBJECTIVE: Standards terminologies may be large and complex, making their quality assurance challenging. Some terminology quality assurance (TQA) methodologies are based on abstraction networks (AbNs), compact terminology summaries. We have tested AbNs and the performance of related TQA methodologies on small terminology hierarchies. However, some standards terminologies, for example, SNOMED, are composed of very large hierarchies. Scaling AbN TQA techniques to such hierarchies poses a significant challenge. We present a scalable subject-based approach for AbN TQA. METHODS: An innovative technique is presented for scaling TQA by creating a new kind of subject-based AbN called a subtaxonomy for large hierarchies. New hypotheses about concentrations of erroneous concepts within the AbN are introduced to guide scalable TQA. RESULTS: We test the TQA methodology for a subject-based subtaxonomy for the Bleeding subhierarchy in SNOMED's large Clinical finding hierarchy. To test the error concentration hypotheses, three domain experts reviewed a sample of 300 concepts. A consensus-based evaluation identified 87 erroneous concepts. The subtaxonomy-based TQA methodology was shown to uncover statistically significantly more erroneous concepts when compared to a control sample. DISCUSSION: The scalability of TQA methodologies is a challenge for large standards systems like SNOMED. We demonstrated innovative subject-based TQA techniques by identifying groups of concepts with a higher likelihood of having errors within the subtaxonomy. Scalability is achieved by reviewing a large hierarchy by subject. CONCLUSIONS: An innovative methodology for scaling the derivation of AbNs and a TQA methodology was shown to perform successfully for the largest hierarchy of SNOMED.

Fatal musculoskeletal injuries of Quarter Horse racehorses: 314 cases (1990-2007).

OBJECTIVE: To determine major causes of death and the anatomic location of musculoskeletal injuries in Quarter Horse racehorses in California. DESIGN: Retrospective case series. ANIMALS: 314 Quarter Horse racehorses with musculoskeletal injuries that were necropsied through the California Horse Racing Board Postmortem Program from 1990 to 2007. PROCEDURES: Postmortem pathology reports were retrospectively reviewed. Musculoskeletal injuries were categorized by anatomic region and described. The number of Quarter Horse starts and starters for the same period of time were obtained from a commercial database for determination of fatal injury incidence. RESULTS: Musculoskeletal injuries accounted for 314 of the 443 (71 %) Quarter Horse racehorses that died during the 18-year study period. Fatal musculoskeletal injuries occurred at a rate of 2.0 deaths/1,000 race starts and 18.6 deaths/1,000 horses that started a race. Musculoskeletal injuries occurred predominantly during racing (84%) and in the forelimbs (81%). The most common fatal musculoskeletal injuries were metacarpophalangeal and metatarsophalangeal joint (fetlock) support injuries (40%) and carpal (24%), vertebral (10%), and scapular (8%) fractures. Proximal interphalangeal (pastern) joint luxations resulted in death of 3% of horses. Fracture configurations of some bones were consistent with those of Thoroughbred racehorses. Evidence of preexisting stress remodeling of bone was reported for some fractures. CONCLUSIONS AND CLINICAL RELEVANCE: Knowledge of common locations and types of fatal musculoskeletal injuries in racing Quarter Horses may enhance practitioners' ability to detect mild injuries early, rest horses, and help prevent catastrophic injuries.

Auditing complex concepts of SNOMED using a refined hierarchical abstraction network.

Auditors of a large terminology, such as SNOMED CT, face a daunting challenge. To aid them in their efforts, it is essential to devise techniques that can automatically identify concepts warranting special attention. "Complex" concepts, which by their very nature are more difficult to model, fall neatly into this category. A special kind of grouping, called a partial-area, is utilized in the characterization of complex concepts. In particular, the complex concepts that are the focus of this work are those appearing in intersections of multiple partial-areas and are thus referred to as overlapping concepts. In a companion paper, an automatic methodology for identifying and partitioning the entire collection of overlapping concepts into disjoint, singly-rooted groups, that are more manageable to work with and comprehend, has been presented. The partitioning methodology formed the foundation for the development of an abstraction network for the overlapping concepts called a disjoint partial-area taxonomy. This new disjoint partial-area taxonomy offers a collection of semantically uniform partial-areas and is exploited herein as the basis for a novel auditing methodology. The review of the overlapping concepts is done in a top-down order within semantically uniform groups. These groups are themselves reviewed in a top-down order, which proceeds from the less complex to the more complex overlapping concepts. The results of applying the methodology to SNOMED's Specimen hierarchy are presented. Hypotheses regarding error ratios for overlapping concepts and between different kinds of overlapping concepts are formulated. Two phases of auditing the Specimen hierarchy for two releases of SNOMED are reported on. With the use of the double bootstrap and Fisher's exact test (two-tailed), the auditing of concepts and especially roots of overlapping partial-areas is shown to yield a statistically significant higher proportion of errors.

Assessment of the quality of multi-institutional data for population-based epidemiological studies: a case study of atoxoplasmosis in Bali mynahs (Leucopsar rothschildi).

To evaluate the potential to conduct population-based epidemiologic studies using multi-institutional data, the quality of data extracted from the MedARKS system was evaluated with regard to atoxoplasmosis in Bali mynahs (Leucopsar rothschildi). Data extracted for 338 Bali mynahs from 34 institutions were analyzed for completeness, correctness, and consistency and subjectively analyzed for accessibility. Data completeness was ascertained by analysis of missing records, tests, and blank fields. Two hundred and sixty four records had animal enclosure information missing, nine records were missing from the parasitology module, and 85 records did not include tests or results. Data correctness was assessed by evaluation of unclear and inaccurate results. From 2,432 parasitology records, 81 tests (3.3%) were not definitive because of uninterpretable entries. Data consistency was assessed by comparing the problem list to positive tests in the parasitology module and listing of buffy coat smears in the clinical pathology module. Overall, six different terms with 28 values were used to denote "positive for Atoxoplasma sp.," and a substantial number of discrepancies were found between problem lists, parasitology modules, and test findings among these data records. In general, this study showed that the data contained in a computer patient record (CPR) system for zoos has great promise for population-level studies if specific areas are addressed, including 1) reduced reliance on free-text data entry, 2) universal use of a standardized vocabulary, 3) use of methods to identify and track individual animals accurately and easily, 4) integration of data checks and maintenance methods, and 5) concerted use of "centralized" animal medical record information.

Association between findings on palmarodorsal radiographic images and detection of a fracture in the proximal sesamoid bones of forelimbs obtained from cadavers of racing Thoroughbreds.

OBJECTIVE: To determine the distribution for limbs and bones in horses with fractures of the proximal sesamoid bones and relationships with findings on palmarodorsal radiographic images. SAMPLE POPULATION: Proximal sesamoid bones obtained from both forelimbs of cadavers of 328 racing Thoroughbreds. PROCEDURE: Osteophytes; large vascular channels; and fracture location, orientation, configuration, and margin distinctness were categorized by use of high-detail contact palmarodorsal radiographs. Distributions of findings were determined. Relationships between radiographic findings and fracture characteristics were examined by use of chi2 and logistic regression techniques. RESULTS: Fractures were detected in 136 (41.5%) horses. Biaxial fractures were evident in 109 (80%) horses with a fracture. Osteophytes and large vascular channels were evident in 266 (81%) and 325 (99%) horses, respectively. Medial bones typically had complete transverse or split transverse simple fractures, indistinct fracture margins, > 1 vascular channel that was > 1 mm in width, and osteophytes in abaxial wing and basilar middle or basilar abaxial locations. Lateral bones typically had an oblique fracture and distinct fracture margins. Odds of proximal sesamoid bone fracture were approximately 2 to 5 times higher in bones without radiographic evidence of osteophytes or large vascular channels, respectively. CONCLUSIONS AND CLINICAL RELEVANCE: Biaxial fractures of proximal sesamoid bones were common in cadavers of racing Thoroughbreds. Differences between medial and lateral bones for characteristics associated with fracture may relate to differences in fracture pathogeneses for these bones. Osteophytes and vascular channels were common findings; however, fractures were less likely to occur in bones with these features.

Perceptions of state public health officers and state veterinarians regarding risks of bioterrorism in the United States.

OBJECTIVE: To assess perceptions of state public health officers and state veterinarians in the United States regarding the risks of bioterrorism and determine the degree of support provided for activities related to bioterrorism. DESIGN: Cross-sectional survey. SAMPLE POPULATION: State public health officers and state veterinarians. PROCEDURE: A questionnaire was sent between April and June 2001 to the state public health officer and state veterinarian in each of the 50 states and the District of Columbia. RESULTS: Perceptions of the risk of bioterrorism attacks were similar for state public health officers and state veterinarians. Veterinarians perceived the risks associated with foot-and-mouth disease and Newcastle disease to be higher than did physicians. State veterinarians perceived the risks associated with an anthrax hoax, brucellosis, and ricin toxicosis to be lower than did state public health officers. Risk posed by agents that affected animals exclusively was perceived to be higher than risk posed by agents that affected humans exclusively and zoonotic agents. Number of full-time-equivalent positions devoted to bioterrorism surveillance and percentage of the budget devoted to bioterrorism activities were significantly lower for offices run by state veterinarians than for offices run by state public health officers. State veterinarians were significantly less likely to have knowledge of bioterrorism incidents within their state or district than were state public health officers. CONCLUSIONS AND CLINICAL RELEVANCE: Provision of additional resources to state veterinarians and explicit integration of their expertise and surveillance capabilities may be important to effectively mitigate the risk of bioterrorism.

Time-space clustering of human brucellosis, California, 1973-1992.

Infection with Brucella spp. continues to pose a human health risk in California despite great strides in eradicating the disease from domestic animals. Clustering of human cases in time and space has important public health implications for understanding risk factors and sources of infection. Temporal-spatial clustering of human brucellosis in California for the 20-year period 1973-1992 was evaluated by the Ederer-Myers-Mantel, Moran's I, and population-adjusted Moran's I procedures. Cases were clustered in concentrated agricultural regions in the first 5-year interval (1973-1977). Time-space clustering of human brucellosis cases in California late in the 20-year study period may reflect the distribution of Hispanic populations. Public health programs in California should focus on educating Hispanic populations about the risk of consuming dairy products, such as soft cheeses, made from unpasteurized milk.