Improving information retrieval using Medical Subject Headings Concepts: a test case on rare and chronic diseases.

BACKGROUND: As more scientific work is published, it is important to improve access to the biomedical literature. Since 2000, when Medical Subject Headings (MeSH) Concepts were introduced, the MeSH Thesaurus has been concept based. Nevertheless, information retrieval is still performed at the MeSH Descriptor or Supplementary Concept level. OBJECTIVE: The study assesses the benefit of using MeSH Concepts for indexing and information retrieval. METHODS: Three sets of queries were built for thirty-two rare diseases and twenty-two chronic diseases: (1) using PubMed Automatic Term Mapping (ATM), (2) using Catalog and Index of French-language Health Internet (CISMeF) ATM, and (3) extrapolating the MEDLINE citations that should be indexed with a MeSH Concept. RESULTS: Type 3 queries retrieve significantly fewer results than type 1 or type 2 queries (about 18,000 citations versus 200,000 for rare diseases; about 300,000 citations versus 2,000,000 for chronic diseases). CISMeF ATM also provides better precision than PubMed ATM for both disease categories. DISCUSSION: Using MeSH Concept indexing instead of ATM is theoretically possible to improve retrieval performance with the current indexing policy. However, using MeSH Concept information retrieval and indexing rules would be a fundamentally better approach. These modifications have already been implemented in the CISMeF search engine.

Performance evaluation of Unified Medical Language System®'s synonyms expansion to query PubMed.

BACKGROUND: PubMed is the main access to medical literature on the Internet. In order to enhance the performance of its information retrieval tools, primarily non-indexed citations, the authors propose a method: expanding users' queries using Unified Medical Language System' (UMLS) synonyms i.e. all the terms gathered under one unique Concept Unique Identifier. METHODS: This method was evaluated using queries constructed to emphasize the differences between this new method and the current PubMed automatic term mapping. Four experts assessed citation relevance. RESULTS: Using UMLS, we were able to retrieve new citations in 45.5% of queries, which implies a small increase in recall. The new strategy led to a heterogeneous 23.7% mean increase in non-indexed citation retrieved. Of these, 82% have been published less than 4 months earlier. The overall mean precision was 48.4% but differed according to the evaluators, ranging from 36.7% to 88.1% (Inter rater agreement was poor: kappa = 0.34). CONCLUSIONS: This study highlights the need for specific search tools for each type of user and use-cases. The proposed strategy may be useful to retrieve recent scientific advancement.

Evaluation of multi-terminology super-concepts for information retrieval.

BACKGROUND: Following a recent change in the indexing policy for French quality controlled health gateway CISMeF, multiple terminologies are now being used for indexing in addition to MeSH®. OBJECTIVE: To evaluate precision and recall of super-concepts for information retrieval in a multi-terminology paradigm compared to MeSH-only. METHODS: We evaluate the relevance of resources retrieved by multi-terminology super-concepts and MeSH-only super-concepts queries. RESULTS: Recall was 8-14% higher for multi-terminology super-concepts compared to MeSH only super-concepts. Precision decreased from 0.66 for MeSH only super-concepts to 0.61 for multi-terminology super-concepts. Retrieval performance was found to vary significantly depending on the super-concepts (p<10-4) and indexing methods (manual vs automatic; p<0.004). CONCLUSION: A multi-terminology paradigm contributes to increase recall but lowers precision. Automated tools for indexing are not accurate enough to allow a very precise information retrieval.

Health multi-terminology portal: a semantic added-value for patient safety.

Since the mid-90s, several quality-controlled health gateways were developed. In France, CISMeF is the leading health gateway. It indexes Internet resources from the main institutions, using the MeSH thesaurus and the Dublin Core metadata element set. Since 2005, the CISMeF Information System (IS) includes 24 health terminologies, classifications and thesauri for indexing and information retrieval. This work aims at creating a Health Multi-Terminology Portal (HMTP) and connect it to the CISMeF Terminology Database mainly for searching concepts and terms among all the health controlled vocabularies available in French (or in English and translated in French) and browsing it dynamically. To integrate the terminologies in the CISMeF IS, three steps are necessary: (1) designing a meta-model into which each terminology can be integrated, (2) developing a process to include terminologies into the HMTP, (3) building and integrating existing and new inter-terminology mappings into the HMTP. A total of 24 terminologies are included in the HMTP, with 575,300 concepts, 852,000 synonyms, 222,800 definitions and 1,180,000 relations. Heightteen of these terminologies are not included yet in the UMLS among them, some from the World Health Organization. Since January 2010, HMTP is daily used by CISMeF librarians to index in multi-terminology mode. A health multiterminology portal is a valuable tool helping the indexing and the retrieval of resources from a quality-controlled patient safety gateway. It can also be very useful for teaching or performing audits in terminology management.

Optimization of the PubMed Automatic Term Mapping.

PubMed, freely available on the internet, is the best known database for medical information. We propose a method of optimization of the PubMed Automatic Term Mapping (ATM) that includes MeSH terms. This method is evaluated using two queries constructed to emphasize the differences between the PubMed queries as they are at present and also between these queries and the optimized one. The proposed query is significantly more precise than the current PubMed query (54.5% vs. 27%). The optimized query proposed would be easy to implement into PubMed.

[Searching French institutional health information sources: catalogue and index of French-language medical sites (CISMeF)]. / Recherche des sources d'information institutionnelle de santé françaises: le "Catalogue et index des sites médicaux francophones" (CISMeF).

The Catalogue and index of French-language medical sites (CISMeF) is a medical portal that provides users with results as pertinent as possible according to their requirements, expectations, and context of use. Indexing and single-term research are based on theMedical subject headings(MeSH) thesaurus. The integration of new medical terminology for indexing the catalogue's resources is intended to minimize false-negatives during searches and to contextualize the users' needs. The creation of a drug information portal makes more targeted research possible, with numerous entries according to user (physicians, pharmacists, chemists, and pharmacologists). For simplicity's sake, the catalogue's index of resources by different nomenclatures is not entirely displayed. The choice of display is left to the user, with MeSH only as the default. These multi-nomenclature tools should be applicable as well to electronic patient records. In this case, the objective is to improve patient care by better searches and identification of the information required during consultations and hospitalization.

[Quality and use of websites presenting public health education and training opportunities]. / Utilisation et qualité des sites présentant l'offre de formations en santé publique.

The objective of this work was to study the use of the Internet and the quality of the websites for postgraduate public health courses in France, and to compare them with equivalent courses in the United States of America. Between June 2004 and January 2005, the authorized public health diplomas proposed in France and in the United States were inventoried and listed, and then all websites of these public health diplomas were systematically visited and reviewed using a standardized questionnaire. In France, 36 public health courses (7 post graduate diplomas [DEA], 13 Masters degrees [DESS] and 16 masters of public health [MPH]) were identified and selected from 53 websites. Information on student profiles, prerequisite skills, the courses' curricula and program descriptions and the potential career opportunities were more frequently available for the MPH compared to the DEA and DESS. In United States, 66 MPH and 127 Master of Science in Public Health (MSPH) programs were accredited. The target public and validation methods were more often indicated on the American sites, while the prerequisite skills were more frequently found on the French sites. The recent implementation of the LMD (Bachelor's-Masters-Doctoral degrees) education system in France has encouraged the utilisation of Internet as an information and communications tool for the presentation and marketing of these new diplomas.

French MeSH Browser: a cross-language tool to access MEDLINE/PubMed.

When searching the medical literature, health professionals and lay people strongly prefer to use their native language. Therefore, Medical Subject Headings (MeSH) translations would be helpful to those who are not fluent in English to access scientific papers indexed in the MEDLINE bibliographic database. Furthermore, medical terminologies such as MeSH are challenging in any language. In this context, a French MeSH Browser was developed.

Evaluation of meta-concepts for information retrieval in a quality-controlled health gateway.

BACKGROUND: CISMeF is a French quality-controlled health gateway that uses the MeSH thesaurus. We introduced two new concepts, metaterms (medical specialty which has semantic links with one or more MeSH terms, subheadings and resource types) and resource types. OBJECTIVE: Evaluate precision and recall of metaterms. METHODS: We created 16 pairs of queries. Each pair concerned the same topic, but one used metaterms and one MeSH terms. To assess precision, each document retrieved by the query was classified as irrelevant, partly relevant or fully relevant. RESULTS: The 16 queries yielded 943 documents for metaterm queries and 139 for MeSH term queries. The recall of MeSH term queries was 0.44 (compared to 1 for metaterm queries) and the precision were identical for MeSH term and metaterm queries. CONCLUSION: Metaconcept such as CISMeF metaterms allows a better recall with a similar precision that MeSH terms in a quality controlled health gateway.

Strategies for health information retrieval.

BACKGROUND: The amount of health data accessible on the Web is increasing and Internet has become a major source of health information. Many tools and search engines are available but medical information retrieval remains difficult for both the health professional and the patients. OBJECTIVE: In this paper we describe heuristics that aim at matching as much as possible queries with the content of the documents in the context of the CISMeF catalogue (Catalogue and Index of Health Resources in French) and its Doc'CISMeF search tool. The queries are represented by terms and the content of the documents is indexed by a terminology based on the MeSH thesaurus. RESULTS: Several operations are performed to match the terms of the terminology: natural language processing techniques on multi-words queries, phonemisation, spelling correction, plain text search with adjacency etc.. Each one is tested to evaluate its contribution in matching the terminology and the indexed documents. CONCLUSION: The implemented heuristics contribute significantly with good results in maximising as much as possible the recall of the Doc'CISMeF search tool.

A method of cross-lingual consumer health information retrieval.

OBJECTIVES: This paper presents a method of cross-language information retrieval aiming to make medical information available to patients in French and English, regardless of the query language they wish to use. METHODS: We describe the two MeSH-related terminologies used in this work. We show that the French patient synonyms included in CISMeF can be automatically mapped to the English consumer-oriented health topics used in MEDLINEplus, via the MeSH thesaurus. The links between French and English patient terms thus inferred can subsequently be exploited to automatically translate patient queries. RESULTS: 129 MEDLINEplus topics have been mapped to 142 CISMeF patient synonyms. Contextual links for cross-language retrieval have been added to the patient dedicated French information Gateway CISMeF. CONCLUSION: we have presented an efficient method for cross-lingual patient information retrieval in French and English, which may also be applied to other language pairs, subject to the availability of patient terminologies and of the MeSH thesaurus in these languages.

A web-based teaching resource to prepare for final undergraduate examination: a French pilot study.

Access to accurate and quality-controlled health information on the Internet for medical students is not an easy task. CISMeF is the search tool of a MeSH-indexed directory of medical Internet resources in French. Since 2004, a new French Pre-Residency Examination (PRE) is compulsory for all medical students in the 6th year of the curriculum. The goal of this study is to evaluate CISMeF as a tool to provide teaching resources available on the Internet covering PRE material. The CISMeF terminology and the PRE CISMeF module are described. To assess the CISMeF performance in covering PRE program, its precision (number of relevant resources/number of overall resources extracted by CISMeF) and coverage (number of PRE questions covered by at least one resource in the CISMeF gateway) were computed. The CISMeF module for the new French Pre-Residency Examination is efficient as it already covers 95.7% of the program with a precision of 82.2%. Our data demonstrates that CISMeF is acceptable to guide students' learning and should be a useful teaching resource for the preparation of the French Pre-Residency Examination.

An introduction to the Semantic Web for health sciences librarians.

OBJECTIVES: The paper (1) introduces health sciences librarians to the main concepts and principles of the Semantic Web (SW) and (2) briefly reviews a number of projects on the handling of biomedical information that uses SW technology. METHODOLOGY: The paper is structured into two main parts. "Semantic Web Technology" provides a high-level description, with examples, of the main standards and concepts: extensible markup language (XML), Resource Description Framework (RDF), RDF Schema (RDFS), ontologies, and their utility in information retrieval, concluding with mention of more advanced SW languages and their characteristics. "Semantic Web Applications and Research Projects in the Biomedical Field" is a brief review of the Unified Medical Language System (UMLS), Generalised Architecture for Languages, Encyclopedias and Nomenclatures in Medicine (GALEN), HealthCyberMap, LinkBase, and the thesaurus of the National Cancer Institute (NCI). The paper also mentions other benefits and by-products of the SW, citing projects related to them. DISCUSSION AND CONCLUSIONS: Some of the problems facing the SW vision are presented, especially the ways in which the librarians' expertise in organizing knowledge and in structuring information may contribute to SW projects.

A MEDLINE categorization algorithm.

BACKGROUND: Categorization is designed to enhance resource description by organizing content description so as to enable the reader to grasp quickly and easily what are the main topics discussed in it. The objective of this work is to propose a categorization algorithm to classify a set of scientific articles indexed with the MeSH thesaurus, and in particular those of the MEDLINE bibliographic database. In a large bibliographic database such as MEDLINE, finding materials of particular interest to a specialty group, or relevant to a particular audience, can be difficult. The categorization refines the retrieval of indexed material. In the CISMeF terminology, metaterms can be considered as super-concepts. They were primarily conceived to improve recall in the CISMeF quality-controlled health gateway. METHODS: The MEDLINE categorization algorithm (MCA) is based on semantic links existing between MeSH terms and metaterms on the one hand and between MeSH subheadings and metaterms on the other hand. These links are used to automatically infer a list of metaterms from any MeSH term/subheading indexing. Medical librarians manually select the semantic links. RESULTS: The MEDLINE categorization algorithm lists the medical specialties relevant to a MEDLINE file by decreasing order of their importance. The MEDLINE categorization algorithm is available on a Web site. It can run on any MEDLINE file in a batch mode. As an example, the top 3 medical specialties for the set of 60 articles published in BioMed Central Medical Informatics & Decision Making, which are currently indexed in MEDLINE are: information science, organization and administration and medical informatics. CONCLUSION: We have presented a MEDLINE categorization algorithm in order to classify the medical specialties addressed in any MEDLINE file in the form of a ranked list of relevant specialties. The categorization method introduced in this paper is based on the manual indexing of resources with MeSH (terms/subheadings) pairs by NLM indexers. This algorithm may be used as a new bibliometric tool.

Enhancing the MeSH thesaurus to retrieve French online health resources in a quality-controlled gateway.

The amount of health information available on the Internet is considerable. In this context, several health gateways have been developed. Among them, CISMeF (Catalogue and Index of Health Resources in French) was designed to catalogue and index health resources in French. The goal of this article is to describe the various enhancements to the MeSH thesaurus developed by the CISMeF team to adapt this terminology to the broader field of health Internet resources instead of scientific articles for the medline bibliographic database. CISMeF uses two standard tools for organizing information: the MeSH thesaurus and several metadata element sets, in particular the Dublin Core metadata format. The heterogeneity of Internet health resources led the CISMeF team to enhance the MeSH thesaurus with the introduction of two new concepts, respectively, resource types and metaterms. CISMeF resource types are a generalization of the publication types of medline. A resource type describes the nature of the resource and MeSH keyword/qualifier pairs describe the subject of the resource. A metaterm is generally a medical specialty or a biological science, which has semantic links with one or more MeSH keywords, qualifiers and resource types. The CISMeF terminology is exploited for several tasks: resource indexing performed manually, resource categorization performed automatically, visualization and navigation through the concept hierarchies and information retrieval using the Doc'CISMeF search engine. The CISMeF health gateway uses several MeSH thesaurus enhancements to optimize information retrieval, hierarchy navigation and automatic indexing.

Using CISMeF MeSH "Encapsulated" terminology and a categorization algorithm for health resources.

INTRODUCTION: CISMeF is a Quality Controlled Health Gateway using a terminology based on the Medical Subject Headings (MeSH) thesaurus that displays medical specialties (metaterms) and the relationships existing between them and MeSH terms. OBJECTIVE: The need to classify the resources within the catalogue has led us to combine this type of semantic information with domain expert knowledge for health resources categorization purposes. MATERIAL AND METHODS: A two-step categorization process consisting of mapping resource keywords to CISMeF metaterms and ranking metaterms by decreasing coverage in the resource has been developed. We evaluate this algorithm on a random set of 123 resources extracted from the CISMeF catalogue. Our gold standard for this evaluation is the manual classification provided by a domain expert, viz. a librarian of the team. RESULTS: The CISMeF algorithm shows 81% precision and 93% recall, and 62% of the resources were assigned a "fully relevant" or "fairly relevant" categorization according to strict standards. DISCUSSION: A thorough analysis of the results has enabled us to find gaps in the knowledge modeling of the CISMeF terminology. The necessary adjustments having been made, the algorithm is currently used in CISMeF for resource categorization.

Towards a unified medical lexicon for French.

Medical Informatics has a constant need for basic Medical Language Processing tasks, e.g., for coding into controlled vocabularies, free text indexing and information retrieval. Most of these tasks involve term matching and rely on lexical resources: lists of words with attached information, including inflected forms and derived words, etc. Such resources are publicly available for the English language with the UMLS Specialist Lexicon, but not in other languages. For the French language, several teams have worked on the subject and built local lexical resources. The goal of the present work is to pool and unify these resources and to add extensively to them by exploiting medical terminologies and corpora, resulting in a unified medical lexicon for French (UMLF). This paper exposes the issues raised by such an objective, describes the methods on which the project relies and illustrates them with experimental results.

Modelisation of consumer health information in a quality-controlled gateway.

The amount of health data accessible on the Web is increasing and Internet has become a major source of health information. Many tools and search engines are available but medical information retrieval remains difficult for both the health professional and the patients. In this paper we describe CISMeF-patients. It is a sub-part of CISMeF, a structured quality-controlled subject gateway. CISMeF-patients has been designed for the patients, their families and the general public who are often unfamiliar with the medical domain and the medical vocabulary. The resources included in CISMeF-patients are described using the Dublin Core metadata format. To index them, CISMeF-patients and CISMeF share the same terminology, which 'encapsulates' the MeSH thesaurus with a layer of synonyms. Unlike Medline-plus and Medline, sharing the same terminology allows to a CISMeF-patients end-user the possibility to extend his query to the CISMeF catalogue (e.g. for searching teaching resources or clinical guidelines).

Metadata element set in a quality-controlled subject gateway: a step to a health semantic Web.

BACKGROUND: Quality-controlled subject gateways are Internet services which apply a selected set of targeted measures to support systematic resource discovery. Considerable manual effort is used to process a selection of resources which meet quality criteria and to display a extensive description and indexing of these resources with standards-based metadata. OBJECTIVE: Several metadata element sets are proposed to describe, index and quality health resources to be included in a French quality-controlled health gateway called CISMeF. The main objectives were to enhance Internet health document retrieval and navigation, and to allow interoperability with other Internet services. RESULTS: The Dublin Core metadata element set is used to describe and index all Internet health resources included in CISMeF. For teaching resources, some elements from IEEE1484 Learning Object Metadata are also used. For evidenced-base medicine resources, specific metadata are employed which assess the health content quality. The HIDDEL metadata set is used to enhance transparency, trust and quality of health information on the Internet. CONCLUSION: Comprehensive metadata element sets can be extremely useful to describe, index and assess health resources on the Internet in a quality-controlled subject gateway. Machine-readable metadata creates an Semantic Web which is more efficient for end-users as compared to the current Web.

UMLF: a Unified Medical Lexicon for French.

Lexical resources for medical language, such as lists of words with inflectional and derivational information, are publicly available for the English lantuate with the UMLS Specialist Lexicon. The goal of the UMLF project is to pool and unify existing resources and to add extensively to them by exploiting medical terminologies and corpora, resulting in a Unified Medical Lexicon for French. We present here the current status of the project.