Open-source Software Sustainability Models: Initial White Paper From the Informatics Technology for Cancer Research Sustainability and Industry Partnership Working Group.

BACKGROUND: The National Cancer Institute Informatics Technology for Cancer Research (ITCR) program provides a series of funding mechanisms to create an ecosystem of open-source software (OSS) that serves the needs of cancer research. As the ITCR ecosystem substantially grows, it faces the challenge of the long-term sustainability of the software being developed by ITCR grantees. To address this challenge, the ITCR sustainability and industry partnership working group (SIP-WG) was convened in 2019. OBJECTIVE: The charter of the SIP-WG is to investigate options to enhance the long-term sustainability of the OSS being developed by ITCR, in part by developing a collection of business model archetypes that can serve as sustainability plans for ITCR OSS development initiatives. The working group assembled models from the ITCR program, from other studies, and from the engagement of its extensive network of relationships with other organizations (eg, Chan Zuckerberg Initiative, Open Source Initiative, and Software Sustainability Institute) in support of this objective. METHODS: This paper reviews the existing sustainability models and describes 10 OSS use cases disseminated by the SIP-WG and others, including 3D Slicer, Bioconductor, Cytoscape, Globus, i2b2 (Informatics for Integrating Biology and the Bedside) and tranSMART, Insight Toolkit, Linux, Observational Health Data Sciences and Informatics tools, R, and REDCap (Research Electronic Data Capture), in 10 sustainability aspects: governance, documentation, code quality, support, ecosystem collaboration, security, legal, finance, marketing, and dependency hygiene. RESULTS: Information available to the public reveals that all 10 OSS have effective governance, comprehensive documentation, high code quality, reliable dependency hygiene, strong user and developer support, and active marketing. These OSS include a variety of licensing models (eg, general public license version 2, general public license version 3, Berkeley Software Distribution, and Apache 3) and financial models (eg, federal research funding, industry and membership support, and commercial support). However, detailed information on ecosystem collaboration and security is not publicly provided by most OSS. CONCLUSIONS: We recommend 6 essential attributes for research software: alignment with unmet scientific needs, a dedicated development team, a vibrant user community, a feasible licensing model, a sustainable financial model, and effective product management. We also stress important actions to be considered in future ITCR activities that involve the discussion of the sustainability and licensing models for ITCR OSS, the establishment of a central library, the allocation of consulting resources to code quality control, ecosystem collaboration, security, and dependency hygiene.

Introduction to Artificial Intelligence and Machine Learning for Pathology.

CONTEXT.­: Recent developments in machine learning have stimulated intense interest in software that may augment or replace human experts. Machine learning may impact pathology practice by offering new capabilities in analysis, interpretation, and outcomes prediction using images and other data. The principles of operation and management of machine learning systems are unfamiliar to pathologists, who anticipate a need for additional education to be effective as expert users and managers of the new tools. OBJECTIVE.­: To provide a background on machine learning for practicing pathologists, including an overview of algorithms, model development, and performance evaluation; to examine the current status of machine learning in pathology and consider possible roles and requirements for pathologists in local deployment and management of machine learning systems; and to highlight existing challenges and gaps in deployment methodology and regulation. DATA SOURCES.­: Sources include the biomedical and engineering literature, white papers from professional organizations, government reports, electronic resources, and authors' experience in machine learning. References were chosen when possible for accessibility to practicing pathologists without specialized training in mathematics, statistics, or software development. CONCLUSIONS.­: Machine learning offers an array of techniques that in recent published results show substantial promise. Data suggest that human experts working with machine learning tools outperform humans or machines separately, but the optimal form for this combination in pathology has not been established. Significant questions related to the generalizability of machine learning systems, local site verification, and performance monitoring remain to be resolved before a consensus on best practices and a regulatory environment can be established.

Career Paths of Pathology Informatics Fellowship Alumni.

BACKGROUND: The alumni of today's Pathology Informatics and Clinical Informatics fellowships fill diverse roles in academia, large health systems, and industry. The evolving training tracks and curriculum of Pathology Informatics fellowships have been well documented. However, less attention has been given to the posttraining experiences of graduates from informatics training programs. Here, we examine the career paths of subspecialty fellowship-trained pathology informaticians. METHODS: Alumni from four Pathology Informatics fellowship training programs were contacted for their voluntary participation in the study. We analyzed various components of training, and the subsequent career paths of Pathology Informatics fellowship alumni using data extracted from alumni provided curriculum vitae. RESULTS: Twenty-three out of twenty-seven alumni contacted contributed to the study. A majority had completed undergraduate study in science, technology, engineering, and math fields and combined track training in anatomic and clinical pathology. Approximately 30% (7/23) completed residency in a program with an in-house Pathology Informatics fellowship. Most completed additional fellowships (15/23) and many also completed advanced degrees (10/23). Common primary posttraining appointments included chief medical informatics officer (3/23), director of Pathology Informatics (10/23), informatics program director (2/23), and various roles in industry (3/23). Many alumni also provide clinical care in addition to their informatics roles (14/23). Pathology Informatics alumni serve on a variety of institutional committees, participate in national informatics organizations, contribute widely to scientific literature, and more than half (13/23) have obtained subspecialty certification in Clinical Informatics to date. CONCLUSIONS: Our analysis highlights several interesting phenomena related to the training and career trajectory of Pathology Informatics fellowship alumni. We note the long training track alumni complete in preparation for their careers. We believe flexible training pathways combining informatics and clinical training may help to alleviate the burden. We highlight the importance of in-house Pathology Informatics fellowships in promoting interest in informatics among residents. We also observe the many important leadership roles in academia, large community health systems, and industry available to early career alumni and believe this reflects a strong market for formally trained informaticians. We hope this analysis will be useful as we continue to develop the informatics fellowships to meet the future needs of our trainees and discipline.

Perceptions of pathology informatics by non-informaticist pathologists and trainees.

BACKGROUND: Although pathology informatics (PI) is essential to modern pathology practice, the field is often poorly understood. Pathologists who have received little to no exposure to informatics, either in training or in practice, may not recognize the roles that informatics serves in pathology. The purpose of this study was to characterize perceptions of PI by noninformatics-oriented pathologists and to do so at two large centers with differing informatics environments. METHODS: Pathology trainees and staff at Cleveland Clinic (CC) and Massachusetts General Hospital (MGH) were surveyed. At MGH, pathology department leadership has promoted a pervasive informatics presence through practice, training, and research. At CC, PI efforts focus on production systems that serve a multi-site integrated health system and a reference laboratory, and on the development of applications oriented to department operations. The survey assessed perceived definition of PI, interest in PI, and perceived utility of PI. RESULTS: The survey was completed by 107 noninformatics-oriented pathologists and trainees. A majority viewed informatics positively. Except among MGH trainees, confusion of PI with information technology (IT) and help desk services was prominent, even in those who indicated they understood informatics. Attendings and trainees indicated desire to learn more about PI. While most acknowledged that having some level of PI knowledge would be professionally useful and advantageous, only a minority plan to utilize it. CONCLUSIONS: Informatics is viewed positively by the majority of noninformatics pathologists at two large centers with differing informatics orientations. Differences in departmental informatics culture can be attributed to the varying perceptions of PI by different individuals. Incorrect perceptions exist, such as conflating PI with IT and help desk services, even among those who claim to understand PI. Further efforts by the PI community could address such misperceptions, which could help enable a better understanding of what PI is and is not, and potentially lead to increased acceptance by non-informaticist pathologists.

Computational Pathology: A Path Ahead.

CONTEXT: We define the scope and needs within the new discipline of computational pathology, a discipline critical to the future of both the practice of pathology and, more broadly, medical practice in general. OBJECTIVE: To define the scope and needs of computational pathology. DATA SOURCES: A meeting was convened in Boston, Massachusetts, in July 2014 prior to the annual Association of Pathology Chairs meeting, and it was attended by a variety of pathologists, including individuals highly invested in pathology informatics as well as chairs of pathology departments. CONCLUSIONS: The meeting made recommendations to promote computational pathology, including clearly defining the field and articulating its value propositions; asserting that the value propositions for health care systems must include means to incorporate robust computational approaches to implement data-driven methods that aid in guiding individual and population health care; leveraging computational pathology as a center for data interpretation in modern health care systems; stating that realizing the value proposition will require working with institutional administrations, other departments, and pathology colleagues; declaring that a robust pipeline should be fostered that trains and develops future computational pathologists, for those with both pathology and nonpathology backgrounds; and deciding that computational pathology should serve as a hub for data-related research in health care systems. The dissemination of these recommendations to pathology and bioinformatics departments should help facilitate the development of computational pathology.

Whole slide imaging for human epidermal growth factor receptor 2 immunohistochemistry interpretation: Accuracy, Precision, and reproducibility studies for digital manual and paired glass slide manual interpretation.

BACKGROUND: The use of digital whole slide imaging for human epidermal growth factor receptor 2 (HER2) immunohistochemistry (IHC) could create improvements in workflow and performance, allowing for central archiving of specimens, distributed and remote interpretation, and the potential for additional computerized automation. PROCEDURES: The accuracy, precision, and reproducibility of manual digital interpretation for HER2 IHC were determined by comparison to manual glass slide interpretation. Inter- and intra-pathologist reproducibility and precision between the glass slide and digital interpretations of HER2 IHC were determined in 5 studies using DAKO HercepTest-stained breast cancer slides with the Philips Digital Pathology System. In 2 inter-method studies, 3 pathologists interpreted glass and digital slides in sequence or in random order with a minimum of 7 days as a washout period. These studies also measured inter-observer reproducibility and precision. Another two studies measured intra-pathologist reproducibility on cases read 10 times by glass and digital methods. One additional study evaluated the effects of adding IHC control slides with each run, using 1 pathologist interpreting glass and digital slides randomized from the sets above along with appropriate controls for each slide in the set. RESULTS: The overall results show that there is no statistical difference between the variance of performance when comparing glass and digital HER2 interpretations; and there were no effects noted when control tissues were evaluated in conjunction with the test slides. CONCLUSIONS: The results show that there is an equivalence of result when interpreting HER2 IHC slides in breast cancer by either glass slides or digital images. Digital interpretation can therefore be safely and effectively used for this purpose.

Longitudinal engagement of pathology residents: a proposed approach for informatics training.

OBJECTIVES: The intersecting of pathology training and practice and the utilization of information technology has become an increasingly common occurrence, and the most effective means of teaching residents informatics during these invaluable years has yet to be firmly established. METHODS: In offering the idea of longitudinal engagement that stresses early and extended trainee involvement, we attempt to provide a different manner of helping address some of the leading time-limited issues surrounding education in informatics. RESULTS: The proposed model is intended to allow building off a base of fundamentals reached through introductory didactics, exposure to and active participation in departmental and hospital-wide administrative bodies, refining of initial skills gained through frequent mentoring and coaching, and combining these cumulative knowledge and experiential underpinnings with graduated responsibility in a particular area of expertise. CONCLUSIONS: In transforming the ways in which pathology residencies teach their trainees informatics, the prospects of realizing its potential utility are heightened.

The ongoing evolution of the core curriculum of a clinical fellowship in pathology informatics.

The Partners HealthCare system's Clinical Fellowship in Pathology Informatics (Boston, MA, USA) faces ongoing challenges to the delivery of its core curriculum in the forms of: (1) New classes of fellows annually with new and varying educational needs and increasingly fractured, enterprise-wide commitments; (2) taxing electronic health record (EHR) and laboratory information system (LIS) implementations; and (3) increasing interest in the subspecialty at the academic medical centers (AMCs) in what is a large health care network. In response to these challenges, the fellowship has modified its existing didactic sessions and piloted both a network-wide pathology informatics lecture series and regular "learning laboratories". Didactic sessions, which had previously included more formal discussions of the four divisions of the core curriculum: Information fundamentals, information systems, workflow and process, and governance and management, now focus on group discussions concerning the fellows' ongoing projects, updates on the enterprise-wide EHR and LIS implementations, and directed questions about weekly readings. Lectures are given by the informatics faculty, guest informatics faculty, current and former fellows, and information systems members in the network, and are open to all professional members of the pathology departments at the AMCs. Learning laboratories consist of small-group exercises geared toward a variety of learning styles, and are driven by both the fellows and a member of the informatics faculty. The learning laboratories have created a forum for discussing real-time and real-world pathology informatics matters, and for incorporating awareness of and timely discussions about the latest pathology informatics literature. These changes have diversified the delivery of the fellowship's core curriculum, increased exposure of faculty, fellows and trainees to one another, and more equitably distributed teaching responsibilities among the entirety of the pathology informatics asset in the network. Though the above approach has been in place less than a year, we are presenting it now as a technical note to allow for further discussion of evolving educational opportunities in pathology informatics and clinical informatics in general, and to highlight the importance of having a flexible fellowship with active participation from its fellows.

Pathology informatics fellowship training: Focus on molecular pathology.

BACKGROUND: Pathology informatics is both emerging as a distinct subspecialty and simultaneously becoming deeply integrated within the breadth of pathology practice. As specialists, pathology informaticians need a broad skill set, including aptitude with information fundamentals, information systems, workflow and process, and governance and management. Currently, many of those seeking training in pathology informatics additionally choose training in a second subspecialty. Combining pathology informatics training with molecular pathology is a natural extension, as molecular pathology is a subspecialty with high potential for application of modern biomedical informatics techniques. METHODS AND RESULTS: Pathology informatics and molecular pathology fellows and faculty evaluated the current fellowship program's core curriculum topics and subtopics for relevance to molecular pathology. By focusing on the overlap between the two disciplines, a structured curriculum consisting of didactics, operational rotations, and research projects was developed for those fellows interested in both pathology informatics and molecular pathology. CONCLUSIONS: The scope of molecular diagnostics is expanding dramatically as technology advances and our understanding of disease extends to the genetic level. Here, we highlight many of the informatics challenges facing molecular pathology today, and outline specific informatics principles necessary for the training of future molecular pathologists.

The 2013 symposium on pathology data integration and clinical decision support and the current state of field.

BACKGROUND: Pathologists and informaticians are becoming increasingly interested in electronic clinical decision support for pathology, laboratory medicine and clinical diagnosis. Improved decision support may optimize laboratory test selection, improve test result interpretation and permit the extraction of enhanced diagnostic information from existing laboratory data. Nonetheless, the field of pathology decision support is still developing. To facilitate the exchange of ideas and preliminary studies, we convened a symposium entitled: Pathology data integration and clinical decision support. METHODS: The symposium was held at the Massachusetts General Hospital, on May 10, 2013. Participants were selected to represent diverse backgrounds and interests and were from nine different institutions in eight different states. RESULTS: The day included 16 plenary talks and three panel discussions, together covering four broad areas. Summaries of each presentation are included in this manuscript. CONCLUSIONS: A number of recurrent themes emerged from the symposium. Among the most pervasive was the dichotomy between diagnostic data and diagnostic information, including the opportunities that laboratories may have to use electronic systems and algorithms to convert the data they generate into more useful information. Differences between human talents and computer abilities were described; well-designed symbioses between humans and computers may ultimately optimize diagnosis. Another key theme related to the unique needs and challenges in providing decision support for genomics and other emerging diagnostic modalities. Finally, many talks relayed how the barriers to bringing decision support toward reality are primarily personnel, political, infrastructural and administrative challenges rather than technological limitations.

Pathology informatics fellowship retreats: The use of interactive scenarios and case studies as pathology informatics teaching tools.

BACKGROUND: Last year, our pathology informatics fellowship added informatics-based interactive case studies to its existing educational platform of operational and research rotations, clinical conferences, a common core curriculum with an accompanying didactic course, and national meetings. METHODS: The structure of the informatics case studies was based on the traditional business school case study format. Three different formats were used, varying in length from short, 15-minute scenarios to more formal multiple hour-long case studies. Case studies were presented over the course of three retreats (Fall 2011, Winter 2012, and Spring 2012) and involved both local and visiting faculty and fellows. RESULTS: Both faculty and fellows found the case studies and the retreats educational, valuable, and enjoyable. From this positive feedback, we plan to incorporate the retreats in future academic years as an educational component of our fellowship program. CONCLUSIONS: Interactive case studies appear to be valuable in teaching several aspects of pathology informatics that are difficult to teach in more traditional venues (rotations and didactic class sessions). Case studies have become an important component of our fellowship's educational platform.

Different tracks for pathology informatics fellowship training: Experiences of and input from trainees in a large multisite fellowship program.

BACKGROUND: Pathology Informatics is a new field; a field that is still defining itself even as it begins the formalization, accreditation, and board certification process. At the same time, Pathology itself is changing in a variety of ways that impact informatics, including subspecialization and an increased use of data analysis. In this paper, we examine how these changes impact both the structure of Pathology Informatics fellowship programs and the fellows' goals within those programs. MATERIALS AND METHODS: As part of our regular program review process, the fellows evaluated the value and effectiveness of our existing fellowship tracks (Research Informatics, Clinical Two-year Focused Informatics, Clinical One-year Focused Informatics, and Clinical 1 + 1 Subspecialty Pathology and Informatics). They compared their education, informatics background, and anticipated career paths and analyzed them for correlations between those parameters and the fellowship track chosen. All current and past fellows of the program were actively involved with the project. RESULTS: Fellows' anticipated career paths correlated very well with the specific tracks in the program. A small set of fellows (Clinical - one or two year - Focused Informatics tracks) anticipated clinical careers primarily focused in informatics (Director of Informatics). The majority of the fellows, however, anticipated a career practicing in a Pathology subspecialty, using their informatics training to enhance that practice (Clinical 1 + 1 Subspecialty Pathology and Informatics Track). Significantly, all fellows on this track reported they would not have considered a Clinical Two-year Focused Informatics track if it was the only track offered. The Research and the Clinical One-year Focused Informatics tracks each displayed unique value for different situations. CONCLUSIONS: It seems a "one size fits all" fellowship structure does not fit the needs of the majority of potential Pathology Informatics candidates. Increasingly, these fellowships must be able to accommodate the needs of candidates anticipating a wide range of Pathology Informatics career paths, be able to accommodate Pathology's increasingly subspecialized structure, and do this in a way that respects the multiple fellowships needed to become a subspecialty pathologist and informatician. This is further complicated as Pathology Informatics begins to look outward and takes its place in the growing, and still ill-defined, field of Clinical Informatics, a field that is not confined to just one medical specialty, to one way of practicing medicine, or to one way of providing patient care.

A core curriculum for clinical fellowship training in pathology informatics.

BACKGROUND: In 2007, our healthcare system established a clinical fellowship program in Pathology Informatics. In 2010 a core didactic course was implemented to supplement the fellowship research and operational rotations. In 2011, the course was enhanced by a formal, structured core curriculum and reading list. We present and discuss our rationale and development process for the Core Curriculum and the role it plays in our Pathology Informatics Fellowship Training Program. MATERIALS AND METHODS: The Core Curriculum for Pathology Informatics was developed, and is maintained, through the combined efforts of our Pathology Informatics Fellows and Faculty. The curriculum was created with a three-tiered structure, consisting of divisions, topics, and subtopics. Primary (required) and suggested readings were selected for each subtopic in the curriculum and incorporated into a curated reading list, which is reviewed and maintained on a regular basis. RESULTS: Our Core Curriculum is composed of four major divisions, 22 topics, and 92 subtopics that cover the wide breadth of Pathology Informatics. The four major divisions include: (1) Information Fundamentals, (2) Information Systems, (3) Workflow and Process, and (4) Governance and Management. A detailed, comprehensive reading list for the curriculum is presented in the Appendix to the manuscript and contains 570 total readings (current as of March 2012). DISCUSSION: The adoption of a formal, core curriculum in a Pathology Informatics fellowship has significant impacts on both fellowship training and the general field of Pathology Informatics itself. For a fellowship, a core curriculum defines a basic, common scope of knowledge that the fellowship expects all of its graduates will know, while at the same time enhancing and broadening the traditional fellowship experience of research and operational rotations. For the field of Pathology Informatics itself, a core curriculum defines to the outside world, including departments, companies, and health systems considering hiring a pathology informatician, the core knowledge set expected of a person trained in the field and, more fundamentally, it helps to define the scope of the field within Pathology and healthcare in general.

Clinical fellowship training in pathology informatics: A program description.

BACKGROUND: In 2007, our healthcare system established a clinical fellowship program in pathology informatics. In 2011, the program benchmarked its structure and operations against a 2009 white paper "Program requirements for fellowship education in the subspecialty of clinical informatics", endorsed by the Board of the American Medical Informatics Association (AMIA) that described a proposal for a general clinical informatics fellowship program. METHODS: A group of program faculty members and fellows compared each of the proposed requirements in the white paper with the fellowship program's written charter and operations. The majority of white paper proposals aligned closely with the rules and activities in our program and comparison was straightforward. In some proposals, however, differences in terminology, approach, and philosophy made comparison less direct, and in those cases, the thinking of the group was recorded. After the initial evaluation, the remainder of the faculty reviewed the results and any disagreements were resolved. RESULTS: The most important finding of the study was how closely the white paper proposals for a general clinical informatics fellowship program aligned with the reality of our existing pathology informatics fellowship. The program charter and operations of the program were judged to be concordant with the great majority of specific white paper proposals. However, there were some areas of discrepancy and the reasons for the discrepancies are discussed in the manuscript. CONCLUSIONS: After the comparison, we conclude that the existing pathology informatics fellowship could easily meet all substantive proposals put forth in the 2009 clinical informatics program requirements white paper. There was also agreement on a number of philosophical issues, such as the advantages of multiple fellows, the need for core knowledge and skill sets, and the need to maintain clinical skills during informatics training. However, there were other issues, such as a requirement for a 2-year fellowship and for informatics fellowships to be done after primary board certification, that pathology should consider carefully as it moves toward a subspecialty status and board certification.

Integration of architectural and cytologic driven image algorithms for prostate adenocarcinoma identification.

INTRODUCTION: The advent of digital slides offers new opportunities within the practice of pathology such as the use of image analysis techniques to facilitate computer aided diagnosis (CAD) solutions. Use of CAD holds promise to enable new levels of decision support and allow for additional layers of quality assurance and consistency in rendered diagnoses. However, the development and testing of prostate cancer CAD solutions requires a ground truth map of the cancer to enable the generation of receiver operator characteristic (ROC) curves. This requires a pathologist to annotate, or paint, each of the malignant glands in prostate cancer with an image editor software - a time consuming and exhaustive process. Recently, two CAD algorithms have been described: probabilistic pairwise Markov models (PPMM) and spatially-invariant vector quantization (SIVQ). Briefly, SIVQ operates as a highly sensitive and specific pattern matching algorithm, making it optimal for the identification of any epithelial morphology, whereas PPMM operates as a highly sensitive detector of malignant perturbations in glandular lumenal architecture. METHODS: By recapitulating algorithmically how a pathologist reviews prostate tissue sections, we created an algorithmic cascade of PPMM and SIVQ algorithms as previously described by Doyle el al. [1] where PPMM identifies the glands with abnormal lumenal architecture, and this area is then screened by SIVQ to identify the epithelium. RESULTS: The performance of this algorithm cascade was assessed qualitatively (with the use of heatmaps) and quantitatively (with the use of ROC curves) and demonstrates greater performance in the identification of malignant prostatic epithelium. CONCLUSION: This ability to semi-autonomously paint nearly all the malignant epithelium of prostate cancer has immediate applications to future prostate cancer CAD development as a validated ground truth generator. In addition, such an approach has potential applications as a pre-screening/quality assurance tool.

Using computerized workflow simulations to assess the feasibility of whole slide imaging full adoption in a high-volume histology laboratory.

BACKGROUND: Whole slide Imaging (WSI) has been touted by many as the future of pathology, with estimates of full adoption occurring sometime in the next 5 to 15 years. While WSI devices have become increasingly capable since their inception, there has been little consideration of how WSI will be implemented and subsequently affect the workflow of high volume histology laboratories. METHODS: Histology workflow process data was collected from a high-volume histology laboratory (Massachusetts General Hospital) and a process model developed using business process management software. Computerized workflow simulations were performed and total histology process time evaluated under a number of different WSI conditions. RESULTS: Total histology process time increased approximately 10-fold to 20-fold over baseline with the presence of one WSI robot in the histology workflow. Depending on the specifications of the WSI robot, anywhere from 9 to 14 WSI robots were required within the histology workflow to minimize the effects of WSI. CONCLUSIONS: Placing a WSI robot into the current workflow of a high-volume histology laboratory with the intent of full adoption is not feasible. Implementing WSI without making significant changes to the current workflow of the histology laboratory would prove to be both disruptive and costly to surgical pathology.

The pathology informatics curriculum wiki: Harnessing the power of user-generated content.

BACKGROUND: The need for informatics training as part of pathology training has never been so critical, but pathology informatics is a wide and complex field and very few programs currently have the resources to provide comprehensive educational pathology informatics experiences to their residents. In this article, we present the "pathology informatics curriculum wiki", an open, on-line wiki that indexes the pathology informatics content in a larger public wiki, Wikipedia, (and other online content) and organizes it into educational modules based on the 2003 standard curriculum approved by the Association for Pathology Informatics (API). METHODS AND RESULTS: In addition to implementing the curriculum wiki at http://pathinformatics.wikispaces.com, we have evaluated pathology informatics content in Wikipedia. Of the 199 non-duplicate terms in the API curriculum, 90% have at least one associated Wikipedia article. Furthermore, evaluation of articles on a five-point Likert scale showed high scores for comprehensiveness (4.05), quality (4.08), currency (4.18), and utility for the beginner (3.85) and advanced (3.93) learners. These results are compelling and support the thesis that Wikipedia articles can be used as the foundation for a basic curriculum in pathology informatics. CONCLUSIONS: The pathology informatics community now has the infrastructure needed to collaboratively and openly create, maintain and distribute the pathology informatics content worldwide (Wikipedia) and also the environment (the curriculum wiki) to draw upon its own resources to index and organize this content as a sustainable basic pathology informatics educational resource. The remaining challenges are numerous, but largest by far will be to convince the pathologists to take the time and effort required to build pathology informatics content in Wikipedia and to index and organize this content for education in the curriculum wiki.