ABSTRACT
Pathogen genomics is a critical tool for public health surveillance, infection control, outbreak investigations as well as research. In order to make use of pathogen genomics data, they must be interpreted using contextual data (metadata). Contextual data include sample metadata, laboratory methods, patient demographics, clinical outcomes and epidemiological information. However, the variability in how contextual information is captured by different authorities and how it is encoded in different databases poses challenges for data interpretation, integration and their use/re-use. The DataHarmonizer is a template-driven spreadsheet application for harmonizing, validating and transforming genomics contextual data into submission-ready formats for public or private repositories. The tool's web browser-based JavaScript environment enables validation and its offline functionality and local installation increases data security. The DataHarmonizer was developed to address the data sharing needs that arose during the COVID-19 pandemic, and was used by members of the Canadian COVID Genomics Network (CanCOGeN) to harmonize SARS-CoV-2 contextual data for national surveillance and for public repository submission. In order to support coordination of international surveillance efforts, we have partnered with the Public Health Alliance for Genomic Epidemiology to also provide a template conforming to its SARS-CoV-2 contextual data specification for use worldwide. Templates are also being developed for One Health and foodborne pathogens. Overall, the DataHarmonizer tool improves the effectiveness and fidelity of contextual data capture as well as its subsequent usability. Harmonization of contextual information across authorities, platforms and systems globally improves interoperability and reusability of data for concerted public health and research initiatives to fight the current pandemic and future public health emergencies. While initially developed for the COVID-19 pandemic, its expansion to other data management applications and pathogens is already underway.
Subject(s)
COVID-19 , Humans , COVID-19/epidemiology , Pandemics , SARS-CoV-2/genetics , Canada , Genomics/methodsABSTRACT
OBJECTIVES: The WHO's standardized measuring unit, "binding antibody units per milliliter (BAU/mL)," should allow the harmonization of quantitative results by different commercial Anti-SARS-CoV-2 immunoassays. However, multiple studies demonstrate inter-assay discrepancies. The antigenic changes of the Omicron variant affect the performance of Spike-specific immunoassays. This study evaluated the variation of quantitative Anti-SARS-CoV-2-Spike antibody measurements among 46, 50, and 44 laboratories in three rounds of a national external quality assessment (EQA) prior to and after the emergence of the Omicron variant in a diagnostic near-to-real-life setting. METHODS: We analyzed results reported by the EQA participant laboratories from single and sequential samples from SARS-CoV-2 convalescent, acutely infected, and vaccinated individuals, including samples obtained after primary and breakthrough infections with the Omicron variant. RESULTS: The three immunoassays most commonly used by the participants displayed a low intra-assay and inter-laboratory variation with excellent reproducibility using identical samples sent to the participants in duplicates. In contrast, the inter-assay variation was very high with all samples. Notably, the ratios of BAU/mL levels quantified by different immunoassays were not equal among all samples but differed between vaccination, past, and acute infection, including primary infection with the Omicron variant. The antibody kinetics measured in vaccinated individuals strongly depended on the applied immunoassay. CONCLUSIONS: Measured BAU/mL levels are only inter-changeable among different laboratories when the same assay was used for their assessment. Highly variable ratios of BAU/mL quantifications among different immunoassays and infection stages argue against the usage of universal inter-assay conversion factors.
ABSTRACT
The demand for testing during the coronavirus disease 2019 (COVID-19) pandemic has resulted in the production of several different commercial platforms and laboratory-developed assays for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This has created several challenges, including, but not limited to, the standardization of diagnostic testing, utilization of cycle threshold (CT) values for quantitation and clinical interpretation, and data harmonization. Using reference standards consisting of a linear range of SARS-CoV-2 concentrations quantitated by viral culture-based methods and droplet digital PCR, we investigated the commutability and standardization of SARS-CoV-2 quantitation across different laboratories in the United States. We assessed SARS-CoV-2 CT values generated on multiple reverse transcription-PCR (RT-PCR) platforms and analyzed PCR efficiencies, linearity, gene targets, and CT value agreement. Our results demonstrate the inappropriateness of using SARS-CoV-2 CT values without established standards for viral quantitation. Further, we emphasize the importance of using reference standards and controls validated to independent assays, to compare results across different testing platforms and move toward better harmonization of COVID-19 quantitative test results. IMPORTANCE From the onset of the COVID-19 pandemic, the demand for SARS-CoV-2 testing has resulted in an explosion of analytical tests with very different approaches and designs. The variability in testing modalities, compounded by the lack of available commercial reference materials for standardization early in the pandemic, has led to several challenges regarding data harmonization for viral quantitation. In this study, we assessed multiple commercially available RT-PCR platforms across different laboratories within the United States using standardized reference materials characterized by viral culture methods and droplet digital PCR. We observed variability in the results generated by different instruments and laboratories, further emphasizing the importance of utilizing validated reference standards for quantitation, to better harmonize SARS-CoV-2 test results.
Subject(s)
COVID-19 , Humans , United States , COVID-19/diagnosis , SARS-CoV-2/genetics , COVID-19 Testing , Pandemics , Clinical Laboratory Techniques/methods , Reference StandardsABSTRACT
BACKGROUND: The adverse impact of COVID-19 on marginalized and under-resourced communities of color has highlighted the need for accurate, comprehensive race and ethnicity data. However, a significant technical challenge related to integrating race and ethnicity data in large, consolidated databases is the lack of consistency in how data about race and ethnicity are collected and structured by health care organizations. OBJECTIVE: This study aims to evaluate and describe variations in how health care systems collect and report information about the race and ethnicity of their patients and to assess how well these data are integrated when aggregated into a large clinical database. METHODS: At the time of our analysis, the National COVID Cohort Collaborative (N3C) Data Enclave contained records from 6.5 million patients contributed by 56 health care institutions. We quantified the variability in the harmonized race and ethnicity data in the N3C Data Enclave by analyzing the conformance to health care standards for such data. We conducted a descriptive analysis by comparing the harmonized data available for research purposes in the database to the original source data contributed by health care institutions. To make the comparison, we tabulated the original source codes, enumerating how many patients had been reported with each encoded value and how many distinct ways each category was reported. The nonconforming data were also cross tabulated by 3 factors: patient ethnicity, the number of data partners using each code, and which data models utilized those particular encodings. For the nonconforming data, we used an inductive approach to sort the source encodings into categories. For example, values such as "Declined" were grouped with "Refused," and "Multiple Race" was grouped with "Two or more races" and "Multiracial." RESULTS: "No matching concept" was the second largest harmonized concept used by the N3C to describe the race of patients in their database. In addition, 20.7% of the race data did not conform to the standard; the largest category was data that were missing. Hispanic or Latino patients were overrepresented in the nonconforming racial data, and data from American Indian or Alaska Native patients were obscured. Although only a small proportion of the source data had not been mapped to the correct concepts (0.6%), Black or African American and Hispanic/Latino patients were overrepresented in this category. CONCLUSIONS: Differences in how race and ethnicity data are conceptualized and encoded by health care institutions can affect the quality of the data in aggregated clinical databases. The impact of data quality issues in the N3C Data Enclave was not equal across all races and ethnicities, which has the potential to introduce bias in analyses and conclusions drawn from these data. Transparency about how data have been transformed can help users make accurate analyses and inferences and eventually better guide clinical care and public policy.
ABSTRACT
BACKGROUND: The adverse impact of COVID-19 on marginalized and under-resourced communities of color has highlighted the need for accurate, comprehensive race and ethnicity data. However, a significant technical challenge related to integrating race and ethnicity data in large, consolidated databases is the lack of consistency in how data about race and ethnicity are collected and structured by health care organizations. OBJECTIVE: This study aims to evaluate and describe variations in how health care systems collect and report information about the race and ethnicity of their patients and to assess how well these data are integrated when aggregated into a large clinical database. METHODS: At the time of our analysis, the National COVID Cohort Collaborative (N3C) Data Enclave contained records from 6.5 million patients contributed by 56 health care institutions. We quantified the variability in the harmonized race and ethnicity data in the N3C Data Enclave by analyzing the conformance to health care standards for such data. We conducted a descriptive analysis by comparing the harmonized data available for research purposes in the database to the original source data contributed by health care institutions. To make the comparison, we tabulated the original source codes, enumerating how many patients had been reported with each encoded value and how many distinct ways each category was reported. The nonconforming data were also cross tabulated by 3 factors: patient ethnicity, the number of data partners using each code, and which data models utilized those particular encodings. For the nonconforming data, we used an inductive approach to sort the source encodings into categories. For example, values such as "Declined" were grouped with "Refused," and "Multiple Race" was grouped with "Two or more races" and "Multiracial." RESULTS: "No matching concept" was the second largest harmonized concept used by the N3C to describe the race of patients in their database. In addition, 20.7% of the race data did not conform to the standard; the largest category was data that were missing. Hispanic or Latino patients were overrepresented in the nonconforming racial data, and data from American Indian or Alaska Native patients were obscured. Although only a small proportion of the source data had not been mapped to the correct concepts (0.6%), Black or African American and Hispanic/Latino patients were overrepresented in this category. CONCLUSIONS: Differences in how race and ethnicity data are conceptualized and encoded by health care institutions can affect the quality of the data in aggregated clinical databases. The impact of data quality issues in the N3C Data Enclave was not equal across all races and ethnicities, which has the potential to introduce bias in analyses and conclusions drawn from these data. Transparency about how data have been transformed can help users make accurate analyses and inferences and eventually better guide clinical care and public policy.
ABSTRACT
Background: There is an urgent need for harmonization between severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) serology platforms and assays prior to defining appropriate correlates of protection and as well inform the development of new rapid diagnostic tests that can be used for serosurveillance as new variants of concern (VOC) emerge. We compared multiple SARS-CoV-2 serology reference materials to the WHO International Standard (WHO IS) to determine their utility as secondary standards, using an international network of laboratories with high-throughput quantitative serology assays. This enabled the comparison of quantitative results between multiple serology platforms. Methods: Between April and December 2020, 13 well-characterized and validated SARS-CoV-2 serology reference materials were recruited from six different providers to qualify as secondary standards to the WHO IS. All the samples were tested in parallel with the National Institute for Biological Standards and Control (NIBSC) 20/136 and parallel-line assays were used to calculate the relevant potency and binding antibody units. Results: All the samples saw varying levels of concordance between diagnostic methods at specific antigen-antibody combinations. Seven of the 12 candidate materials had high concordance for the spike-immunoglobulin G (IgG) analyte [percent coefficient of variation (%CV) between 5 and 44%]. Conclusion: Despite some concordance between laboratories, qualification of secondary materials to the WHO IS using arbitrary international units or binding antibody units per milliliter (BAU/ml) does not provide any benefit to the reference materials overall, due to the lack of consistent agreeable international unit (IU) or BAU/ml conversions between laboratories. Secondary standards should be qualified to well-characterized reference materials, such as the WHO IS, using serology assays that are similar to the ones used for the original characterization of the WHO IS.
ABSTRACT
Harmonizing measures across studies can facilitate comparisons and strengthen the science, but procedures for establishing common data elements are rarely documented. We detail a rigorous, 2-year process to harmonize measures across the Prevention And Treatment through a Comprehensive Care Continuum for HIV-affected Adolescents in Resource Constrained Settings (PATC3H) consortium, consisting of eight federally-funded studies. We created a repository of measured constructs from each study, classified and selected constructs for harmonization, and identified survey instruments. Measures were harmonized for implementation science, HIV prevention and care, demographics and sexual behavior, mental health and substance use, and economic assessment. Importantly, we present our harmonized implementation science constructs. A common set of implementation science constructs have yet to be recommended in the literature for low-to-middle-income countries despite increasing recognition of their importance to delivering and scaling up effective interventions. Drawing on RE-AIM (Reach Effectiveness Adoption Implementation Maintenance) and the Implementation Outcomes Framework, items were harmonized for staff/administrators and study participants to measure reach, adoption, implementation, maintenance, feasibility, acceptability, appropriateness, and fidelity. The process undertaken to harmonize measures and the codified set of implementation science measures developed by our consortium can inform future data harmonization efforts, critical to strengthening the replication and generalizability of findings while facilitating collaborative research-especially in resource-limited settings. We conclude with recommendations for research consortia, namely ensuring representation from all study teams and research priorities; adopting a flexible, transparent, and systematic approach; strengthening the literature on implementation science harmonization; and being responsive to life events (e.g., COVID-19). Supplementary Information: The online version contains supplementary material available at 10.1007/s43477-022-00042-7.
ABSTRACT
In October 2020, the National Cancer Institute (NCI) Serological Sciences Network (SeroNet) was established to study the immune response to COVID-19, and "to develop, validate, improve, and implement serological testing and associated technologies" (https://www.cancer.gov/research/key-initiatives/covid-19/coronavirus-research-initiatives/serological-sciences-network). SeroNet is comprised of 25 participating research institutions partnering with the Frederick National Laboratory for Cancer Research (FNLCR) and the SeroNet Coordinating Center. Since its inception, SeroNet has supported collaborative development and sharing of COVID-19 serological assay procedures and has set forth plans for assay harmonization. To facilitate collaboration and procedure sharing, a detailed survey was sent to collate comprehensive assay details and performance metrics on COVID-19 serological assays within SeroNet. In addition, FNLCR established a protocol to calibrate SeroNet serological assays to reference standards, such as the U.S. severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) serology standard reference material and first WHO international standard (IS) for anti-SARS-CoV-2 immunoglobulin (20/136), to facilitate harmonization of assay reporting units and cross-comparison of study data. SeroNet institutions reported development of a total of 27 enzyme-linked immunosorbent assay (ELISA) methods, 13 multiplex assays, and 9 neutralization assays and use of 12 different commercial serological methods. FNLCR developed a standardized protocol for SeroNet institutions to calibrate these diverse serological assays to reference standards. In conclusion, SeroNet institutions have established a diverse array of COVID-19 serological assays to study the immune response to SARS-CoV-2 and vaccines. Calibration of SeroNet serological assays to harmonize results reporting will facilitate future pooled data analyses and study cross-comparisons. IMPORTANCE SeroNet institutions have developed or implemented 61 diverse COVID-19 serological assays and are collaboratively working to harmonize these assays using reference materials to establish standardized reporting units. This will facilitate clinical interpretation of serology results and cross-comparison of research data.
Subject(s)
COVID-19 , Antibodies, Viral , COVID-19/diagnosis , COVID-19 Testing , Humans , SARS-CoV-2 , Serologic Tests/methodsABSTRACT
This paper extends frequency domain quantitative electroencephalography (qEEG) methods pursuing higher sensitivity to detect Brain Developmental Disorders. Prior qEEG work lacked integration of cross-spectral information omitting important functional connectivity descriptors. Lack of geographical diversity precluded accounting for site-specific variance, increasing qEEG nuisance variance. We ameliorate these weaknesses. (i) Create lifespan Riemannian multinational qEEG norms for cross-spectral tensors. These norms result from the HarMNqEEG project fostered by the Global Brain Consortium. We calculate the norms with data from 9 countries, 12 devices, and 14 studies, including 1564 subjects. Instead of raw data, only anonymized metadata and EEG cross-spectral tensors were shared. After visual and automatic quality control, developmental equations for the mean and standard deviation of qEEG traditional and Riemannian DPs were calculated using additive mixed-effects models. We demonstrate qEEG "batch effects" and provide methods to calculate harmonized z-scores. (ii) We also show that harmonized Riemannian norms produce z-scores with increased diagnostic accuracy predicting brain dysfunction produced by malnutrition in the first year of life and detecting COVID induced brain dysfunction. (iii) We offer open code and data to calculate different individual z-scores from the HarMNqEEG dataset. These results contribute to developing bias-free, low-cost neuroimaging technologies applicable in various health settings.