Accuracy of quantitative viral secondary standards: a re-examination.

Interlaboratory agreement of viral load assays depends on the accuracy and uniformity of quantitative calibrators. Previous work demonstrated poor agreement of secondary cytomegalovirus (CMV) standards with nominal values. This study re-evaluated this issue among commercially produced secondary standards for both BK virus (BKV) and CMV, using digital polymerase chain reaction (dPCR) to compare the materials from three different manufacturers. Overall, standards showed an improved agreement compared to prior work, against nominal values in both log10 copies/mL and log10 international unit (IU)/mL, with bias from manufacturer-assigned nominal values of 0.0-0.9 log10 units (either copies or IU)/mL. Standards normalized to IU and those values assigned by dPCR rather than by real-time PCR (qPCR) showed better agreement with nominal values. The latter reinforces prior conclusions regarding the utility of using such methods for quantitative value assignment in reference materials. Quantitative standards have improved over the last several years, and the remaining bias from nominal values might be further reduced by universal implementation of dPCR methods for value assignment, normalized to IU. IMPORTANCE: Interlaboratory agreement of viral load assays depends on accuracy and uniformity of quantitative calibrators. Previous work, published in JCM several years ago, demonstrated poor agreement of secondary cytomegalovirus (CMV) standards with nominal values. This study re-evaluated this issue among commercially produced secondary standards for both BK virus (BKV) and CMV, using digital polymerase chain reaction (dPCR) to compare the materials from three different manufacturers. Overall, standards showed an improved agreement compared to prior work, against nominal values, indicating a substantial improvement in the production of accurate secondary viral standards, while supporting the need for further work in this area and for the broad adaption of international unit (IU) as a reporting standard for quantitative viral load results.

A prospective bacterial whole-genome-sequencing-based surveillance programme for comprehensive early detection of healthcare-associated infection transmission in paediatric oncology patients.

BACKGROUND: Bacterial whole-genome sequencing (WGS) and determination of genetic relatedness is an important tool for investigation of epidemiologically suspected outbreaks. AIM: This prospective cohort study evaluated a comprehensive, prospective bacterial WGS-based surveillance programme for early detection of transmission of most bacterial pathogens among patients at a paediatric oncology hospital. METHODS: Cultured bacterial isolates from clinical diagnostic specimens collected prospectively from both inpatient and outpatient encounters between January 2019 and December 2021 underwent routine WGS and core genome multi-locus sequence typing to determine isolates' relatedness. Previously collected isolates from January to December 2018 were retrospectively analysed for identification of prior or ongoing transmission. Multi-patient clusters were investigated to identify potential transmission events based on temporal and spatial epidemiological links and interventions were introduced. FINDINGS: A total of 1497 bacterial isolates from 1025 patients underwent WGS. A total of 259 genetically related clusters were detected, of which 18 (6.9%) multi-patient clusters involving 38 (3.7%) patients were identified. Sixteen clusters involved two patients each, and two clusters involved three patients. Following investigation, epidemiologically plausible transmission links were identified in five (27.8%) multi-patient clusters. None of the multi-patient clusters were suspected by conventional epidemiological surveillance. CONCLUSION: Bacterial WGS-based surveillance for early detection of hospital transmission detected several limited multi-patient clusters that were unrecognized by conventional epidemiological methods. Genomic surveillance helped efficiently focus interventions while reducing unnecessary investigations.

Matrix Matters: Assessment of Commutability among BK Virus Assays and Standards.

Quantitative testing of BK virus (BKPyV) nucleic acid has become the standard of care in transplant patients. While the relationship between interassay harmonization and commutability has been well characterized for other transplant-related viruses, it has been less well studied for BKPyV, particularly regarding differences in commutability between matrices. Here, interassay agreement was evaluated among six real-time nucleic acid amplification tests (NAATs) and one digital PCR (dPCR) BKPyV assay. Differences in the commutability of three quantitative standards was examined across all assays using a variety of statistical approaches. Panels, including 40 samples each of plasma and urine samples previously positive for BKPyV, together with one previously negative plasma sample and four previously negative urine samples, were tested using all assays, with each real-time NAAT utilizing its usual quantitative calibrators. Serial dilutions of WHO, National Institute for Standards and Technology (NIST), and commercially produced (Exact/Bio-Rad) reference materials were also run by each assay as unknowns. The agreement of the clinical sample values was assessed as a group and in a pairwise manner. The commutability was estimated using both relativistic and quantitative means. The quantitative agreement across assays in the urine samples was within a single log10 unit across all assays, while the results from the plasma samples varied by 2 to 3 log10 IU/mL. The commutability showed a similar disparity between the matrices. Recalibration using international standards diminished the resulting discrepancies in some but not all cases. Differences in the sample matrix can affect the commutability and interassay agreement of quantitative BKPyV assays. Differences in commutability between matrices may largely be due to factors other than those such as amplicon size, previously described as important in the case of cytomegalovirus. Continued efforts to standardize viral load measurements must address multiple sources of variability and account for differences in assay systems, quantitative standards, and sample matrices.

Comparative Evaluation of Assays for Broad Detection of Molecular Resistance Mechanisms in Enterobacterales Isolates.

Rapid detection of antimicrobial resistance in both surveillance and diagnostic settings is still a major challenge for the clinical lab, compounded by the rapid evolution of antibiotic resistance mechanisms. This study compares four methods for the broad detection of antibiotic resistance genes in Enterobacterales isolates: two multiplex PCR assays (the Streck ARM-D beta-lactamase kit and the OpGen Acuitas AMR Gene Panel u5.47 (research use only [RUO]) and one microarray assay (the Check-MDR CT103XL assay), with whole-genome sequencing as a reference standard. A total of 65 Gram-negative bacterial isolates, from 56 patients, classified by phenotypic antimicrobial susceptibility testing (AST) as showing resistance to beta-lactam antimicrobials (extended-spectrum beta-lactamase [ESBL] positive or resistance to third-generation cephalosporins or carbapenems) were included in the study. Overall concordance between the molecular assays and sequencing was high. While all three assays had similar performance, the OpGen Acuitas AMR assay had the highest overall percent concordance with sequencing results. The primary differences between the assays tested were the number and diversity of targets, ranging from 9 for Streck to 34 for OpGen. This study shows that commercially available PCR-based assays can provide accurate identification of antimicrobial resistance loci in clinically significant Gram-negative bacteria. Further studies are needed to determine the clinical diagnostic role and potential benefit of such methods.

Accuracy of Broad-Panel PCR-Based Bacterial Identification for Blood Cultures in a Pediatric Oncology Population.

Bloodstream infections are a major cause of morbidity and mortality and result in significant costs to health care systems. Rapid identification of the causative agent of bloodstream infections is critical for patient treatment and improved outcomes. Multiplex PCR systems that provide bacterial identification directly from the blood culture bottle allow for earlier detection of pathogens. The GenMark Dx ePlex blood culture identification (BCID) panels have an expanded number of targets for both identification and genotypic markers of antimicrobial resistance. The performance of the ePlex BCID Gram-negative (GN) and Gram-positive (GP) panels were evaluated in a predominantly pediatric oncology population. A total of 112 blood cultures were tested by the ePlex BCID GN and GP panels and results were compared to those from standard-of-care testing. Accuracy for on-panel organisms was 89% (CI, 76% to 95%) for the Gram-positive panel, with four misidentifications and one not detected, and 93% (CI, 82% to 98%) for the Gram-negative panel, with two misidentifications and one not detected. The results showed good overall performance of these panels for rapid, accurate detection of bloodstream pathogens in this high-risk population. IMPORTANCE Bloodstream infections are a major cause of morbidity and mortality and result in significant costs to health care systems. Rapid identification of the causative agent of bloodstream infections is critical for patient treatment and improved outcomes. Multiplex PCR systems that provide bacterial identification directly from the blood culture bottle allow for earlier characterization of pathogens. The GenMark Dx ePlex blood culture identification (BCID) panels, recently cleared by the FDA, have an expanded number of targets for both identification and resistance, much larger than other, automated, broad-panel PCR assays. The performance of the ePlex BCID Gram-negative and Gram-positive panels was evaluated in a predominantly pediatric oncology population, providing a unique look at its performance in a high-risk group, where rapid diagnostic information for bloodstream infections could be of particular value for clinical care providers.

Persistent Challenges of Interassay Variability in Transplant Viral Load Testing.

While quantification of viruses that cause important infections in transplant recipients has been the standard of care for years, important challenges related to standardization remain. The issues are wide ranging, and until they are adequately addressed, the full impact of viral load testing regarding clinical management decisions will not be realized. This review focuses on a broad array of problems, including the lack of available FDA-approved/cleared tests, limited uptake of international standards, accurate quantification of secondary standards, specific assay characteristics, and commutability. Though some of these topics are nuanced, taken together they greatly influence the clinical utility of testing. For example, it has not been possible to define thresholds that predict the risk of developing disease and determine significant changes in serial viral load values for a given patient. Moreover, the utility of international guidelines may be limited due to the lack of a standardized assay. By summarizing the issues, the hope is that commercial companies, regulatory agencies, and professional societies can come together to advance the field and solve these problems.

Impact of Fragmentation on Commutability of Epstein-Barr Virus and Cytomegalovirus Quantitative Standards.

Despite the adaptation of international standards, quantitative viral load testing of transplant-associated viruses continues to be limited by interlaboratory disagreement. Studies have suggested that this disagreement and the poor commutability of standards may, in some cases, be linked to amplicon size and the fragmentation of circulating viral DNA. We evaluated target fragmentation as a cause of noncommutability and pretest fragmentation of quantitative standards as a potential means of increasing commutability and interassay agreement. Forty-two cytomegalovirus (CMV)-positive and 41 Epstein-Barr virus (EBV)-positive plasma samples, together with two different quantitative standards for each virus, were tested as unknowns using 10 different quantitative PCR assays at 5 different laboratories. Standards were tested both intact and after intentional fragmentation by ultrasonication. Quantitative agreement between methods was assessed, together with commutability, using multiple statistical approaches. Most assays yielded results within 0.5 log10 IU/ml of the mean for CMV, while for EBV a greater variability of up to 1.5 log10 IU/ml of the mean was shown. Commutability showed marked improvement following fragmentation of both CMV standards but not after fragmentation of the EBV standards. These findings confirm the impact of amplicon size and target fragmentation on commutability for CMV and suggest that for some (but not all) viruses, interlaboratory harmonization can be improved through the use of fragmented quantitative standards.

Cost-Effective Respiratory Virus Testing.

The timely and accurate diagnosis of respiratory virus infections has the potential to optimize downstream (posttesting) use of limited health care resources, including antibiotics, antivirals, ancillary testing, and inpatient and emergency department beds. Cost-effective algorithms for respiratory virus testing must take into consideration numerous factors, including which patients should be tested, what testing should be performed (for example, antigen testing versus reverse transcription-PCR testing or influenza A/B testing versus testing with a comprehensive respiratory virus panel), and the turnaround time necessary to achieve the desired posttesting outcomes. Despite the clinical impact of respiratory virus infections, the cost-effectiveness of respiratory virus testing is incompletely understood. In this article, we review the literature pertaining to the cost-effectiveness of respiratory virus testing in pediatric and adult patient populations, in emergency department, outpatient, and inpatient clinical settings. Furthermore, we consider the cost-effectiveness of a variety of testing methods, including rapid antigen tests, direct fluorescent antibody assays, and nucleic acid amplification tests.

A Comprehensive Statistical Framework for Determination of Commutability, Accuracy, and Agreement in Clinical DNAemia Assays.

Despite advances in the standardization of quantitative DNAemia tests and efforts to better understand and characterize the performance of reference materials in different assays, it remains unclear how the commutability performance of reference materials is related to intra- and interassay agreement. Building upon previous work, we describe a comprehensive framework to determine the relationship of commutability with assay accuracy and agreement. The use of this framework is illustrated using previously generated data regarding the performance of four quantitative Epstein-Bar virus (EBV) PCR assays with the WHO and ABI standards as examples. The use of these statistical tools can link the performance characteristics of one or more assays with predetermined clinical decision limits and may help improve the development, validation, and clinical utility of such DNAemia tests.

Whole-Genome Characterization of Bacillus cereus Associated with Specific Disease Manifestations.

Bacillus cereus remains an important cause of infections, particularly in immunocompromised hosts. While typically associated with enteric infections, disease manifestations can be quite diverse and include skin infections, bacteremia, pneumonia, and meningitis. Whether there are any genetic correlates of bacterial strains with particular clinical manifestations remains unknown. To address this gap in understanding, we undertook whole-genome analysis of B. cereus strains isolated from patients with a range of disease manifestations, including noninvasive colonizing disease, superficial skin infections, and invasive bacteremia. Interestingly, strains involved in skin infection tended to form a distinct genetic cluster compared to isolates associated with invasive disease. Other disease manifestations, despite not being exclusively clustered, nonetheless had unique genetic features. The unique features associated with the specific types of infections ranged from traditional virulence determinants to metabolic pathways and gene regulators. These data represent the largest genetic analysis to date of pathogenic B. cereus isolates with associated clinical parameters.

Evaluation of rapid phenotypic identification and antimicrobial susceptibility testing in a pediatric oncology center.

Identification (ID) and antimicrobial susceptibility testing (AST) remain rate limiting steps in producing actionable data for clinical care of bloodstream infections. Rapid, automated phenotypic ID and AST by fluorescent in situ hybridization and automated microscopy were used to characterize blood stream infections in a predominantly pediatric oncology patient population. Results were compared to standard of care (SOC) phenotypic methods. The Accelerate Pheno System (AXDX) had a sensitivity of 91.2% and an accuracy of 100% to the genus level for identification, and an overall categorical agreement 91.2-91.8% for susceptibility, depending on the breakpoints used. The AXDX required a mean time of 1.4hours for identification and 6.6hours for susceptibility testing compared to SOC, requiring 32.5 and 46.7hours, respectively. Identification and susceptibility by rapid phenotypic methods shows a high degree of accuracy; the marked reduction in time to results may have significant implications for patient care.

Infection-related complications during treatment for childhood acute lymphoblastic leukemia.

Background: Comprehensive studies on neutropenia and infection-related complications in patients with acute lymphoblastic leukemia (ALL) are lacking. Patients and methods: We evaluated infection-related complications that were grade ≥3 on National Cancer Institute's Common Terminology Criteria for Adverse Events (version 3.0) and their risk factors in 409 children with newly diagnosed ALL throughout the treatment period. Results: Of the 2420 infection episodes, febrile neutropenia and clinically or microbiologically documented infection were seen in 1107 and 1313 episodes, respectively. Among documented infection episodes, upper respiratory tract was the most common site (n = 389), followed by ear (n = 151), bloodstream (n = 147), and gastrointestinal tract (n = 145) infections. These episodes were more common during intensified therapy phases such as remission induction and reinduction, but respiratory and ear infections, presumably viral in origin, also occurred during continuation phases. The 3-year cumulative incidence of infection-related death was low (1.0±0.9%, n = 4), including 2 from Bacillus cereus bacteremia. There was no fungal infection-related mortality. Age 1-9.9 years at diagnosis was associated with febrile neutropenia (P = 0.002) during induction and febrile neutropenia and documented infection (both P < 0.001) during later continuation. White race was associated with documented infection (P = 0.034) during induction. Compared with low-risk patients, standard- and high-risk patients received more intensive therapy during early continuation and had higher incidences of febrile neutropenia (P < 0.001) and documented infections (P = 0.043). Furthermore, poor neutrophil surge after dexamethasone pulses during continuation, which can reflect the poor bone marrow reserve, was associated with infections (P < 0.001). Conclusions: The incidence of infection-related death was low. However, young age, white race, intensive chemotherapy, and lack of neutrophil surge after dexamethasone treatment were associated with infection-related complications. Close monitoring for prompt administration of antibiotics and modification of chemotherapy should be considered in these patients.

Progress in Quantitative Viral Load Testing: Variability and Impact of the WHO Quantitative International Standards.

It has been hoped that the recent availability of WHO quantitative standards would improve interlaboratory agreement for viral load testing; however, insufficient data are available to evaluate whether this has been the case. Results from 554 laboratories participating in proficiency testing surveys for quantitative PCR assays of cytomegalovirus (CMV), Epstein-Barr virus (EBV), BK virus (BKV), adenovirus (ADV), and human herpesvirus 6 (HHV6) were evaluated to determine overall result variability and then were stratified by assay manufacturer. The impact of calibration to international units/ml (CMV and EBV) on variability was also determined. Viral loads showed a high degree of interlaboratory variability for all tested viruses, with interquartile ranges as high as 1.46 log10 copies/ml and the overall range for a given sample up to 5.66 log10 copies/ml. Some improvement in result variability was seen when international units were adopted. This was particularly the case for EBV viral load results. Variability in viral load results remains a challenge across all viruses tested here; introduction of international quantitative standards may help reduce variability and does so more or less markedly for certain viruses.

Comparative Performance of Reagents and Platforms for Quantitation of Cytomegalovirus DNA by Digital PCR.

A potential benefit of digital PCR is a reduction in result variability across assays and platforms. Three sets of PCR reagents were tested on two digital PCR systems (Bio-Rad and RainDance), using three different sets of PCR reagents for quantitation of cytomegalovirus (CMV). Both commercial quantitative viral standards and 16 patient samples (n = 16) were tested. Quantitative accuracy (compared to nominal values) and variability were determined based on viral standard testing results. Quantitative correlation and variability were assessed with pairwise comparisons across all reagent-platform combinations for clinical plasma sample results. The three reagent sets, when used to assay quantitative standards on the Bio-Rad system, all showed a high degree of accuracy, low variability, and close agreement with one another. When used on the RainDance system, one of the three reagent sets appeared to have a much better correlation to nominal values than did the other two. Quantitative results for patient samples showed good correlation in most pairwise comparisons, with some showing poorer correlations when testing samples with low viral loads. Digital PCR is a robust method for measuring CMV viral load. Some degree of result variation may be seen, depending on platform and reagents used; this variation appears to be greater in samples with low viral load values.

Quantitative Assessment of Commutability for Clinical Viral Load Testing Using a Digital PCR-Based Reference Standard.

Given recent advances in the development of quantitative standards, particularly WHO international standards, efforts to better understand the commutability of reference materials have been made. Existing approaches in evaluating commutability include prediction intervals and correspondence analysis; however, the results obtained from existing approaches may be ambiguous. We have developed a "deviation-from-ideal" (DFI) approach to evaluate commutability of standards and applied it to the assessment of Epstein-Bar virus (EBV) load testing in four quantitative PCR assays, treating digital PCR as a reference assay. We then discuss advantages and limitations of the DFI approach as well as experimental design to best evaluate the commutability of an assay in practice.

Commutability of the First World Health Organization International Standard for Human Cytomegalovirus.

Quantitative detection of cytomegalovirus (CMV) DNA has become a standard part of care for many groups of immunocompromised patients; recent development of the first WHO international standard for human CMV DNA has raised hopes of reducing interlaboratory variability of results. Commutability of reference material has been shown to be necessary if such material is to reduce variability among laboratories. Here we evaluated the commutability of the WHO standard using 10 different real-time quantitative CMV PCR assays run by eight different laboratories. Test panels, including aliquots of 50 patient samples (40 positive samples and 10 negative samples) and lyophilized CMV standard, were run, with each testing center using its own quantitative calibrators, reagents, and nucleic acid extraction methods. Commutability was assessed both on a pairwise basis and over the entire group of assays, using linear regression and correspondence analyses. Commutability of the WHO material differed among the tests that were evaluated, and these differences appeared to vary depending on the method of statistical analysis used and the cohort of assays included in the analysis. Depending on the methodology used, the WHO material showed poor or absent commutability with up to 50% of assays. Determination of commutability may require a multifaceted approach; the lack of commutability seen when using the WHO standard with several of the assays here suggests that further work is needed to bring us toward true consensus.

Comparative evaluation of three commercial quantitative cytomegalovirus standards by use of digital and real-time PCR.

The recent development of the 1st WHO International Standard for human cytomegalovirus (CMV) and the introduction of commercially produced secondary standards have raised hopes of improved agreement among laboratories performing quantitative PCR for CMV. However, data to evaluate the trueness and uniformity of secondary standards and the consistency of results achieved when these materials are run on various assays are lacking. Three concentrations of each of the three commercially prepared secondary CMV standards were tested in quadruplicate by three real-time and two digital PCR methods. The mean results were compared in a pairwise fashion with nominal values provided by each manufacturer. The agreement of results among all methods for each sample and for like concentrations of each standard was also assessed. The relationship between the nominal values of standards and the measured values varied, depending upon the assay used and the manufacturer of the standards, with the degree of bias ranging from +0.6 to -1.0 log10 IU/ml. The mean digital PCR result differed significantly among the secondary standards, as did the results of the real-time PCRs, particularly when plotted against nominal log10 IU values. Commercially available quantitative secondary CMV standards produce variable results when tested by different real-time and digital PCR assays, with various magnitudes of bias compared to nominal values. These findings suggest that the use of such materials may not achieve the intended uniformity among laboratories measuring CMV viral load, as envisioned by adaptation of the WHO standard.