NGS for (Hemato-) Oncology in Belgium: Evaluation of Laboratory Performance and Feasibility of a National External Quality Assessment Program.

Next-generation sequencing (NGS) is being integrated into routine clinical practice in the field of (hemato-) oncology to search for variants with diagnostic, prognostic, or therapeutic value at potentially low allelic frequencies. The complex sequencing workflows used require careful validation and continuous quality control. Participation in external quality assessments (EQA) helps laboratories evaluate their performance and guarantee the validity of tests results with the ultimate goal of ensuring high-quality patient care. Here, we describe three benchmarking trials performed during the period 2017-2018 aiming firstly at establishing the state-of-the-art and secondly setting up a NGS-specific EQA program at the national level in the field of clinical (hemato-) oncology in Belgium. DNA samples derived from cell line mixes and artificially mutated cell lines, designed to carry variants of clinical relevance occurring in solid tumors, hematological malignancies, and BRCA1/BRCA2 genes, were sent to Belgian human genetics, anatomic pathology, and clinical biology laboratories, to be processed following routine practices, together with surveys covering technical aspects of the NGS workflows. Despite the wide variety of platforms and workflows currently applied in routine clinical practice, performance was satisfactory, since participating laboratories identified the targeted variants with success rates ranging between 93.06% and 97.63% depending on the benchmark, and few false negative or repeatability issues were identified. However, variant reporting and interpretation varied, underlining the need for further standardization. Our approach showcases the feasibility of developing and implementing EQA for routine clinical practice in the field of (hemato-) oncology, while highlighting the challenges faced.

Application of the characteristic function to evaluate and compare analytical variability in an external quality assessment scheme for serum ethanol.

BACKGROUND: As a cornerstone of quality management in the laboratory, External Quality Assessment (EQA) schemes are used to assess laboratory and analytical method performance. The characteristic function is used to describe the relation between the target concentration and the EQA standard deviation, which is an essential part of the evaluation process. The characteristic function is also used to compare the variability of different analytical methods. METHODS: We fitted the characteristic function to data from the Belgian External Quality Assessment program for serum ethanol. Data included results from headspace gas chromatography and the enzymatic methods of Abbott, Roche, Siemens, and Ortho-Clinical Diagnostics. We estimated the characteristic function with weighted nonlinear regression. By introducing dummy variables, we rewrote the original formula of the characteristic function to assess statistical inference for comparing the variability of the different analytical methods. RESULTS: The characteristic function fitted the data precisely. Comparison between methods showed that there was little difference between the estimated variability for low concentrations, and that the increase in SD with increasing target concentration was slower for Abbott and Roche than for the other methods. CONCLUSIONS: The characteristic function can successfully be introduced in clinical schemes, although its applicability to fit the data should always be assessed. Because of its easy parameterization, it can be used to assess differences in performance between analytical methods and to assess laboratory performance. The characteristic function also offers an alternative framework for coefficients of variation to describe variability of analytical methods.

Influence of dabigatran and rivaroxaban on routine coagulation assays. A nationwide Belgian survey.

The Belgian national External Quality Assessment Scheme performed a nationwide survey using lyophilised plasma samples spiked with dabigatran or rivaroxaban to demonstrate to the Belgian clinical laboratories how these drugs affect their routine coagulation assays prothrombin time (PT), activated partial thromboplastin time (aPTT), fibrinogen and antithrombin. Virtually all Belgian laboratories performing routine coagulation testing (189/192) participated in the survey. Both, dabigatran and rivaroxaban significantly prolonged the PT and aPTT in a concentration- and reagent-dependent manner. PT reagents were more influenced by rivaroxaban than by dabigatran and aPTT reagents more influenced by dabigatran than by rivaroxaban. Among PT reagents, Neoplastin R® was the most sensitive to rivaroxaban and Innovin® and Thromborel S® the least sensitive. Converting PT results to INR only increased the variability between reagents. Among aPTT reagents, Actin FSL® was the least sensitive to dabigatran while the other aPTT reagents showed slightly higher sensitivities. The presence of dabigatran led to falsely reduced fibrinogen concentrations when measured with a low thrombin concentration reagent. The presence of dabigatran caused an overestimation of the antithrombin level when measured with a thrombin-based activity assay and the presence of rivaroxaban an overestimation of the antithrombin level when measured with a FXa-based assay. Instrument-related differences were found for all tested parameters. In conclusion, this paper provides detailed information on the effect of dabigatran and rivaroxaban on routine coagulation assays as performed with a large number of reagent/instrument combinations.

Comparison of different approaches to evaluate External Quality Assessment Data.

In EQA programs, Z-scores are used to evaluate laboratory performance. They should indicate poorly performing laboratories, regardless of the presence of outliers. For this, two different types of approaches exist. The first type are "outlier-based" approaches, which first exclude outlying values, calculate the average and standard deviation on the remaining data and obtain Z-scores for all values (e.g., Grubbs and Dixon). The second type includes the "robust" approaches (e.g., Tukey and Qn or the algorithm recommended by ISO). The different approaches were assessed by randomly generated samples from the Normal and Student t distributions. Part of the sample data were contaminated with outliers. The number of false and true outliers was recorded and subsequently, Positive and Negative Predictive Values were derived. Also, the sampling mean and variability were calculated for location and scale estimators. The various approaches performed similarly for sample sizes above 10 and when outliers were at good distance from the centre. For smaller sample sizes and closer outliers, however, the approaches performed quite differently. Tukey's method was characterised by a high true and a high false outlier rate, while the ISO and Qn approaches demonstrated weak performance. Grubbs test yielded overall the best results.