Towards radiologist-level cancer risk assessment in CT lung screening using deep learning.

PURPOSE: Lung cancer is the leading cause of cancer mortality in the US, responsible for more deaths than breast, prostate, colon and pancreas cancer combined and large population studies have indicated that low-dose computed tomography (CT) screening of the chest can significantly reduce this death rate. Recently, the usefulness of Deep Learning (DL) models for lung cancer risk assessment has been demonstrated. However, in many cases model performances are evaluated on small/medium size test sets, thus not providing strong model generalization and stability guarantees which are necessary for clinical adoption. In this work, our goal is to contribute towards clinical adoption by investigating a deep learning framework on larger and heterogeneous datasets while also comparing to state-of-the-art models. METHODS: Three low-dose CT lung cancer screening datasets were used: National Lung Screening Trial (NLST, n = 3410), Lahey Hospital and Medical Center (LHMC, n = 3154) data, Kaggle competition data (from both stages, n = 1397 + 505) and the University of Chicago data (UCM, a subset of NLST, annotated by radiologists, n = 132). At the first stage, our framework employs a nodule detector; while in the second stage, we use both the image context around the nodules and nodule features as inputs to a neural network that estimates the malignancy risk for the entire CT scan. We trained our algorithm on a part of the NLST dataset, and validated it on the other datasets. Special care was taken to ensure there was no patient overlap between the train and validation sets. RESULTS AND CONCLUSIONS: The proposed deep learning model is shown to: (a) generalize well across all three data sets, achieving AUC between 86% to 94%, with our external test-set (LHMC) being at least twice as large compared to other works; (b) have better performance than the widely accepted PanCan Risk Model, achieving 6 and 9% better AUC score in our two test sets; (c) have improved performance compared to the state-of-the-art represented by the winners of the Kaggle Data Science Bowl 2017 competition on lung cancer screening; (d) have comparable performance to radiologists in estimating cancer risk at a patient level.

High fidelity system modeling for high quality image reconstruction in clinical CT.

Today, while many researchers focus on the improvement of the regularization term in IR algorithms, they pay less concern to the improvement of the fidelity term. In this paper, we hypothesize that improving the fidelity term will further improve IR image quality in low-dose scanning, which typically causes more noise. The purpose of this paper is to systematically test and examine the role of high-fidelity system models using raw data in the performance of iterative image reconstruction approach minimizing energy functional. We first isolated the fidelity term and analyzed the importance of using focal spot area modeling, flying focal spot location modeling, and active detector area modeling as opposed to just flying focal spot motion. We then compared images using different permutations of all three factors. Next, we tested the ability of the fidelity terms to retain signals upon application of the regularization term with all three factors. We then compared the differences between images generated by the proposed method and Filtered-Back-Projection. Lastly, we compared images of low-dose in vivo data using Filtered-Back-Projection, Iterative Reconstruction in Image Space, and the proposed method using raw data. The initial comparison of difference maps of images constructed showed that the focal spot area model and the active detector area model also have significant impacts on the quality of images produced. Upon application of the regularization term, images generated using all three factors were able to substantially decrease model mismatch error, artifacts, and noise. When the images generated by the proposed method were tested, conspicuity greatly increased, noise standard deviation decreased by 90% in homogeneous regions, and resolution also greatly improved. In conclusion, the improvement of the fidelity term to model clinical scanners is essential to generating higher quality images in low-dose imaging.

Sinogram-affirmed iterative reconstruction of low-dose chest CT: effect on image quality and radiation dose.

OBJECTIVE. The purpose of this study is to compare sinogram-affirmed iterative reconstruction (SAFIRE) and filtered back projection (FBP) reconstruction of chest CT acquired with 65% radiation dose reduction. MATERIALS AND METHODS. In this prospective study involving 24 patients (11 women and 13 men; mean [± SD] age, 66 ± 10 years), two scan series were acquired using 100 and 40 Quality Reference mAs over a 10-cm scan length in the chest with a 128-MDCT scanner. The 40 Quality Reference mAs CT projection data were reconstructed with FBP and four settings of SAFIRE (S1, S2, S3, and S4). Six image datasets (FBP with 100 and 40 Quality Reference mAs, and S1, S2, S3, S4 with 40 Quality Reference mAs) were displayed on a DICOM-compliant 55-inch 2-megapixel monitor for blinded evaluation by two thoracic radiologists for number and location of lesions, lesion size, lesion margins, visibility of small structures and fissures, and diagnostic confidence. Objective noise and CT values were measured in thoracic aorta for each image series, and the noise power spectrum was assessed. Data were analyzed with analysis of variance and Wilcoxon signed rank tests. RESULTS. All 186 lesions were seen on 40 Quality Reference mAs SAFIRE images. Diagnostic confidence on SAFIRE images was higher than that for FBP images. Except for the minor blotchy appearance on SAFIRE settings S3 and S4, no significant artifacts were noted. Objective noise with 40 Quality Reference mAs S1 images (21.1 ± 6.1 SD of HU) was significantly lower than that for 40 Quality Reference mAs FBP images (28.5 ± 8.1 SD of HU) (p < 0.001). Noise power spectra were identical for SAFIRE and FBP with progressive noise reduction with higher iteration SAFIRE settings. CONCLUSION. Iterative reconstruction (SAFIRE) allows reducing the radiation exposure by approximately 65% without losing diagnostic information in chest CT.

Automated quantification of pneumothorax in CT.

An automated, computer-aided diagnosis (CAD) algorithm for the quantification of pneumothoraces from Multidetector Computed Tomography (MDCT) images has been developed. Algorithm performance was evaluated through comparison to manual segmentation by expert radiologists. A combination of two-dimensional and three-dimensional processing techniques was incorporated to reduce required processing time by two-thirds (as compared to similar techniques). Volumetric measurements on relative pneumothorax size were obtained and the overall performance of the automated method shows an average error of just below 1%.

Comparison of hybrid and pure iterative reconstruction techniques with conventional filtered back projection: dose reduction potential in the abdomen.

PURPOSE: Assess the effect of filtered back projection (FBP) and hybrid (adaptive statistical iterative reconstruction [ASIR]) and pure (model-based iterative reconstruction [MBIR]) iterative reconstructions on abdominal computed tomography (CT) acquired with 75% radiation dose reduction. MATERIALS AND METHODS: In an institutional review board-approved prospective study, 10 patients (mean [standard deviation] age, 60 (8) years; 4 men and 6 women) gave informed consent for acquisition of additional abdominal images on 64-slice multidetector-row CT (GE 750HD, GE Healthcare). Scanning was repeated over a 10-cm scan length at 200 and 50 milliampere second (mA s), with remaining parameters held constant at 120 kilovolt (peak), 0.984:1 pitch, and standard reconstruction kernel. Projection data were deidentified, exported, and reconstructed to obtain 4 data sets (200-mA s FBP, 50-mA s FBP, 50-mA s ASIR, 50-mA s MBIR), which were evaluated by 2 abdominal radiologists for lesions and subjective image quality. Objective noise and noise spectral density were measured for each image series. RESULTS: Among the 10 patients, the maximum weight recorded was 123 kg, with maximum transverse diameter measured as 43.7 cm. Lesion conspicuity at 50-mA s MBIR was better than on 50-mA s FBP and ASIR images (P < 0.01). Image noise was rated as suboptimal on low-dose FBP and ASIR but deemed acceptable in MBIR images. Objective noise with 50-mA s MBIR was 2 to 3 folds lower compared to 50-mA s ASIR, 50-mA s FBP, and 200-mA s FBP (P < 0.0001). Noise spectral density analyses demonstrated that ASIR retains the noise spectrum signature of FBP, whereas MBIR has much lower noise with a more regularized noise spectrum pattern. CONCLUSION: Model-based iterative reconstruction renders acceptable image quality and diagnostic confidence in 50- mA s abdominal CT images, whereas FBP and ASIR images are associated with suboptimal image quality at this radiation dose level.

A decomposition-based CT reconstruction formulation for reducing blooming artifacts.

Cardiac computed tomography represents an important advancement in the ability to assess coronary vessels. The accuracy of these non-invasive imaging studies is limited, however, by the presence of calcium, since calcium blooming artifacts lead to an over-estimation of the degree of luminal narrowing. To address this problem, we have developed a unified decomposition-based iterative reconstruction formulation, where different penalty functions are imposed on dense objects (i.e. calcium) and soft tissue. The result is a quantifiable reduction in blooming artifacts without the introduction of new distortions away from the blooming observed in other methods. Results are shown for simulations, phantoms, ex vivo, and in vivo studies.

Adaptive statistical iterative reconstruction technique for radiation dose reduction in chest CT: a pilot study.

PURPOSE: To compare lesion detection and image quality of chest computed tomographic (CT) images acquired at various tube current-time products (40-150 mAs) and reconstructed with adaptive statistical iterative reconstruction (ASIR) or filtered back projection (FBP). MATERIALS AND METHODS: In this Institutional Review Board-approved HIPAA-compliant study, CT data from 23 patients (mean age, 63 years ± 7.3 [standard deviation]; 10 men, 13 women) were acquired at varying tube current-time products (40, 75, 110, and 150 mAs) on a 64-row multidetector CT scanner with 10-cm scan length. All patients gave informed consent. Data sets were reconstructed at 30%, 50%, and 70% ASIR-FBP blending. Two thoracic radiologists assessed image noise, visibility of small structures, lesion conspicuity, and diagnostic confidence. Objective noise and CT number were measured in the thoracic aorta. CT dose index volume, dose-length product, weight, and transverse diameter were recorded. Data were analyzed by using analysis of variance and the Wilcoxon signed rank test. RESULTS: FBP had unacceptable noise at 40 and 75 mAs in 17 and five patients, respectively, whereas ASIR had acceptable noise at 40-150 mAs. Objective noise with 30%, 50%, and 70% ASIR blending (11.8 ± 3.8, 9.6 ± 3.1, and 7.5 ± 2.6, respectively) was lower than that with FBP (15.8 ± 4.8) (P < .0001). No lesions were missed on FBP or ASIR images. Lesion conspicuity was graded as well seen on both FBP and ASIR images (P < .05). Mild pixilated blotchy texture was noticed with 70% blended ASIR images. CONCLUSION: Acceptable image quality can be obtained for chest CT images acquired at 40 mAs by using ASIR without any substantial artifacts affecting diagnostic confidence. SUPPLEMENTAL MATERIAL: http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.11101450/-/DC1.

Abdominal CT: comparison of adaptive statistical iterative and filtered back projection reconstruction techniques.

PURPOSE: To compare image quality and lesion conspicuity on abdominal computed tomographic (CT) images acquired with different x-ray tube current-time products (50-200 mAs) and reconstructed with adaptive statistical iterative reconstruction (ASIR) and filtered back projection (FBP) techniques. MATERIALS AND METHODS: Twenty-two patients (mean age, 60.1 years ± 7.3 [standard deviation]; age range, 52.8-67.4 years; mean weight, 78.9 kg ± 18.3; 12 men, 10 women) gave informed consent for this prospective institutional review board-approved and HIPAA-compliant study, which involved the acquisition of four additional image series at multidetector CT. Images were acquired at different tube current-time products (200, 150, 100, and 50 mAs) and encompassed an abdominal lesion over a 10-cm scan length. Images were reconstructed separately with FBP and with three levels of ASIR-FBP blending. Two radiologists reviewed FBP and ASIR images for image quality in a blinded and randomized manner. Volume CT dose index (CTDI(vol)), dose-length product, patient weight, objective noise, and CT numbers were recorded. Data were analyzed by using analysis of variance and the Wilcoxon signed rank test. RESULTS: CTDI(vol) values were 16.8, 12.6, 8.4, and 4.2 mGy for 200, 150, 100, and 50 mAs, respectively (P < .001). Subjective noise was graded as below average at 150 mAs and average at 100 and 50 mAs for ASIR images, as compared with FBP images, on which noise was graded as average at 150 mAs, above average at 100 mAs, and unacceptable at 50 mAs. A substantial blotchy image appearance was noted in four of 22 image series acquired at 4.2 mGy with 70% ASIR. Lesion conspicuity was significantly better at 4.2 mGy on ASIR than on FBP images (observed P < .044), and overall diagnostic confidence changed from unacceptable on FBP to acceptable on ASIR images. CONCLUSION: ASIR lowers noise and improves diagnostic confidence in and conspicuity of subtle abdominal lesions at 8.4 mGy when images are reconstructed with 30% ASIR blending and at 4.2 mGy in patients weighing 90 kg or less when images are reconstructed with 50% or 70% ASIR blending.

A spatio-temporal deconvolution method to improve perfusion CT quantification.

Perfusion imaging is a useful adjunct to anatomic imaging in numerous diagnostic and therapy-monitoring settings. One approach to perfusion imaging is to assume a convolution relationship between a local arterial input function and the tissue enhancement profile of the region of interest via a "residue function" and subsequently solve for this residue function. This ill-posed problem is generally solved using singular-value decomposition based approaches, and the hemodynamic parameters are solved for each voxel independently. In this paper, we present a formulation which incorporates both spatial and temporal correlations, and show through simulations that this new formulation yields higher accuracy and greater robustness with respect to image noise. We also show using rectal cancer tumor images that this new formulation results in better segregation of normal and cancerous voxels.

Radiation dose reduction with chest computed tomography using adaptive statistical iterative reconstruction technique: initial experience.

PURPOSE: To assess radiation dose reduction and image quality for weight-based chest computed tomographic (CT) examination results reconstructed using adaptive statistical iterative reconstruction (ASIR) technique. MATERIALS AND METHODS: With local ethical committee approval, weight-adjusted chest CT examinations were performed using ASIR in 98 patients and filtered backprojection (FBP) in 54 weight-matched patients on a 64-slice multidetector CT. Patients were categorized into 3 groups: 60 kg or less (n = 32), 61 to 90 kg (n = 77), and 91 kg or more (n = 43) for weight-based adjustment of noise indices for automatic exposure control (Auto mA; GE Healthcare, Waukesha, Wis). Remaining scan parameters were held constant at 0.984:1 pitch, 120 kilovolts (peak), 40-mm table feed per rotation, and 2.5-mm section thickness. Patients' weight, scanning parameters, and CT dose index volume were recorded. Effective doses (EDs) were estimated. Image noise was measured in the descending thoracic aorta at the level of the carina. Data were analyzed using analysis of variance. RESULTS: Compared with FBP, ASIR was associated with an overall mean (SD) decrease of 27.6% in ED (ASIR, 8.8 [2.3] mSv; FBP, 12.2 [2.1] mSv; P < 0.0001). With the use of ASIR, the ED values were 6.5 (1.8) mSv (28.8% decrease), 7.3 (1.6) mSv (27.3% decrease), and 12.8 (2.3) mSv (26.8% decrease) for the weight groups of 60 kg or less, 61 to 90 kg, and 91 kg or more, respectively, compared with 9.2 (2.3) mSv, 10.0 (2.0) mSv, and 17.4 (2.1) mSv with FBP (P < 0.0001). Despite dose reduction, there was less noise with ASIR (12.6 [2.9] mSv) than with FBP (16.6 [6.2] mSv; P < 0.0001). CONCLUSIONS: Adaptive statistical iterative reconstruction helps reduce chest CT radiation dose and improve image quality compared with the conventionally used FBP image reconstruction.

Reducing abdominal CT radiation dose with adaptive statistical iterative reconstruction technique.

PURPOSE: To assess radiation dose reduction for abdominal computed tomography (CT) examinations with adaptive statistical iterative reconstruction (ASIR) technique. MATERIALS AND METHODS: With institutional review board approval, retrospective review of weight adapted abdominal CT exams were performed in 156 consecutive patients with ASIR and in 66 patients with filtered back projection (FBP) on a 64-slice MDCT. Patients were categorized into 3 groups of <60 kg (n = 42), 61 to 90 kg (n = 100), and >or=91 kg (n = 80) for weight-based adjustment of automatic exposure control technique. Remaining scan parameters were held constant at 1.375:1 pitch, 120 kVp, 55 mm table feed per rotation, 5 mm section thickness. Two radiologists reviewed all CT examinations for image noise and diagnostic acceptability. CT dose index volume, and dose length product were recorded. Image noise and transverse abdominal diameter were measured in all patients. Data were analyzed using analysis of variance. RESULTS: ASIR allowed for an overall average decrease of 25.1% in CT dose index volume compared with the FBP technique (ASIR, 11.9 +/- 3.6 mGy; FBP, 15.9 +/- 4.3 mGy) (P < 0.0001). In each of the 3 weight categories, CT examinations reconstructed with ASIR technique were associated with significantly lower radiation dose compared with FBP technique (P < 0.0001). There was also significantly less objective image noise with ASIR (6.9 +/- 2.2) than with FBP (9.5 +/- 2.0) (P < 0.0001). For the subjective analysis, all ASIR and FBP reconstructed abdominal CTs had optimal or less noise. However, 9% of FBP and 3.8% of ASIR reconstructed CT examinations were diagnostically unacceptable because of the presence of artifacts. Use of ASIR reconstruction kernel results in a blotchy pixilated appearance in 39% of CT sans which however, was mild and did not affect the diagnostic acceptability of images. The critical reproduction of visually sharp anatomic structures was preserved in all but one ASIR 40% reconstructed CT examination. CONCLUSION: ASIR technique allows radiation dose reduction for abdominal CT examinations whereas improving image noise compared with the FBP technique.

Calcified plaque: measurement of area at thin-section flat-panel CT and 64-section multidetector CT and comparison with histopathologic findings.

The purpose of this study was to assess the blooming artifacts in ex vivo coronary arteries at multidetector computed tomography (CT) and flat-panel-volume CT by comparing measured areas of calcified plaque with respect to the reference standard of histopathologic findings. Three ex vivo hearts were scanned with multidetector CT and flat-panel-volume CT after institutional review board approval. The area of calcified plaque was measured at histopathologic examination, multidetector CT, and flat-panel-volume CT. The plaque area was overestimated at multidetector CT by 400% (4.61/1.15) on average, and the predicted difference between the measurements was significant (3.46 mm(2), P = .018). The average overestimation of plaque area at flat-panel-volume CT was twofold (214% [2.18/1.02]), and the predicted difference was smaller (1.16 mm(2), P = .08). The extent of the blooming artifact in visualizing calcified coronary plaque is reduced by using flat-panel-volume CT.

Trends in the use and role of biomarkers in phase I oncology trials.

PURPOSE: There has been interest in using biomarkers that aid the evaluation of new anticancer agents. We evaluated trends in the use of biomarkers and their contribution to the main goals of phase I trials. EXPERIMENTAL DESIGN: We did a systematic review of abstracts submitted to the American Society of Clinical Oncology annual meeting from 1991 to 2002 and the publications related to these abstracts. We analyzed the use of biomarkers and their contribution to published phase I trials. RESULTS: Twenty percent of American Society of Clinical Oncology phase I abstracts (503 of 2458) from 1991 to 2002 included biomarkers. This proportion increased over time (14% in 1991 compared with 26% in 2002; P < 0.02). Independent predictors of the use of biomarkers included National Cancer Institute sponsorship, submission in the time period of 1999 to 2002, adult population, and drug family (biological agents). Biomarkers supported dose selection for phase II studies in 11 of 87 of the trials (13%) emanating from these abstracts. However, the primary determinants of phase II dose and schedule were toxicity and/or efficacy in all but one of these 87 trials (1%). Biomarker studies provided evidence supporting the proposed mechanism of action in 34 of 87 of the published trials (39%). CONCLUSIONS: The use of biomarkers in phase I trials has increased over the period from 1991 to 2002. To date, biomarker utilization has made a limited and primarily supportive contribution to dose selection, the primary end point of phase I studies. Additional studies are needed to determine what type of biomarker information is most valuable to evaluate in phase I trials.

Model-based variational smoothing and segmentation for diffusion tensor imaging in the brain.

This article applies a unified approach to variational smoothing and segmentation to brain diffusion tensor image data along user-selected attributes derived from the tensor, with the aim of extracting detailed brain structure information. The application of this framework simultaneously segments and denoises to produce edges and smoothed regions within the white matter of the brain that are relatively homogeneous with respect to the diffusion tensor attributes of choice. This approach enables the visualization of a smoothed, scale invariant representation of the tensor data field in a variety of diverse forms. In addition to known attributes such as fractional anisotropy, these representations include selected directional tensor components and additionally associated continuous valued edge fields that might be used for further segmentation. A comparison is presented of the results of three different data model selections with respect to their ability to resolve white matter structure. The resulting images are integrated to provide better perspective of the model properties (edges, smoothed image, and so forth) and their relationship to the underlying brain anatomy. The improvement in brain image quality is illustrated both qualitatively and quantitatively, and the robust performance of the algorithm in the presence of added noise is shown. Smoothing occurs without loss of edge features because of the simultaneous segmentation aspect of the variational approach, and the output enables better delineation of tensors representative of local and long-range association, projection, and commissural fiber systems.

Using imaging biomarkers to accelerate drug development and clinical trials.

There is increasing evidence that human medical imaging can help answer key questions that arise during the drug development process. Imaging modalities such as magnetic resonance imaging, computed tomography and positron emission tomography can offer significant insights into the bioactivity, pharmacokinetics and dosing of drugs, in addition to supporting registration applications. In this review, examples from oncology, neurology, psychiatry, infectious diseases and inflammatory diseases are used to illustrate the role imaging can play. We conclude with some remarks concerning new developments that will be required to significantly advance the field of pharmaco-imaging.

Imaging angiogenesis: applications and potential for drug development.

Recognition of the importance of angiogenesis to tumor growth and metastasis has led to efforts to develop new drugs that are targeted to angiogenic vasculature. Clinical trials of these agents are challenging, both because there is no agreed upon method of establishing the correct dosage for drugs whose mechanism of action is not primarily cytotoxic and because of the long time it takes to determine whether such drugs have a clinical effect. Therefore, there is a need for rapid and effective biomarkers to establish drug dosage and monitor clinical response. This review addresses the potential of imaging as a way to accurately and reliably assess changes in angiogenic vasculature in response to therapy. We describe the advantages and disadvantages of several imaging modalities, including positron emission tomography, x-ray computed tomography, magnetic resonance imaging, ultrasound, and optical imaging, for imaging angiogenic vasculature. We also discuss the analytic methods used to derive blood flow, blood volume, empirical semiquantitative hemodynamic parameters, and quantitative hemodynamic parameters from pharmacokinetic modeling. We examine the validity of these methods, citing studies that test correlations between data derived from imaging and data derived from other established methods, their reproducibility, and correlations between imaging-derived hemodynamic parameters and other pathologic indicators, such as microvessel density, pathology score, and disease outcome. Finally, we discuss which imaging methods are most likely to have the sensitivity and reliability required for monitoring responses to cancer therapy and describe ways in which imaging has been used in clinical trials to date.

A label-free optical technique for detecting small molecule interactions.

A novel approach for the label-free detection of molecular interactions is presented in which a colorimetric resonant grating is used as a surface binding platform. The grating, when illuminated with white light, is designed to reflect only a single wavelength. When molecules are attached to the surface, the reflected wavelength (color) is shifted due to the change of the optical path of light that is coupled into the grating. By linking receptor molecules to the grating surface, complementary binding molecules can be detected without the use of any kind of fluorescent probe or radioactive label. The detection technique is capable of detecting the addition and removal of small molecules as they interact with receptor molecules on the sensor surface or enzymes in the solution surrounding the sensor. Two assays are presented to exemplify the detection of small molecule interactions with the biosensor. First, an avidin receptor layer is used to detect 244 Da biotin binding. Second, a protease assay is performed in which a 136 Da p-nitroanilide (pNA) moeity is cleaved from an immobilized substrate. Because the sensor structure can be embedded in the plastic surfaces of microtiter plates or the glass surfaces of microarray slides, it is expected that this technology will be most useful in applications where large numbers of biomolecular interactions are measured in parallel, particularly when molecular labels will alter or inhibit the functionality of the molecules under study. Screening of pharmaceutical compound libraries with protein targets, and microarray screening of protein-protein interactions for proteomics are examples of applications that require the sensitivity and throughput afforded by this approach.

Functional MRI activity characterization using response time shift estimates from curve evolution.

Characterizing the response of the brain to a stimulus based on functional magnetic resonance imaging data is a major challenge due to the fact that the response time delay of the brain may be different from one stimulus phase to the next and from pixel to pixel. To enhance detectability, this work introduces the use of a curve evolution approach that provides separate estimates of the response time shifts at each phase of the stimulus on a pixel-by-pixel basis. The approach relies on a parsimonious but simple model that is nonlinear in the time shifts of the response relative to the stimulus and linear in the gains. To effectively use the response time shift estimates in a subspace detection framework, we implement a robust hypothesis test based on a Laplacian noise model. The algorithm provides a pixel-by-pixel functional characterization of the brain's response. The results based on experimental data show that response time shift estimates, when properly implemented, enhance detectability without sacrificing robustness.