Real-time breast lesion classification combining diffuse optical tomography frequency domain data and BI-RADS assessment.

Ultrasound (US)-guided diffuse optical tomography (DOT) has demonstrated potential for breast cancer diagnosis, in which real-time or near real-time diagnosis with high accuracy is desired. However, DOT's relatively slow data processing and image reconstruction speeds have hindered real-time diagnosis. Here, we propose a real-time classification scheme that combines US breast imaging reporting and data system (BI-RADS) readings and DOT frequency domain measurements. A convolutional neural network is trained to generate malignancy probability scores from DOT measurements. Subsequently, these scores are integrated with BI-RADS assessments using a support vector machine classifier, which then provides the final diagnostic output. An area under the receiver operating characteristic curve of 0.978 is achieved in distinguishing between benign and malignant breast lesions in patient data without image reconstruction.

PA-NeRF, a neural radiance field model for 3D photoacoustic tomography reconstruction from limited Bscan data.

We introduce a novel deep-learning-based photoacoustic tomography method called Photoacoustic Tomography Neural Radiance Field (PA-NeRF) for reconstructing 3D volumetric PAT images from limited 2D Bscan data. In conventional 3D volumetric imaging, a 3D reconstruction requires transducer element data obtained from all directions. Our model employs a NeRF-based PAT 3D reconstruction method, which learns the relationship between transducer element positions and the corresponding 3D imaging. Compared with convolution-based deep-learning models, such as Unet and TransUnet, PA-NeRF does not learn the interpolation process but rather gains insight from 3D photoacoustic imaging principles. Additionally, we introduce a forward loss that improves the reconstruction quality. Both simulation and phantom studies validate the performance of PA-NeRF. Further, we apply the PA-NeRF model to clinical examples to demonstrate its feasibility. To the best of our knowledge, PA-NeRF is the first method in photoacoustic tomography to successfully reconstruct a 3D volume from sparse Bscan data.

Acoustic resolution photoacoustic Doppler flowmetry for assessment of patient rectal cancer blood perfusion.

Significance: Photoacoustic Doppler flowmetry offers quantitative blood perfusion information in addition to photoacoustic vascular contrast for rectal cancer assessment. Aim: We aim to develop and validate a correlational Doppler flowmetry utilizing an acoustic resolution photoacoustic microscopy (AR-PAM) system for blood perfusion analysis. Approach: To extract blood perfusion information, we implemented AR-PAM Doppler flowmetry consisting of signal filtering and conditioning, A-line correlation, and angle compensation. We developed flow phantoms and contrast agent to systemically investigate the flowmetry's efficacy in a series of phantom studies. The developed correlational Doppler flowmetry was applied to images collected during in vivo AR-PAM for post-treatment rectal cancer evaluation. Results: The linearity and accuracy of the Doppler flow measurement system were validated in phantom studies. Imaging rectal cancer patients treated with chemoradiation demonstrated the feasibility of using correlational Doppler flowmetry to assess treatment response and distinguish residual cancer from cancer-free tumor bed tissue and normal rectal tissue. Conclusions: A new correlational Doppler flowmetry was developed and validated through systematic phantom evaluations. The results of its application to in vivo patients suggest it could be a useful addition to photoacoustic endoscopy for post-treatment rectal cancer assessment.

Automated pipeline for breast cancer diagnosis using US assisted diffuse optical tomography.

Ultrasound (US)-guided diffuse optical tomography (DOT) is a portable and non-invasive imaging modality for breast cancer diagnosis and treatment response monitoring. However, DOT data pre-processing and imaging reconstruction often require labor intensive manual processing which hampers real-time diagnosis. In this study, we aim at providing an automated US-assisted DOT pre-processing, imaging and diagnosis pipeline to achieve near real-time diagnosis. We have developed an automated DOT pre-processing method including motion detection, mismatch classification using deep-learning approach, and outlier removal. US-lesion information needed for DOT reconstruction was extracted by a semi-automated lesion segmentation approach combined with a US reading algorithm. A deep learning model was used to evaluate the quality of the reconstructed DOT images and a two-step deep-learning model developed earlier is implemented to provide final diagnosis based on US imaging features and DOT measurements and imaging results. The presented US-assisted DOT pipeline accurately processed the DOT measurements and reconstruction and reduced the procedure time to 2 to 3 minutes while maintained a comparable classification result with manually processed dataset.

Two-stage classification strategy for breast cancer diagnosis using ultrasound-guided diffuse optical tomography and deep learning.

Significance: Ultrasound (US)-guided diffuse optical tomography (DOT) has demonstrated great potential for breast cancer diagnosis in which real-time or near real-time diagnosis with high accuracy is desired. Aim: We aim to use US-guided DOT to achieve an automated, fast, and accurate classification of breast lesions. Approach: We propose a two-stage classification strategy with deep learning. In the first stage, US images and histograms created from DOT perturbation measurements are combined to predict benign lesions. Then the non-benign suspicious lesions are passed through to the second stage, which combine US image features, DOT histogram features, and 3D DOT reconstructed images for final diagnosis. Results: The first stage alone identified 73.0% of benign cases without image reconstruction. In distinguishing between benign and malignant breast lesions in patient data, the two-stage classification approach achieved an area under the receiver operating characteristic curve of 0.946, outperforming the diagnoses of all single-modality models and of a single-stage classification model that combines all US images, DOT histogram, and imaging features. Conclusions: The proposed two-stage classification strategy achieves better classification accuracy than single-modality-only models and a single-stage classification model that combines all features. It can potentially distinguish breast cancers from benign lesions in near real-time.

Enhanced 3D visualization of human fallopian tube morphology using a miniature optical coherence tomography catheter.

We demonstrate the use of our miniature optical coherence tomography catheter to acquire three-dimensional human fallopian tube images. Images of the fallopian tube's tissue morphology, vasculature, and tissue heterogeneity distribution are enhanced by adaptive thresholding, masking, and intensity inverting, making it easier to differentiate malignant tissue from normal tissue. The results show that normal fallopian tubes tend to have rich vasculature accompanied by a patterned tissue scattering background, features that do not appear in malignant cases. This finding suggests that miniature OCT catheters may have great potential for fast optical biopsy of the fallopian tube.

Cylindrical lens configuration for optimizing light delivery in a curvilinear endocavity photoacoustic imaging system.

Curvilinear endocavity ultrasound images capture a wide field of view with a miniature probe. In adapting photoacoustic imaging (PAI) to work with such ultrasound systems, light delivery is challenged by the trade-off between image quality and laser safety concerns. Here, we present two novel, to the best of our knowledge, designs based on cylindrical lenses that are optimized for transvaginal PAI B-scan imaging. Our simulation and experimental results demonstrate that, compared to conventional light delivery methods for PAI imaging, the proposed designs are safer for higher pulse energies and provide deeper imaging and a wider lateral field of view. The proposed designs could also improve the performance of endoscopic co-registered ultrasound/photoacoustic imaging in other clinical applications.

Deep learning based on co-registered ultrasound and photoacoustic imaging improves the assessment of rectal cancer treatment response.

Identifying complete response (CR) after rectal cancer preoperative treatment is critical to deciding subsequent management. Imaging techniques, including endorectal ultrasound and MRI, have been investigated but have low negative predictive values. By imaging post-treatment vascular normalization using photoacoustic microscopy, we hypothesize that co-registered ultrasound and photoacoustic imaging will better identify complete responders. In this study, we used in vivo data from 21 patients to develop a robust deep learning model (US-PAM DenseNet) based on co-registered dual-modality ultrasound (US) and photoacoustic microscopy (PAM) images and individualized normal reference images. We tested the model's accuracy in differentiating malignant from non-cancer tissue. Compared to models based on US alone (classification accuracy 82.9 ± 1.3%, AUC 0.917(95%CI: 0.897-0.937)), the addition of PAM and normal reference images improved the model performance significantly (accuracy 92.4 ± 0.6%, AUC 0.968(95%CI: 0.960-0.976)) without increasing model complexity. Additionally, while US models could not reliably differentiate images of cancer from those of normalized tissue with complete treatment response, US-PAM DenseNet made accurate predictions from these images. For use in the clinical settings, US-PAM DenseNet was extended to classify entire US-PAM B-scans through sequential ROI classification. Finally, to help focus surgical evaluation in real time, we computed attention heat maps from the model predictions to highlight suspicious cancer regions. We conclude that US-PAM DenseNet could improve the clinical care of rectal cancer patients by identifying complete responders with higher accuracy than current imaging techniques.

Fusion deep learning approach combining diffuse optical tomography and ultrasound for improving breast cancer classification.

Diffuse optical tomography (DOT) is a promising technique that provides functional information related to tumor angiogenesis. However, reconstructing the DOT function map of a breast lesion is an ill-posed and underdetermined inverse process. A co-registered ultrasound (US) system that provides structural information about the breast lesion can improve the localization and accuracy of DOT reconstruction. Additionally, the well-known US characteristics of benign and malignant breast lesions can further improve cancer diagnosis based on DOT alone. Inspired by a fusion model deep learning approach, we combined US features extracted by a modified VGG-11 network with images reconstructed from a DOT deep learning auto-encoder-based model to form a new neural network for breast cancer diagnosis. The combined neural network model was trained with simulation data and fine-tuned with clinical data: it achieved an AUC of 0.931 (95% CI: 0.919-0.943), superior to those achieved using US images alone (0.860) or DOT images alone (0.842).

A Coregistered Ultrasound and Photoacoustic Imaging Protocol for the Transvaginal Imaging of Ovarian Lesions.

Ovarian cancer remains the deadliest of all the gynecological malignancies due to the lack of reliable screening tools for early detection and diagnosis. Photoacoustic imaging or tomography (PAT) is an emerging imaging modality that can provide the total hemoglobin concentration (relative scale, rHbT) and blood oxygen saturation (%sO2) of ovarian/adnexal lesions, which are important parameters for cancer diagnosis. Combined with coregistered ultrasound (US), PAT has demonstrated great potential for detecting ovarian cancers and for accurately diagnosing ovarian lesions for effective risk assessment and the reduction of unnecessary surgeries of benign lesions. However, PAT imaging protocols in clinical applications, to our knowledge, largely vary among different studies. Here, we report a transvaginal ovarian cancer imaging protocol that can be beneficial to other clinical studies, especially those using commercial ultrasound arrays for the detection of photoacoustic signals and standard delay-and-sum beamforming algorithms for imaging.

Depth-resolved attenuation mapping of the human ovary and fallopian tube using optical coherence tomography.

Due to the lack of reliable early-diagnostic tools, most ovarian cancers are diagnosed at late stages. Although optical coherence tomography (OCT) has shown promise for identifying diseased ovaries and fallopian tubes at an earlier stage, previous studies either did not provide quantitative scattering mapping or simply used Beer's law to fit the scattering coefficients of each A-line. In this paper, we calculated the pixel-wise attenuation coefficients of ovaries and fallopian tubes in OCT images. Data from 73 freshly excised human ovaries and fallopian tubes from 36 patients have shown that statistical features are statistically different between cancerous ovaries, infundibula, and fimbriae and normal ones.

Ultrasound-enhanced Unet model for quantitative photoacoustic tomography of ovarian lesions.

Quantitative photoacoustic tomography (QPAT) is a valuable tool in characterizing ovarian lesions for accurate diagnosis. However, accurately reconstructing a lesion's optical absorption distributions from photoacoustic signals measured with multiple wavelengths is challenging because it involves an ill-posed inverse problem with three unknowns: the Grüneisen parameter ( Γ ) , the absorption distribution, and the optical fluence ( ϕ ) . Here, we propose a novel ultrasound-enhanced Unet model (US-Unet) that reconstructs optical absorption distribution from PAT data. A pre-trained ResNet-18 extracts the US features typically identified as morphologies of suspicious ovarian lesions, and a Unet is implemented to reconstruct optical absorption coefficient maps, using the initial pressure and US features extracted by ResNet-18. To test this US-Unet model, we calculated the blood oxygenation saturation values and total hemoglobin concentrations from 655 regions of interest (ROIs) (421 benign, 200 malignant, and 34 borderline ROIs) obtained from clinical images of 35 patients with ovarian/adnexal lesions. A logistic regression model was used to compute the ROC, the area under the ROC curve (AUC) was 0.94, and the accuracy was 0.89. To the best of our knowledge, this is the first study to reconstruct quantitative PAT with PA signals and US-based structural features.

Quantification of ovarian lesion and fallopian tube vasculature using optical-resolution photoacoustic microscopy.

The heterogeneity in the pathological and clinical manifestations of ovarian cancer is a major hurdle impeding early and accurate diagnosis. A host of imaging modalities, including Doppler ultrasound, MRI, and CT, have been investigated to improve the assessment of ovarian lesions. We hypothesized that pathologic conditions might affect the ovarian vasculature and that these changes might be detectable by optical-resolution photoacoustic microscopy (OR-PAM). In our previous work, we developed a benchtop OR-PAM and demonstrated it on a limited set of ovarian and fallopian tube specimens. In this study, we collected data from over 50 patients, supporting a more robust statistical analysis. We then developed an efficient custom analysis pipeline for characterizing the vascular features of the samples, including the mean vessel diameter, vascular density, global vascular directionality, local vascular definition, and local vascular tortuosity/branchedness. Phantom studies using carbon fibers showed that our algorithm was accurate within an acceptable error range. Between normal ovaries and normal fallopian tubes, we observed significant differences in five of six extracted vascular features. Further, we showed that distinct subsets of vascular features could distinguish normal ovaries from cystic, fibrous, and malignant ovarian lesions. In addition, a statistically significant difference was found in the mean vascular tortuosity/branchedness values of normal and abnormal tubes. The findings support the proposition that OR-PAM can help distinguish the severity of tubal and ovarian pathologies.

A review of co-registered transvaginal photoacoustic and ultrasound imaging for ovarian cancer diagnosis.

Ovarian cancer is the deadliest of all gynecological malignancies. When ovarian cancer is detected at an early, localized stage, surgery and chemotherapy can cure 70%-90% of patients, compared with 20% or fewer when it is diagnosed at later stages. Clearly, early detection is critical, yet the lack of early symptoms and effective screening tools means that only 20-25% of ovarian cancers are diagnosed early. Photoacoustic imaging (PAI) is an emerging modality that uses a short-pulsed laser to excite tissue. The resulting photoacoustic waves are used to image tissue optical contrast, which is directly related to tissue microvasculature and thus to cancer growth. When co-registered with transvaginal ultrasound (US), PAI offers great promise in diagnosing earlier stage ovarian cancers and distinguishing benign processes from malignant ovarian masses. In this article, we review the limitations of the current imaging tools for early ovarian cancer diagnosis and present recent advances in co-registered PAI/US.

Difference imaging from single measurements in diffuse optical tomography: a deep learning approach.

SIGNIFICANCE: "Difference imaging," which reconstructs target optical properties using measurements with and without target information, is often used in diffuse optical tomography (DOT) in vivo imaging. However, taking additional reference measurements is time consuming, and mismatches between the target medium and the reference medium can cause inaccurate reconstruction. AIM: We aim to streamline the data acquisition and mitigate the mismatch problems in DOT difference imaging using a deep learning-based approach to generate data from target measurements only. APPROACH: We train an artificial neural network to output data for difference imaging from target measurements only. The model is trained and validated on simulation data and tested with simulations, phantom experiments, and clinical data from 56 patients with breast lesions. RESULTS: The proposed method has comparable performance to the traditional approach using measurements without mismatch between the target side and the reference side, and it outperforms the traditional approach using measurements when there is a mismatch. It also improves the target-to-artifact ratio and lesion localization in patient data. CONCLUSIONS: The proposed method can simplify the data acquisition procedure, mitigate mismatch problems, and improve reconstructed image quality in DOT difference imaging.

Human colorectal cancer tissue assessment using optical coherence tomography catheter and deep learning.

Optical coherence tomography (OCT) can differentiate normal colonic mucosa from neoplasia, potentially offering a new mechanism of endoscopic tissue assessment and biopsy targeting, with a high optical resolution and an imaging depth of ~1 mm. Recent advances in convolutional neural networks (CNN) have enabled application in ophthalmology, cardiology, and gastroenterology malignancy detection with high sensitivity and specificity. Here, we describe a miniaturized OCT catheter and a residual neural network (ResNet)-based deep learning model manufactured and trained to perform automatic image processing and real-time diagnosis of the OCT images. The OCT catheter has an outer diameter of 3.8 mm, a lateral resolution of ~7 µm, and an axial resolution of ~6 µm. A customized ResNet is utilized to classify OCT catheter colorectal images. An area under the receiver operating characteristic (ROC) curve (AUC) of 0.975 is achieved to distinguish between normal and cancerous colorectal tissue images.

Prospective assessment of adjunctive ultrasound-guided diffuse optical tomography in women undergoing breast biopsy: Impact on BI-RADS assessments.

PURPOSE: To assess the impact of adjunctive ultrasound guided diffuse optical tomography (US-guided DOT) on BI-RADS assessment in women undergoing US-guided breast biopsy. METHOD: This prospective study enrolled women referred for US-guided breast biopsy between 3/5/2019 and 3/19/2020. Participants underwent US-guided DOT immediately before biopsy. The US-guided DOT acquisition generated average maximum total hemoglobin (HbT) spatial maps and quantitative HbT values. Four radiologists blinded to histopathology assessed conventional imaging (CI) to assign a CI BI-RADS assessment and then integrated DOT information in assigning a CI&DOT BI-RADS assessment. HbT was compared between benign and malignant lesions using an ANOVA test and Tukey's test. Benign biopsies were tabulated, deeming BI-RADS ≥ 4A as positive. Reader agreement was assessed. RESULTS: Among 61 included women (mean age 48 years), biopsy demonstrated 15 (24.6%) malignant and 46 (75.4%) benign lesions. Mean HbT was 55.3 ± 22.6 µM in benign lesions versus 85.4 ± 15.6 µM in cancers (p < .001). HbT threshold of 78.5 µM achieved sensitivity 80% (12/15) and specificity 89% (41/46) for malignancy. Across readers and patients, 197 pairs of CI BI-RADS and CI&DOT BI-RADS assessments were assigned. Adjunctive US-guided DOT achieved a net decrease in 23.5% (31/132) of suspicious (CI BI-RADS ≥ 4A) assessments of benign lesions (34 correct downgrades and 3 incorrect upgrades). 38.3% (31/81) of 4A assessments were appropriately downgraded. No cancer was downgraded to a non-actionable assessment. Interreader agreement analysis demonstrated kappa = 0.48-0.53 for CI BI-RADS and kappa = 0.28-0.44 for CI&DOT BI-RADS. CONCLUSIONS: Integration of US-guided DOT information achieved a 23.5% reduction in suspicious BI-RADS assessments for benign lesions. Larger studies are warranted, with attention to improved reader agreement.

Machine learning model with physical constraints for diffuse optical tomography.

A machine learning model with physical constraints (ML-PC) is introduced to perform diffuse optical tomography (DOT) reconstruction. DOT reconstruction is an ill-posed and under-determined problem, and its quality suffers by model mismatches, complex boundary conditions, tissue-probe contact, noise etc. Here, for the first time, we combine ultrasound-guided DOT with ML to facilitate DOT reconstruction. Our method has two key components: (i) a neural network based on auto-encoder is adopted for DOT reconstruction, and (ii) physical constraints are implemented to achieve accurate reconstruction. Both qualitative and quantitative results demonstrate that the accuracy of the proposed method surpasses that of existing models. In a phantom study, compared with the Born conjugate gradient descent (Born-CGD) reconstruction method, the ML-PC method decreases the mean percentage error of the reconstructed maximum absorption coefficient from 16.41% to 13.4% for high contrast phantoms and from 23.42% to 9.06% for low contrast phantoms, with improved depth distribution of the target absorption maps. In a clinical study, better contrast was obtained between malignant and benign breast lesions, with the ratio of the medians of the maximum absorption coefficient improved from 1.63 to 2.22.

Rectal Cancer Treatment Management: Deep-Learning Neural Network Based on Photoacoustic Microscopy Image Outperforms Histogram-Feature-Based Classification.

We have developed a novel photoacoustic microscopy/ultrasound (PAM/US) endoscope to image post-treatment rectal cancer for surgical management of residual tumor after radiation and chemotherapy. Paired with a deep-learning convolutional neural network (CNN), the PAM images accurately differentiated pathological complete responders (pCR) from incomplete responders. However, the role of CNNs compared with traditional histogram-feature based classifiers needs further exploration. In this work, we compare the performance of the CNN models to generalized linear models (GLM) across 24 ex vivo specimens and 10 in vivo patient examinations. First order statistical features were extracted from histograms of PAM and US images to train, validate and test GLM models, while PAM and US images were directly used to train, validate, and test CNN models. The PAM-CNN model performed superiorly with an AUC of 0.96 (95% CI: 0.95-0.98) compared to the best PAM-GLM model using kurtosis with an AUC of 0.82 (95% CI: 0.82-0.83). We also found that both CNN and GLMs derived from photoacoustic data outperformed those utilizing ultrasound alone. We conclude that deep-learning neural networks paired with photoacoustic images is the optimal analysis framework for determining presence of residual cancer in the treated human rectum.

Deep learning-based method to accurately estimate breast tissue optical properties in the presence of the chest wall.

SIGNIFICANCE: In general, image reconstruction methods used in diffuse optical tomography (DOT) are based on diffusion approximation, and they consider the breast tissue as a homogenous, semi-infinite medium. However, the semi-infinite medium assumption used in DOT reconstruction is not valid when the chest wall is underneath the breast tissue. AIM: We aim to reduce the chest wall's effect on the estimated average optical properties of breast tissue and obtain accurate forward model for DOT reconstruction. APPROACH: We propose a deep learning-based neural network approach where a convolution neural network (CNN) is trained to simultaneously obtain accurate optical property values for both the breast tissue and the chest wall. RESULTS: The CNN model shows great promise in reducing errors in estimating the optical properties of the breast tissue in the presence of a shallow chest wall. For patient data, the CNN model predicted the breast tissue optical absorption coefficient, which was independent of chest wall depth. CONCLUSIONS: Our proposed method can be readily used in DOT and diffuse spectroscopy measurements to improve the accuracy of estimated tissue optical properties.