Artifact reduction in photoacoustic images by generating virtual dense array sensor from hemispheric sparse array sensor using deep learning.

PURPOSE: Vascular distribution is important information for diagnosing diseases and supporting surgery. Photoacoustic imaging is a technology that can image blood vessels noninvasively and with high resolution. In photoacoustic imaging, a hemispherical array sensor is especially suitable for measuring blood vessels running in various directions. However, as a hemispherical array sensor, a sparse array sensor is often used due to technical and cost issues, which causes artifacts in photoacoustic images. Therefore, in this study, we reduce these artifacts using deep learning technology to generate signals of virtual dense array sensors. METHODS: Generating 2D virtual array sensor signals using a 3D convolutional neural network (CNN) requires huge computational costs and is impractical. Therefore, we installed virtual sensors between the real sensors along the spiral pattern in three different directions and used a 2D CNN to generate signals of the virtual sensors in each direction. Then we reconstructed a photoacoustic image using the signals from both the real sensors and the virtual sensors. RESULTS: We evaluated the proposed method using simulation data and human palm measurement data. We found that these artifacts were significantly reduced in the images reconstructed using the proposed method, while the artifacts were strong in the images obtained only from the real sensor signals. CONCLUSION: Using the proposed method, we were able to significantly reduce artifacts, and as a result, it became possible to recognize deep blood vessels. In addition, the processing time of the proposed method was sufficiently applicable to clinical measurement.

Novel image-guided marker aimed at organ-preserving therapies for prostate cancer.

OBJECTIVE: We developed fiducial imaging-guidance markers for the prostate with less imaging artifacts than currently commercially available markers. The aim of this study was to evaluate the imaging artifacts and potential usefulness and safety of these novel fiducial imaging markers in preclinical experiments. METHODS: We selected specific metal materials and a shape that can minimize artifacts in line with a license we obtained for a metal with a gold-platinum (Au-Pt) alloy composition that maximized artifact-free MRI images. Both phantom and canine prostate tests were conducted in order to evaluate the imaging artifacts for three imaging modalities, MRI, CT and ultrasound, and the risk of migration of the markers from the site of insertion to elsewhere, as well as crushing. RESULTS: The newly developed Au-Pt material had less imaging artifacts in the MRI, CT and ultrasound imaging modalities in comparison with current commercially available fiducial markers made from gold materials only. The Au-Pt markers had sufficient strength and durability and were considered to be potentially clinically useful and safe markers. CONCLUSION: The developed Au-Pt markers could be potential tools for accurate lesion-targeted, organ-preserving therapies such as lesion-targeted focal therapy and active surveillance in addition to conventional radiation therapies.

A study on the optimal condition of ground truth area for liver tumor detection in ultrasound images using deep learning.

PURPOSE: In recent years, efforts to apply artificial intelligence (AI) to the medical field have been growing. In general, a vast amount of high-quality training data is necessary to make great AI. For tumor detection AI, annotation quality is important. In diagnosis and detection of tumors using ultrasound images, humans use not only the tumor area but also the surrounding information, such as the back echo of the tumor. Therefore, we investigated changes in detection accuracy when changing the size of the region of interest (ROI, ground truth area) relative to liver tumors in the training data for the detection AI. METHODS: We defined D/L as the ratio of the maximum diameter (D) of the liver tumor to the ROI size (L). We created training data by changing the D/L value, and performed learning and testing with YOLOv3. RESULTS: Our results showed that the detection accuracy was highest when the training data were created with a D/L ratio between 0.8 and 1.0. In other words, it was found that the detection accuracy was improved by setting the ground true bounding box for detection AI training to be in contact with the tumor or slightly larger. We also found that when the D/L ratio was distributed in the training data, the wider the distribution, the lower the detection accuracy. CONCLUSIONS: Therefore, we recommend that the detector be trained with the D/L value close to a certain value between 0.8 and 1.0 for liver tumor detection from ultrasound images.

Artificial intelligence (AI) models for the ultrasonographic diagnosis of liver tumors and comparison of diagnostic accuracies between AI and human experts.

BACKGROUND: Ultrasonography (US) is widely used for the diagnosis of liver tumors. However, the accuracy of the diagnosis largely depends on the visual perception of humans. Hence, we aimed to construct artificial intelligence (AI) models for the diagnosis of liver tumors in US. METHODS: We constructed three AI models based on still B-mode images: model-1 using 24,675 images, model-2 using 57,145 images, and model-3 using 70,950 images. A convolutional neural network was used to train the US images. The four-class liver tumor discrimination by AI, namely, cysts, hemangiomas, hepatocellular carcinoma, and metastatic tumors, was examined. The accuracy of the AI diagnosis was evaluated using tenfold cross-validation. The diagnostic performances of the AI models and human experts were also compared using an independent test cohort of video images. RESULTS: The diagnostic accuracies of model-1, model-2, and model-3 in the four tumor types are 86.8%, 91.0%, and 91.1%, whereas those for malignant tumor are 91.3%, 94.3%, and 94.3%, respectively. In the independent comparison of the AIs and physicians, the percentages of correct diagnoses (accuracies) by the AIs are 80.0%, 81.8%, and 89.1% in model-1, model-2, and model-3, respectively. Meanwhile, the median percentages of correct diagnoses are 67.3% (range 63.6%-69.1%) and 47.3% (45.5%-47.3%) by human experts and non-experts, respectively. CONCLUSION: The performance of the AI models surpassed that of human experts in the four-class discrimination and benign and malignant discrimination of liver tumors. Thus, the AI models can help prevent human errors in US diagnosis.

Shear wave speed measurement bias in a viscoelastic phantom across six ultrasound elastography systems: a comparative study with transient elastography and magnetic resonance elastography.

PURPOSE: To quantify the bias of shear wave speed (SWS) measurements in a viscoelastic phantom across six different ultrasound (US) systems and to compare the SWS with those from transient elastography (TE) and magnetic resonance elastography (MRE). METHODS: A viscoelastic phantom of stiffness representing fibrotic liver or healthy thyroid was measured with nine (linear probe) and 10 (convex probe) modes of six different US-based shear wave elastography (SWE) systems using linear and convex probes. SWS measurements of three regions of interest were repeated thrice at two focal depths, coupling the probe to the phantom using a jig. An MRE system using three motion-encoding gradient frequencies of 60, 90, and 120 Hz and TE were also used to measure the stiffness of the phantom. RESULTS: The SWS from different SWE systems had mean coefficients of variation of 9.0-9.2% and 5.4-5.6% with linear and convex probes, respectively, in viscoelastic phantom measurement. The focal depth was a less significant source of SWS variability than the system. The total average SWS obtained with US-SWE systems was 19.9% higher than that obtained with MRE at 60 Hz, which is commonly used in clinical practice, and 31.5% higher than that obtained with TE using the M probe. CONCLUSIONS: Despite the measurement biases associated with the SWE systems, biases were not necessarily consistent, and they changed with the probes used and depth measured. The SWS of the viscoelastic phantom obtained using different modalities increased according to the shear wave frequency used.

A review of physical and engineering factors potentially affecting shear wave elastography.

It has been recognized that tissue stiffness provides useful diagnostic information, as with palpation as a screening for diseases such as cancer. In recent years, shear wave elastography (SWE), a technique for evaluating and imaging tissue elasticity quantitatively and objectively in diagnostic imaging, has been put into practical use, and the amount of clinical knowledge about SWE has increased. In addition, some guidelines and review papers regarding technology and clinical applications have been published, and the status as a diagnostic technology is in the process of being established. However, there are still unclear points about the interpretation of shear wave speed (SWS) and converted elastic modulus in SWE. To clarify these, it is important to investigate the factors that affect the SWS and elastic modulus. Therefore, physical and engineering factors that potentially affect the SWS and elastic modulus are discussed in this review paper, based on the principles of SWE and a literature review. The physical factors include the propagation properties of shear waves, mechanical properties (viscoelasticity, nonlinearity, and anisotropy), and size and shape of target tissues. The engineering factors include the region of interest depth and signal processing. The aim of this review paper is not to provide an answer to the interpretation of SWS. It is to provide information for readers to formulate and verify the hypothesis for the interpretation. Therefore, methods to verify the hypothesis for the interpretation are also reviewed. Finally, studies on the safety of SWE are discussed.

Development of Gold Nanorods Conjugated with Radiolabeled Anti-human Epidermal Growth Factor Receptor 2 (HER2) Monoclonal Antibody as Single-Photon Emission Computed Tomography/Photoacoustic Dual-Imaging Probes Targeting HER2-Positive Tumors.

Surgery remains one of the main treatments of cancer and both precise pre- and intraoperative diagnoses are crucial in order to guide the operation. We consider that using an identical probe for both pre- and intra-operative diagnoses would bridge the gap between surgical planning and image-guided resection. Therefore, in this study, we developed gold nanorods (AuNRs) conjugated with radiolabeled anti-human epidermal growth factor receptor 2 (HER2) monoclonal antibody, and investigated their feasibility as novel HER2-targeted dual-imaging probes for both single photon emission computed tomography (SPECT) (preoperative diagnosis) and photoacoustic (PA) imaging (intraoperative diagnosis). To achieve the purpose, AuNRs conjugated with different amount of trastuzumab (Tra) were prepared, and Tra-AuNRs were labeled with indium-111. After the evaluation of binding affinity to HER2, cell binding assay and biodistribution studies were carried out for optimization. AuNRs with moderate trastuzumab conjugation (Tra2-AuNRs) were proposed as the novel probe and demonstrated significantly higher accumulation in NCI-N87 (HER2 high-expression) tumors than in SUIT2 (low-expression) tumors 96 h post-injection along with good affinity towards HER2. Thereafter, in vitro PA imaging and in vivo SPECT imaging studies were performed. In in vitro PA imaging, Tra2-AuNRs-treated N87 cells exhibited significant PA signal increase than SUIT2 cells. In in vivo SPECT, signal increase in N87 tumors was more notable than that in SUIT2 tumors. Herein, we report that the Tra2-AuNRs enabled HER2-specific imaging, suggesting the potential as a robust HER2-targeted SPECT and PA dual-imaging probe.

Study for quantitative evaluation of photoaging with photoacoustic microscopy.

Skin aging caused by ultraviolet light exposure is one of the serious problems from the viewpoint of beauty and healthcare. This is because ultraviolet light can cause age spot, wrinkles, at the worst case, skin cancer and so on. To evaluate skin aging, various modalities are being used, such as histopathological diagnosis, optical coherence tomography, ultrasound examination (B-mode imaging). However, they have disadvantages in terms of invasiveness, penetration depth and tissue specificity, respectively. To overcome these defects, photoacoustic imaging (PAI), a novel modality was used in this work. This modality can sense differences of tissue characteristics non-invasively. In this experiment, human skin tissues in various generations (i.e. various degrees of photoaging) were measured by using acoustic resolution photoacoustic microscopy (AR-PAM). To verify the feasibility of quantitative skin aging evaluation with PA technique, signals from sectioned human skin (cheek and buttock; female from 28 to 95 years old) were measured with PA microscopy. The effects of photoaging progress on the signal intensity were investigated. The results demonstrated that the PA signal from the dermis significantly increases with aging progress (p < 0.05). These analyses demonstrate the feasibility of quantitative skin aging evaluation with a PAI system.

Sensitive Photoacoustic/Magnetic Resonance Dual Imaging Probe for Detection of Malignant Tumors.

In order to completely remove tumors in surgeries, probes are needed both preoperatively and intraoperatively. For tumor diagnosis, magnetic resonance imaging (MRI) has been widely used as a precise preoperative method, and photoacoustic imaging (PAI) is a recently emerged intraoperative (or preoperative) method, which detects ultrasonic waves thermoelastically induced by optical absorbers irradiated by laser. Iron oxide nanoparticles (IONPs) can be used as both MR and PA imaging probes. In order to improve the sensitivity of IONPs as MR/PA imaging probes, we newly prepared liposomes encapsulated with a number of IONPs (Lipo-IONPs). Interestingly, Lipo-IONPs showed 2.6 and 3.8-times higher PA and MR signals, respectively, compared to dispersed IONPs at the same concentration. Furthermore, trastuzumab (Tra) (anti-human epidermal growth factor receptor 2 (EGFR2; HER2) monoclonal antibody) was introduced onto the surface of liposomes for detection of HER2 related to tumor malignancy. In an cellular uptake study, Tra-Lipo-IONPs were taken up by HER2-positive tumor cells and HER2-specific MR/PA dual imaging was achieved. Finally, a biodistribution study using radiolabeled Tra-Lipo-IONPs showed HER2-specific tumor accumulation. In conclusion, we demonstrated the usefulness of Lipo-IONPs as platforms for sensitive MR/PA dual imaging and the possibility of HER2-specific tumor MR/PA imaging using Tra-Lipo-IONPs.

Correction to: Intervendor variability of carotid intima-media thickness measurement: validation study using newly developed ultrasound phantom.

In the Original publication of the article Table 3 was published incorrectly.

Intervendor variability of carotid intima-media thickness measurement: validation study using newly developed ultrasound phantom.

PURPOSE: Ultrasonography-derived carotid artery intima-media thickness (IMT) has been established as an early atherosclerotic imaging biomarker. The IMT reference value of a healthy person is approximately 0.1 × (every 10 years of age) + 0.2 (mm); accordingly, it requires an accuracy of at least 0.1 mm. However, one concern of IMT measurement is intervendor variability. In this study, we aimed to verify the intervendor variability using an IMT phantom. METHODS: An improved IMT phantom was developed, and it was possible to analyze the IMT by software for all vendors. RESULTS: With the vendor-specific software, the maximum difference between the devices was 0.08 mm, and the difference in quartile range was 0.06 mm. On the other hand, with the vendor-independent offline software, the maximum difference between the devices was 0.16 mm, and the quartile range of variation was 0.06 mm. CONCLUSION: The intervendor variability assessed using our IMT phantom was less than 0.10 mm, and the on-board vendor-specific software was shown to reduce the difference between the devices significantly compared with the vendor-independent offline software. To further improve the vender difference, adjustment by means of vendor-specific software based on a standardized IMT phantom is warranted.

Ring-array photoacoustic tomography for imaging human finger vasculature.

For early diagnosis of rheumatoid arthritis (RA), it is important to visualize its potential marker, vascularization in the synovial membrane of the finger joints. Photoacoustic (PA) imaging, which can image blood vessels at high contrast and resolution, is expected to be a potential modality for earlier diagnosis of RA. In previous studies of PA finger imaging, different acoustic schemes, such as linear-shaped arrays, have been utilized, but these have limited detection views, rendering inaccurate reconstruction, and most of them require rotational detection. We are developing a PA system for finger vascular imaging using a ring-shaped array ultrasound (US) transducer. By designing the ring-array sensor based on simulations, using phantom experiments, it was demonstrated that we have created a system that can image small objects around 0.1 to 0.5 mm in diameter. The full width at half maximum of the slice direction of the system was within 2 mm and corresponded to that of the simulation. Moreover, we could clearly visualize healthy index finger vasculature and the location of the distal interphalangeal and proximal interphalangeal joints by PA and US echo images. In the future, this system could be used as a method for visualizing the three-dimensional vascularization of RA patients' fingers.

Current status and perspectives for computer-aided ultrasonic diagnosis of liver lesions using deep learning technology.

An ultrasound (US) examination is a common noninvasive technique widely applied for diagnosis of a variety of diseases. Based on the rapid development of US equipment, many US images have been accumulated and are now available and ready for the preparation of a database for the development of computer-aided US diagnosis with deep learning technology. On the contrary, because of the unique characteristics of the US image, there could be some issues that need to be resolved for the establishment of computer-aided diagnosis (CAD) system in this field. For example, compared to the other modalities, the quality of a US image is, currently, highly operator dependent; the conditions of examination should also directly affect the quality of US images. So far, these factors have hampered the application of deep learning-based technology in the field of US diagnosis. However, the development of CAD and US technologies will contribute to an increase in diagnostic quality, facilitate the development of remote medicine, and reduce the costs in the national health care through the early diagnosis of diseases. From this point of view, it may have a large enough potential to induce a paradigm shift in the field of US imaging and diagnosis of liver diseases.

Cancer Cell-Derived Granulocyte-Macrophage Colony-Stimulating Factor Is Dispensable for the Progression of 4T1 Murine Breast Cancer.

We previously reported that 4T1 murine breast cancer cells produce GM-CSF that up-regulates macrophage expression of several cancer promoting genes, including Mcp-1/Ccl2, Ccl17 and Rankl, suggesting a critical role of cancer cell-derived GM-CSF in cancer progression. Here, we attempted to define whether 4T1 cell-derived GM-CSF contributes to the expression of these genes by 4T1tumors, and their subsequent progression. Intraperitoneal injection of anti-GM-CSF neutralizing antibody did not decrease the expression of Mcp-1, Ccl17 or Rankl mRNA by 4T1 tumors. To further examine the role of cancer cell-derived GM-CSF, we generated GM-CSF-deficient 4T1 cells by using the Crisper-Cas9 system. As previously demonstrated, 4T1 cells are a mixture of cells and cloning of cells by itself significantly reduced tumor growth and lung metastasis. By contrast, GM-CSF-deficiency did not affect tumor growth, lung metastasis or the expression of these chemokine and cytokine genes in tumor tissues. By in-situ hybridization, the expression of Mcp-1 mRNA was detected in both F4/80-expressing and non-expressing cells in tumors of GM-CSF-deficient cells. These results indicate that cancer cell-derived GM-CSF is dispensable for the tuning of the 4T1 tumor microenvironment and the production of MCP-1, CCL17 or RANKL in the 4T1 tumor microenvironment is likely regulated by redundant mechanisms.

Visualising peripheral arterioles and venules through high-resolution and large-area photoacoustic imaging.

Photoacoustic (PA) imaging (PAI) has been shown to be a promising tool for non-invasive blood vessel imaging. A PAI system comprising a hemispherical detector array (HDA) has been reported previously as a method providing high morphological reproducibility. However, further improvements in diagnostic capability will require improving the image quality of PAI and fusing functional and morphological imaging. Our newly developed PAI system prototype not only enhances the PA image resolution but also acquires ultrasonic (US) B-mode images at continuous positions in the same coordinate axes. In addition, the pulse-to-pulse alternating laser irradiation shortens the measurement time difference between two wavelengths. We scanned extremities and breasts in an imaging region 140 mm in diameter and obtained 3D-PA images of fine blood vessels, including arterioles and venules. We could estimate whether a vessel was an artery or a vein by using the S-factor obtained from the PA images at two wavelengths, which corresponds approximately to the haemoglobin oxygen saturation. Furthermore, we observed tumour-related blood vessels around breast tumours with unprecedented resolution. In the future, clinical studies with our new PAI system will help to elucidate various mechanisms of vascular-associated diseases and events.

Vascular branching point counts using photoacoustic imaging in the superficial layer of the breast: A potential biomarker for breast cancer.

This study aimed to identify the characteristics of the vascular network in the superficial subcutaneous layer of the breast and to analyze differences between breasts with cancer and contralateral unaffected breasts using vessel branching points (VBPs) detected by three-dimensional photoacoustic imaging with a hemispherical detector array. In 22 patients with unilateral breast cancer, the average VBP counts to a depth of 7 mm below the skin surface were significantly greater in breasts with cancer than in the contralateral unaffected breasts (p < 0.01). The ratio of the VBP count in the breasts with cancer to that in the contralateral breasts was significantly increased in patients with a high histologic grade (p = 0.03), those with estrogen receptor-negative disease (p < 0.01), and those with highly proliferative disease (p < 0.01). These preliminary findings indicate that a higher number of VBPs in the superficial subcutaneous layer of the breast might be a biomarker for primary breast cancer.

Real-time 3D Photoacoustic Visualization System with a Wide Field of View for Imaging Human Limbs.

Background: A breast-specific photoacoustic imaging (PAI) system prototype equipped with a hemispherical detector array (HDA) has been reported as a promising system configuration for providing high morphological reproducibility for vascular structures in living bodies. Methods: To image the vasculature of human limbs, a newly designed PAI system prototype (PAI-05) with an HDA with a higher density sensor arrangement was developed. The basic device configuration mimicked that of a previously reported breast-specific PAI system. A new imaging table and a holding tray for imaging a subject's limb were adopted. Results: The device's performance was verified using a phantom. Contrast of 8.5 was obtained at a depth of 2 cm, and the viewing angle reached up to 70 degrees, showing sufficient performance for limb imaging. An arbitrary wavelength was set, and a reasonable PA signal intensity dependent on the wavelength was obtained. To prove the concept of imaging human limbs, various parts of the subject were scanned. High-quality still images of a living human with a wider size than that previously reported were obtained by scanning within the horizontal plane and averaging the images. The maximum field of view (FOV) was 270 mm × 180 mm. Even in movie mode, one-shot 3D volumetric data were obtained in an FOV range of 20 mm in diameter, which is larger than values in previous reports. By continuously acquiring these images, we were able to produce motion pictures. Conclusion: We developed a PAI prototype system equipped with an HDA suitable for imaging limbs. As a result, the subject could be scanned over a wide range while in a more comfortable position, and high-quality still images and motion pictures could be obtained.

Development and clinical translation of photoacoustic mammography.

To practically apply photoacoustic (PA) imaging technology in medicine, we have developed prototypes of a photoacoustic mammography (PAM) device to acquire images for diagnosing breast cancer in the Kyoto University/Canon joint research project (CK project supported by MEXT, Japan). First, the basic ability of the PAM system to visualize the network of blood vessels and the Hb saturation index was evaluated using a prototype of PAM that has a flat scanning detector and is capable of simultaneously acquiring photoacoustic (PA) and ultrasound images. Next, another prototype of a PAM device with hemispherical sensors was developed to improve the visibility of the 3D structure of vessels by reducing the limited view effect. In clinical examination of breast cancer cases, the PAM system allowed 3D visualization of fine vessel networks with a spatial resolution of a half-millimeter and enabled us to determine the features of tumor-related vascular structures in human breast cancer. In addition, the oxygen saturation status of Hb was visualized using two different wavelengths, enabling more precise characterization of the tumor microenvironment. Results of clinical evaluation using our developed prototype of a PAM device confirmed that PA imaging technology has the potential to promote early detection of breast cancer, and realization of its practical use is expected in the near future.

Visualization of photoacoustic images in a limited-View measuring system using eigenvalues of a photoacoustic transmission matrix.

Photoacoustic imaging is a unique imaging method that involves extracting information from points at different depths, an advantage of ultrasound imaging, while maintaining functional information, a key feature of conventional photo imaging. This makes it easy to add functional images to ultrasound images by adding a laser pulse source to the conventional ultrasound imaging device and detecting a photo-ultrasound signal via a conventional ultrasound probe. One challenge when using normal one-dimensional (1D) probes and generating photoacoustic images is the limited-view problem, in which artefacts are observed due to the positions of the ultrasound transducers. In this study, we used a photoacoustic transmission matrix (PA-TM) for simulation and performed a verification test using a 1D probe and a phantom. The results confirmed that the eigenvalues of the PA-TM visualized the light absorber itself in the limited-view measurement system, which eliminates reconstruction artefacts and further scattering artefacts, and that visualization is possible by signal intensity amplification through further phase modulation.