Confocal microscopy based on dual blur depth measurement.

In this paper, we propose a confocal microscopy based on dual blur depth measurement (DBCM). The first blur is defocus blur, and the second blur is artificial convolutional blur. First, the DBCM blurs the defocus image using a known Gaussian kernel and calculates the edge gradient ratio between it and the re-blurred image. Then, the axial measurement of edge positions is based on a calibration measurement curve. Finally, depth information is inferred from the edges using the original image. Experiments show that the DBCM can achieve depth measurement in a single image. In a 10×/0.25 objective, the error measured for a step sample of 4.7397 µm is 0.23 µm. The relative error rate is 4.8%.

Confocal microscopy multi-focus image fusion method based on axial information guidance.

Aiming at the problems of poor anti-interference of existing pixel-level fusion rules and low efficiency of transform domain fusion rules, this study proposes a confocal microscopic multi-focus image fusion method (IGCM) based on differential confocal axial information guidance. Unlike traditional multi-focus image fusion (MFIF) methods, IGCM uses height information rather than grayscale or frequency to determine clear areas. First, the differential confocal axial measurement curve is calibrated to determine the suitable scan step u. Second, the image set required for fusion is constructed by performing a hierarchical scan of the measurement samples. Then, multiple differential image pairs are constructed using the step size u and the set of images, and the extraction area of the current reference image is decided based on the height obtained from the differential image. Finally, the regions determined by each reference image are extracted and the duplicated pixels are averaged to obtain the MFIF image. The results were that IGCM improves the interference immunity based on pixel-level image fusion compared to the maximum peak fusion method. Compared with other MFIFs, IGCM has excellent fusion efficiency while ensuring fusion clarity, which can meet the application scenario of real-time fusion and offers a new approach to panoramic depth images for confocal devices.

Micro 4D Imaging Sensor Using Snapshot Narrowband Imaging Method.

The spectral and depth (SAD) imaging method plays an important role in the field of computer vision. However, accurate depth estimation and spectral image capture from a single image without increasing the volume of the imaging sensor is still an unresolved problem. Our research finds that a snapshot narrow band imaging (SNBI) method can discern wavelength-dependent spectral aberration and simultaneously capture spectral-aberration defocused images for quantitative depth estimation. First, a micro 4D imaging (M4DI) sensor is proposed by integrating a mono-chromatic imaging sensor with a miniaturized narrow-band microarrayed spectral filter mosaic. The appearance and volume of the M4DI sensor are the same as the integrated mono-chromatic imaging sensor. A simple remapping algorithm was developed to separate the raw image into four narrow spectral band images. Then, a depth estimation algorithm is developed to generate 3D data with a dense depth map at every exposure of the M4DI sensor. Compared with existing SAD imaging method, the M4DI sensor has the advantages of simple implementation, low computational burden, and low cost. A proof-of-principle M4DI sensor was applied to sense the depth of objects and to track a tiny targets trajectory. The relative error in the three-dimensional positioning is less than 7% for objects within 1.1 to 2.8 m.

Differential confocal over-range determination method based on an information theory.

The existing differential confocal axial three-dimensional (3D) measurement method cannot determine whether the surface height of the sample in the field of view is within its effective measurement range. Therefore, in this paper, we propose a differential confocal over-range determination method (IT-ORDM) based on an information theory to determine whether the surface height information of the sample to be examined is within the effective measurement range of the differential confocal axial measurement. First, the IT-ORDM finds the boundary position of the axial effective measurement range by the differential confocal axial light intensity response curve. Then the effective intensity measurement ranges of the pre-focus axial response curve (ARC) and the post-focus ARC are determined by the correspondence between the boundary position and the ARC. Finally, the intersection operation of the pre-focus image of effective measurement and the post-focus image of effective measurement is used to realize the extraction of the effective measurement area of the differential confocal image. The experimental results show that the IT-ORDM can effectively determine and restore the 3D shape of the measured sample surface at the reference plane position in the multi-stage sample experiments.

Smartphone-Based Optical Fiber Fluorescence Temperature Sensor.

Optical fiber sensors are one preferred solution for temperature sensing, especially for their capability of real-time monitoring and remote detection. However, many of them still suffer from a huge sensing system and complicated signal demodulate process. In order to solve these problems, we propose a smartphone-based optical fiber fluorescence temperature sensor. All the components, including the laser, filter, fiber coupler, batteries, and smartphone, are integrated into a 3D-printed shell, on the side of which there is a fiber flange used for the sensing probe connection. The fluorescence signal of the rhodamine B solution encapsulated in the sensing probe can be captured by the smartphone camera and extracted into the R value and G value by a self-developed smartphone application. The temperature can be quantitatively measured by the calibrated G/R-temperature relation, which can be unified using the same linear relationship in all solid-liquid-gas environments. The performance verifications prove that the sensor can measure temperature in high accuracy, good stability and repeatability, and has a long conservation time for at least 3 months. The proposed sensor not only can measure the temperature for remote and real-time detection needs, but it is also handheld with a small size of 167 mm × 85 mm × 75 mm supporting on-site applications. It is a potential tool in the temperature sensing field.

Recognition of Vehicles Entering Expressway Service Areas and Estimation of Dwell Time Using ETC Data.

To scientifically and effectively evaluate the service capacity of expressway service areas (ESAs) and improve the management level of ESAs, we propose a method for the recognition of vehicles entering ESAs (VeESAs) and estimation of vehicle dwell times using electronic toll collection (ETC) data. First, the ETC data and their advantages are described in detail, and then the cleaning rules are designed according to the characteristics of the ETC data. Second, we established feature engineering according to the characteristics of VeESA and proposed the XGBoost-based VeESA recognition (VR-XGBoost) model. Studied the driving rules in depth, we constructed a kinematics-based vehicle dwell time estimation (K-VDTE) model. The field validation in Part A/B of Yangli ESA using real ETC transaction data demonstrates that the effectiveness of our proposal outperforms the current state-of-the-art. Specifically, in Part A and Part B, the recognition accuracies of VR-XGBoost are 95.9% and 97.4%, respectively, the mean absolute errors (MAEs) of dwell time are 52 and 14 s, respectively, and the root mean square errors (RMSEs) are 69 and 22 s, respectively. In addition, the confidence level of controlling the MAE of dwell time within 2 min is more than 97%. This work can effectively recognize the VeESA and accurately estimate the dwell time, which can provide a reference idea and theoretical basis for the service capacity evaluation and layout optimization of the ESA.

Automatic optical inspection platform for real-time surface defects detection on plane optical components based on semantic segmentation.

The tendency to increase the accuracy and quality of optical parts inspection can be observed all over the world. The imperfection of manufacturing techniques can cause different defects on the optical component surface, making surface defects inspection a crucial part of the manufacturing of optical components. Currently, the inspection of lenses, filters, mirrors, and other optical components is performed by human inspectors. However, human-based inspections are time-consuming, subjective, and incompatible with a highly efficient high-quality digital workflow. Moreover, they cannot meet the complex criteria of ISO 10110-7 for the quality pass and fail optical element samples. To meet the high demand for high-quality products, intelligent visual inspection systems are being used in many manufacturing processes. Automated surface imperfection detection based on machine learning has become a fascinating and promising area of research, with a great direct impact on different visual inspection applications. In this paper, an optical inspection platform combining parallel deep learning-based image-processing approaches with a high-resolution optomechanical module was developed to detect surface defects on optical plane components. The system involves the mechanical modules, the illumination and imaging modules, and the machine vision algorithm. Dark-field images were acquired using a 2448×2048-pixel line-scanning CMOS camera with 3.45 µm per-pixel resolution. Convolutional neural networks and semantic segmentation were used for a machine vision algorithm to detect and classify defects on captured images of optical bandpass filters. The experimental results on different bandpass filter samples have shown the best performance compared to traditional methods by reaching an impressive detection speed of 0.07 s per image and an overall detection pixel accuracy of 0.923.

Development of a handheld dual-channel optical fiber fluorescence sensor based on a smartphone.

In order to meet the needs of on-site, accurate, fast, and remote detection, we design a smartphone-based handheld dual-channel optical fiber fluorescence sensor (DOFFS), which is composed of a semiconductor laser for exciting fluorescence signals, a smartphone with a dual-bandpass filter for collecting fluorescence signals, a fiber coupler for transmitting light, and batteries for laser power supply. All the components are integrated into a 3D printed shell, on the side of which there are two fiber flanges used for fiber probe connection. The fluorescence signals of green and red quantum dots modified on the fiber probes can be captured by the smartphone camera and calculated by a self-developed Android application. The comparisons of single-channel and dual-channel fluorescence signals with pH show that the performance of the sensor is good. The proposed sensor not only can simultaneously detect dual-channel signals for fast detection needs, but it also is handheld with a small size of 79×57×154mm3 and inner power supply, and the fiber probes can be easily replaced, supporting remote and on-site applications. It is a potential tool for many occasions in many fields.

Contrast-Enhancing Snapshot Narrow-Band Imaging Method for Real-Time Computer-Aided Cervical Cancer Screening.

Composition of cervical precancerous lesions and carcinoma in situ is rich in hemoglobin, unlike healthy tissues. In this study, we aimed to utilize this difference to enhance the contrast between healthy and diseased tissues via snapshot narrow-band imaging (SNBI). Four narrow-band images centered at wavelengths of characteristic absorption/reflection peaks of hemoglobin were captured with zero-time delay in between by a custom-designed SNBI video camera. Then these spectral images were fused in real time into a single combined image to enhance the contrast between normal and abnormal tissues. Finally, a Euclidean distance algorithm was employed to classify the tissue into clinical meaningful tissue types. Two pre-clinical experiments were conducted to validate the proposed method. Experimental results indicate that contrast between different grades of diseased tissues in the SNBI generated image was indeed enhanced, as compared to conventional white light image (WLI). The computer-aided classification accuracy was 100% and 50% as compared to the gold standard histopathological diagnosis results with the SNBI and the conventional WLI methods, respectively. Further, the boundary contour between health tissue, cervical precancerous regions, and carcinoma in situ can be automatically delineated in SNBI. The proposed SNBI method was also fast, and it generated automatic diagnostic results with clear boundary contours at over 11 fps on a Pentium 1.6-GHz laptop. Hence, the proposed SNBI is of great significance to enlarge worldwide the coverage of regular cervical screening program, and to live guide surgeries such as biopsy sample collection and accurate cervical cancer treatment.

Electrostatic Capture Following Laser Microdissection for the Preparation of Homogeneous Biological Specimens.

There is an unmet need for researchers in life sciences and clinical pathology to obtain untainted target cells with very high accuracy, which are suitable for subsequent genome and protein analysis. In this paper, an electrostatic capture laser microdissection technology (ECM) is proposed and explained. Following microscopic identification and separation of target cells from the surrounding tissues using laser cutting, the ECM uses electrostatic forces to capture target cells. Validation experiments indicate that ECM can capture a wide assortment of contamination-free homogeneous samples, ranging from very tiny pieces of a few micrometers in diameter to large pieces with a surface area of over 40,000 µm2. Evidence is also provided indicating that uncontaminated homogeneous tissue materials collected by ECM are suitable for further DNA and RNA analysis. This suggests that ECM capture causes little or no identifiable damage to the collected tissues. This technique has significant advantages compared with existing traditional capture methods, such as very low risk of biological sample damage and the fact that it can be applied to both upright and inverted microscopy. The latter allows for separating target cells in tissue culture. ECM usage provides a cost-effective alternative to more traditional laser capture microdissection techniques.

Real-time multispectral imager for home-based health care.

Multispectral imaging (MSI) is becoming a powerful tool for tissue abnormality detection. Conventional MSI systems, however, are not readily suitable for challenges of routine clinical uses due to the fact that they are expensive, bulky, and time consuming to acquire the data. In this letter we report a novel approach to instrument MSI technology into a handheld, low-cost, standing-alone, real-time operational device that is suitable for home-based health care. It covers techniques used to produce multiple images at discrete signature wavelengths of tissues with a single shot.

Single sensor that outputs narrowband multispectral images.

We report the work of developing a hand-held (or miniaturized), low-cost, stand-alone, real-time-operation, narrow bandwidth multispectral imaging device for the detection of early stage pressure ulcers.

Skeletonization of volumetric angiograms for display.

The display of three-dimensional angiograms can benefit from the knowledge of quantitative shape features such as tangent and curvature of the centerline of vessels. These can be obtained from a curve-like skeleton representation. If connectivity and topology are preserved, and if geometrical constraints such as smoothness and centeredness are satisfied, it is possible to estimate length, orientation, curvature, and torsion. It is also required that no part of the original object be left unrepresented. An efficient method for the identification of such shape components is developed. First, a suitable representation is obtained using a voxel coding approach to yield connected and labeled unit-thick paths. The desired features are estimated from a smoothed version of the skeleton produced by a moving average filter. The computational cost is linear, of the order of N(object), the total number of object voxels contained in the binary volumetric data. The method is also shown to be robust to boundary noise. Examples are discussed.