Multi-laser source interference suppression using the time-coding method for SPAD-based flash DToF systems.

Flash-type direct time-of-flight (DToF) image sensors use an in-pixel successive approximation register time-to-digital converter (SAR TDC) for time quantization. However, in a scene where multiple DToF systems exist simultaneously, different laser signals from multiple sources will produce mutual signal interference between DToF systems, causing the DToF system's incorrect measurement. In this paper, we present a method called time coding, which inserts delay time bins between different working periods to suppress the interference laser together with the SAR TDC. The time-coding method is designed using a 110 nm complementary metal oxide semiconductor (CMOS) technology and verified by behavioral model and circuit simulation. Regardless of traditional systems or systems equipped with time coding, DToF systems with certain patterns of time coding can reduce interference noise by at least 95%, maintaining a measurement accuracy of 99% or higher at long distances.

Method of depth simulation imaging and depth image super-resolution reconstruction for a 2D/3D compatible CMOS image sensor.

This paper presents a depth simulation imaging and depth image super-resolution (SR) method for two-dimensional/three-dimensional compatible CMOS image sensors. A depth perception model is established to analyze the effects of depth imaging parameters and evaluate the real imaging effects. We verify its validity by analyzing the depth error, imaging simulation, and auxiliary physical verification. By means of the depth simulation images, we then propose a depth SR reconstruction algorithm to recover the low-resolution depth maps to the high-resolution depth maps in two types of datasets. With the best situation in depth accuracy kept, the root mean square error (RMSE) of Middlebury dataset images are 0.0156, 0.0179, and 0.0183 m. The RMSE of RGB-D dataset images are 0.0223 and 0.0229 m. Compared with other listed conventional algorithms, our algorithm reduces the RMSE by more than 16.35%, 17.19%, and 23.90% in the Middlebury dataset images. Besides, our algorithm reduces the RMSE by more than 9.71% and 8.76% in the RGB-D dataset images. The recovery effects achieve optimized results.

Mathematical model of potential distribution in a pinned photodiode of an indirect time of flight CMOS image sensor.

This paper focuses on the rapid charge transfer of lock-in pixels in time of flight 3D image sensors. Through the principal analysis, a mathematical model of potential distribution in a pinned photodiode (PPD) in different comb shapes is established. Based on this model, the influence of different comb shapes on the accelerating electric field in PPD is analyzed. The semiconductor device simulation tool SPECTRA is applied to verify the effectiveness of the model, and the simulation results are analyzed and discussed. When the width of comb tooth is in narrow and medium range, the potential changes more obviously with the increase of comb tooth angle α, whereas the potential becomes stable even if the comb tooth angle α increases sharply with the wide comb tooth width. The proposed mathematical model contributes to instructing the design of pixel transferring electrons rapidly and resolving image lag.

Ambient light rejection method for multiphoton coincidence detection in single-photon avalanche diode-based DToF sensors.

This paper presents an adaptive control method used for multiphoton coincidence detection to reduce the effect of ambient light that exists in accessing flight time. Behavioral and statistical models are used to demonstrate the working principle with MATLAB, and the method is achieved through a compact circuit. The adaptive coincidence detection in accessing flight time achieves a higher probability of 66.5% than fixed parameter coincidence detection's 46%, while ambient light intensity is 75 klux. Additionally, it also can achieve a dynamic detection range 43.8 times higher than the fixed parameter detection. The circuit is designed in 0.11 µm complementary metal-oxide semiconductor process, and the area consumption is 0.00178m m 2. The postsimulation experiment through Virtuoso shows that the histogram of coincidence detection under adaptive control circuit is consistent with the behavioral model. The proposed method acieves the coefficient of variance as 0.0495 smaller than fixed parameter coincidence's 0.0853, which means better ambient light tolerance in accessing flight time for three-dimensional imaging.

Mask configurable readout circuit architecture for an ultra-high-resolution CMOS image sensor: publisher's note.

This publisher's note reports a correction in Appl. Opt.61, 2565 (2022)APOPAI0003-693510.1364/AO.453904.

Mask configurable readout circuit architecture for an ultra-high-resolution CMOS image sensor.

As the level of detail in today's images increases, so does the demand for resolution. Due to the necessity of mask stitching technology for full exposure of large array chips, we propose a mask configurable readout circuit architecture, which is suitable for large array structures. However, the stitchable readout circuit architecture has some non-ideal effects: row driver function failure and the column non-consistency problem. In our design, we solve the problem of column non-consistency after stitching. At the same time, we changed the signal transmission structure in order to avoid the row driver function failure caused by the mask stitching. In this paper, a prototype 2130×2130 CMOS image sensor is fabricated in 0.11 µm CMOS technology. The chip can capture images at 20 fps and reduce fixed pattern noise (FPN) from 3.5% to 1.5% through correction techniques. The architecture proposed in this paper is suitable for large array image sensors.

Method for power reduction of demodulation driver circuit in indirect time-of-flight CMOS image sensor.

To improve the depth accuracy of an indirect time-of-flight CMOS image sensor, high modulation frequency is often adopted. It will result in high power consumption of an on-chip demodulation driver, and this problem will be much more serious when the resolution of the sensor is much higher. In this paper, a power reduction method that can lower the power consumption of the demodulation driver circuit during the integration time while obtaining accurate high-resolution depth maps is proposed and analyzed theoretically. The method decreases the number of driven pixels at a high-modulation frequency by a programmable resolution adjustment circuit to obtain an accurate low-resolution depth map. A low-depth accuracy high-resolution depth map is obtained at a low modulation frequency, and then a modified super-resolution algorithm is used to obtain an accurate high resolution solution depth map. To demonstrate the effectiveness of the proposed method, a model is established based on the actual indirect time-of-flight sensor architecture, then the depth error and power consumption are analyzed by the simulation results of the model. In the simulation, 25 MHz and 100 MHz are used as the low modulation frequency and high modulation frequency, respectively. With the best scenario in depth accuracy kept, average power consumption decreases 38.47% and peak power consumption decreases 49.83% while the depth error that is represented by RMSE merely increases 8.08%.

Effect of the Transition Points Mismatch on Quanta Image Sensors.

Mathematical models and imaging models that show the relationship between the transition points mismatch of analog-to-digital converters (ADCs) and the bit error rate (BER) in single-bit and multi-bit quanta image sensors (QISs) are established. The mathematical models suggest that when the root-mean-square (r.m.s.) of the read noise in jots is 0.15e-, the standard deviation of the transition points should be less than 0.15e- to ensure that the BER is lower than 1% in the single-bit QIS, and 0.21e- to ensure that the BER is lower than 5% in the multi-bit QIS. Based on the mathematical models, the imaging models prove that the fixed-pattern noise (FPN) increases with a stronger transition point mismatch. The imaging models also compare the imaging quality in the case of different spatial oversampling factors and bit depths. The grayscale similarity index (GSI) is 3.31 LSB and 1.74 LSB when the spatial oversampling factors are 256 and 4096, respectively, in the single-bit QIS. The GSI is 1.93 LSB and 1.13 LSB when the bit depth is 3 and 4, respectively, in the multi-bit QIS. It indicates that a higher bit depth and a larger spatial oversampling factor could reduce the effect of the transition points mismatch of1-bit or n-bit ADCs.

The Analysis and Suppressing of Non-Uniformity in a High-Speed Spike-Based Image Sensor.

In this paper, the non-ideal factors, which include spatial noise and temporal noise, are analyzed and suppressed in the high-speed spike-based image sensor, which combines the high-speed scanning sequential format with the method that uses the interspike time interval to indicate the scene information. In this imager, spatial noise contains device mismatch, which results in photo response non-uniformity (PRNU) and the non-uniformity of dark current. By multiplying the measured coefficient matrix the photo response non-uniformity is suppressed, and the non-uniformity of dark current is suppressed by correcting the interspike time interval based on the time interval of dark current. The temporal noise is composed of the shot noise and thermal noise. This kind of noise can be eliminated when using the spike frequency to restore the image. The experimental results show that, based on the spike frequency method, the standard deviation of the image decreases from 18.4792 to 0.5683 in the uniform bright light by using the calibration algorithm. While in the relatively uniform dark condition, the standard deviation decreases from 1.5812 to 0.4516. Based on interspike time interval method, because of time mismatch and temporal noise, the standard deviation of the image changes from 27.4252 to 27.4977 in the uniform bright light by using the calibration algorithm. While in the uniform dark condition, the standard deviation decreases from 2.361 to 0.3678.

Fixed-pattern noise correction method based on improved moment matching for a TDI CMOS image sensor.

In this paper, an improved moment matching method based on a spatial correlation filter (SCF) and bilateral filter (BF) is proposed to correct the fixed-pattern noise (FPN) of a time-delay-integration CMOS image sensor (TDI-CIS). First, the values of row FPN (RFPN) and column FPN (CFPN) are estimated and added to the original image through SCF and BF, respectively. Then the filtered image will be processed by an improved moment matching method with a moving window. Experimental results based on a 128-stage TDI-CIS show that, after correcting the FPN in the image captured under uniform illumination, the standard deviation of row mean vector (SDRMV) decreases from 5.6761 LSB to 0.1948 LSB, while the standard deviation of the column mean vector (SDCMV) decreases from 15.2005 LSB to 13.1949LSB. In addition, for different images captured by different TDI-CISs, the average decrease of SDRMV and SDCMV is 5.4922/2.0357 LSB, respectively. Comparative experimental results indicate that the proposed method can effectively correct the FPNs of different TDI-CISs while maintaining image details without any auxiliary equipment.

Calibration method for the center of mass method to enlarge the solvable range of fluorescence lifetime.

This paper presents a calibration method for the center of mass method based on rough rapid lifetime determination (RLD) to enlarge the solvable range of fluorescence lifetime. The proposed method defines the ratio of two photon count numbers as a threshold parameter to characterize the length of the sample lifetime. When detecting long lifetimes beyond the threshold, a raw lifetime is estimated first through RLD. Then the raw lifetime is compensated to get a precise one. Simulation results show the solvable range is extended from T/τ>4 to T/τ>1.5 with less than 1% error. The extended range with 40 dB SNR guaranteed enables higher-frequency laser pulses to solve long lifetimes or incomplete decays and has promising biomedical applications, such as quantum dots.

A Low Power Digital Accumulation Technique for Digital-Domain CMOS TDI Image Sensor.

In this paper, an accumulation technique suitable for digital domain CMOS time delay integration (TDI) image sensors is proposed to reduce power consumption without degrading the rate of imaging. In terms of the slight variations of quantization codes among different pixel exposures towards the same object, the pixel array is divided into two groups: one is for coarse quantization of high bits only, and the other one is for fine quantization of low bits. Then, the complete quantization codes are composed of both results from the coarse-and-fine quantization. The equivalent operation comparably reduces the total required bit numbers of the quantization. In the 0.18 µm CMOS process, two versions of 16-stage digital domain CMOS TDI image sensor chains based on a 10-bit successive approximate register (SAR) analog-to-digital converter (ADC), with and without the proposed technique, are designed. The simulation results show that the average power consumption of slices of the two versions are 6 . 47 × 10 - 8 J/line and 7 . 4 × 10 - 8 J/line, respectively. Meanwhile, the linearity of the two versions are 99.74% and 99.99%, respectively.

A Full Parallel Event Driven Readout Technique for Area Array SPAD FLIM Image Sensors.

This paper presents a full parallel event driven readout method which is implemented in an area array single-photon avalanche diode (SPAD) image sensor for high-speed fluorescence lifetime imaging microscopy (FLIM). The sensor only records and reads out effective time and position information by adopting full parallel event driven readout method, aiming at reducing the amount of data. The image sensor includes four 8 × 8 pixel arrays. In each array, four time-to-digital converters (TDCs) are used to quantize the time of photons' arrival, and two address record modules are used to record the column and row information. In this work, Monte Carlo simulations were performed in Matlab in terms of the pile-up effect induced by the readout method. The sensor's resolution is 16 × 16. The time resolution of TDCs is 97.6 ps and the quantization range is 100 ns. The readout frame rate is 10 Mfps, and the maximum imaging frame rate is 100 fps. The chip's output bandwidth is 720 MHz with an average power of 15 mW. The lifetime resolvability range is 5-20 ns, and the average error of estimated fluorescence lifetimes is below 1% by employing CMM to estimate lifetimes.

A Dynamic Range Enhanced Readout Technique with a Two-Step TDC for High Speed Linear CMOS Image Sensors.

This paper presents a dynamic range (DR) enhanced readout technique with a two-step time-to-digital converter (TDC) for high speed linear CMOS image sensors. A multi-capacitor and self-regulated capacitive trans-impedance amplifier (CTIA) structure is employed to extend the dynamic range. The gain of the CTIA is auto adjusted by switching different capacitors to the integration node asynchronously according to the output voltage. A column-parallel ADC based on a two-step TDC is utilized to improve the conversion rate. The conversion is divided into coarse phase and fine phase. An error calibration scheme is also proposed to correct quantization errors caused by propagation delay skew within -T(clk)~+T(clk). A linear CMOS image sensor pixel array is designed in the 0.13 µm CMOS process to verify this DR-enhanced high speed readout technique. The post simulation results indicate that the dynamic range of readout circuit is 99.02 dB and the ADC achieves 60.22 dB SNDR and 9.71 bit ENOB at a conversion rate of 2 MS/s after calibration, with 14.04 dB and 2.4 bit improvement, compared with SNDR and ENOB of that without calibration.

A Fixed-Pattern Noise Correction Method Based on Gray Value Compensation for TDI CMOS Image Sensor.

In order to eliminate the fixed-pattern noise (FPN) in the output image of time-delay-integration CMOS image sensor (TDI-CIS), a FPN correction method based on gray value compensation is proposed. One hundred images are first captured under uniform illumination. Then, row FPN (RFPN) and column FPN (CFPN) are estimated based on the row-mean vector and column-mean vector of all collected images, respectively. Finally, RFPN are corrected by adding the estimated RFPN gray value to the original gray values of pixels in the corresponding row, and CFPN are corrected by subtracting the estimated CFPN gray value from the original gray values of pixels in the corresponding column. Experimental results based on a 128-stage TDI-CIS show that, after correcting the FPN in the image captured under uniform illumination with the proposed method, the standard-deviation of row-mean vector decreases from 5.6798 to 0.4214 LSB, and the standard-deviation of column-mean vector decreases from 15.2080 to 13.4623 LSB. Both kinds of FPN in the real images captured by TDI-CIS are eliminated effectively with the proposed method.

A 12-bit high-speed column-parallel two-step single-slope analog-to-digital converter (ADC) for CMOS image sensors.

A 12-bit high-speed column-parallel two-step single-slope (SS) analog-to-digital converter (ADC) for CMOS image sensors is proposed. The proposed ADC employs a single ramp voltage and multiple reference voltages, and the conversion is divided into coarse phase and fine phase to improve the conversion rate. An error calibration scheme is proposed to correct errors caused by offsets among the reference voltages. The digital-to-analog converter (DAC) used for the ramp generator is based on the split-capacitor array with an attenuation capacitor. Analysis of the DAC's linearity performance versus capacitor mismatch and parasitic capacitance is presented. A prototype 1024 × 32 Time Delay Integration (TDI) CMOS image sensor with the proposed ADC architecture has been fabricated in a standard 0.18 µm CMOS process. The proposed ADC has average power consumption of 128 µW and a conventional rate 6 times higher than the conventional SS ADC. A high-quality image, captured at the line rate of 15.5 k lines/s, shows that the proposed ADC is suitable for high-speed CMOS image sensors.