Key frame extraction method for lecture videos based on spatio-temporal subtitles.

Affected by the Corona Virus Disease 2019 (COVID-19), online lecture videos have witnessed an explosive growth. In the face of massive videos, this paper proposes a method for extracting key frames of lecture videos based on spatio-temporal subtitles, which can efficiently and quickly obtain effective information. Firstly, the spatio-temporal slices of subtitle area of the video sequence are extracted and spliced along the time axis to construct the video spatio-temporal subtitle. Then, the video spatio-temporal subtitle is processed in binarization, and the projection method is used to construct the SSPA curve of the video spatio-temporal subtitle. Finally, a selection method for steady-state key frame is designed, that is, the key frame extraction is realized by combining curve edge detection and subtitle existence threshold, which ensures the robustness of the proposed method. The test results of 8 videos show that the average value of the comprehensive index F1-score of the key frame extracted by the algorithm can reach 0.97, the average precision is 0.97, and the average recall rate is 0.98. It can effectively extract the key frames in lecture videos, and compared with other algorithms, the average running time is reduced to 0.072 of the original, which is helpful to extract video information quickly and accurately.

Robust registration for ultra-field infrared and visible binocular images.

The ultra-field infrared and visible image registration is a challenging task due to its nonlinear imaging and multi-modal image features. In this paper, a robust registration method is proposed for the ultra-field infrared and visible images. First, control points are extracted utilizing phase congruency and optimized based on the guidance map, which is proposed according to significant structures information. Second, ROI pair matching is accomplished based on epipolar curve. Its effect is equivalent to a search window that is popular in methods with the standard field of view, and it can overcome the content differences in the search window caused by nonlinear imaging and vision disparity. Third, a descriptor, named multiple phase congruency directional pattern (MPCDP), is established and composed of distribution information and main direction. The phase congruency amplitudes are encoded as binary patterns, and then they are represented as weighted histogram for distribution information. Six pairs of ultra-field infrared and visible images are employed for registration experiments, and the results demonstrate that the performance of the proposed is robust and accurate in five types of ultra-field scenes and two different camera relationships.

High-performance compact spectrometer based on multimode interference in a tapered spiral-shaped waveguide.

Multimode interference patterns are strongly dependent on spectral components and can be used as fingerprints to reconstruct a spectrum with random amplitudes. Motivated by this concept, we designed and realized a high-performance compact spectrometer based on a tapered spiral-shaped waveguide with a detector array integrated directly on top. The device relies on imaging the multimode interference from leaky modes, resulting in a resolution of 20 pm in the visible range and a bandwidth from 545 to 725 nm with a 250 µm radius structure. Spectra of multiple narrow lines and synthesized broadband are well reconstructed. The ability to achieve such high resolution and broad bandwidth in a compact footprint is expected to have a significant role in low-cost and multifunctional integrated systems.