Design and Validation of an Augmented Reality Teaching System for Primary Logic Programming Education.

Programming is a skill that requires high levels of logical thinking and problem-solving abilities. According to the Curriculum Guidelines for the 12-Year Basic Education currently implemented in Taiwan, programming has been included in the mandatory courses of middle and high schools. Nevertheless, the guidelines simply recommend that elementary schools conduct fundamental instructions in related fields during alternative learning periods. This may result in the problem of a rough transition in programming learning for middle school freshmen. To alleviate this problem, this study proposes an augmented reality (AR) logic programming teaching system that combines AR technologies and game-based teaching material designs on the basis of the fundamental concepts for seventh-grade structured programming. This system can serve as an articulation curriculum for logic programming in primary education. Thus, students are able to develop basic programming logic concepts through AR technologies by performing simple command programming. This study conducted an experiment using the factor-based quasi-experimental research design and questionnaire survey method, with 42 fifth and sixth graders enrolled as the experimental subjects. The statistical analysis showed the following results: In terms of learning effectiveness, both AR-based and traditional learning groups displayed a significant performance. However, of the two groups, the former achieved more significant effectiveness in the posttest results. Regarding learning motivation, according to the evaluation results of the Attention, Relevance, Confidence, and Satisfaction (ARCS) motivation model, the AR-based learning group manifested significantly higher levels of learning motivation than the traditional learning group, with particularly significant differences observed in the dimension of Attention. Therefore, the experimental results validate that the proposed AR-based logic programming teaching system has significant positive effects on enhancing students' learning effectiveness and motivation.

Manipulation Planning for Object Re-Orientation Based on Semantic Segmentation Keypoint Detection.

In this paper, a manipulation planning method for object re-orientation based on semantic segmentation keypoint detection is proposed for robot manipulator which is able to detect and re-orientate the randomly placed objects to a specified position and pose. There are two main parts: (1) 3D keypoint detection system; and (2) manipulation planning system for object re-orientation. In the 3D keypoint detection system, an RGB-D camera is used to obtain the information of the environment and can generate 3D keypoints of the target object as inputs to represent its corresponding position and pose. This process simplifies the 3D model representation so that the manipulation planning for object re-orientation can be executed in a category-level manner by adding various training data of the object in the training phase. In addition, 3D suction points in both the object's current and expected poses are also generated as the inputs of the next operation stage. During the next stage, Mask Region-Convolutional Neural Network (Mask R-CNN) algorithm is used for preliminary object detection and object image. The highest confidence index image is selected as the input of the semantic segmentation system in order to classify each pixel in the picture for the corresponding pack unit of the object. In addition, after using a convolutional neural network for semantic segmentation, the Conditional Random Fields (CRFs) method is used to perform several iterations to obtain a more accurate result of object recognition. When the target object is segmented into the pack units of image process, the center position of each pack unit can be obtained. Then, a normal vector of each pack unit's center points is generated by the depth image information and pose of the object, which can be obtained by connecting the center points of each pack unit. In the manipulation planning system for object re-orientation, the pose of the object and the normal vector of each pack unit are first converted into the working coordinate system of the robot manipulator. Then, according to the current and expected pose of the object, the spherical linear interpolation (Slerp) algorithm is used to generate a series of movements in the workspace for object re-orientation on the robot manipulator. In addition, the pose of the object is adjusted on the z-axis of the object's geodetic coordinate system based on the image features on the surface of the object, so that the pose of the placed object can approach the desired pose. Finally, a robot manipulator and a vacuum suction cup made by the laboratory are used to verify that the proposed system can indeed complete the planned task of object re-orientation.

SurfNetv2: An Improved Real-Time SurfNet and Its Applications to Defect Recognition of Calcium Silicate Boards.

This paper presents an improved Convolutional Neural Network (CNN) architecture to recognize surface defects of the Calcium Silicate Board (CSB) using visual image information based on a deep learning approach. The proposed CNN architecture is inspired by the existing SurfNet architecture and is named SurfNetv2, which comprises a feature extraction module and a surface defect recognition module. The output of the system is the recognized defect category on the surface of the CSB. In the collection of the training dataset, we manually captured the defect images presented on the surface of the CSB samples. Then, we divided these defect images into four categories, which are crash, dirty, uneven, and normal. In the training stage, the proposed SurfNetv2 is trained through an end-to-end supervised learning method, so that the CNN model learns how to recognize surface defects of the CSB only through the RGB image information. Experimental results show that the proposed SurfNetv2 outperforms five state-of-the-art methods and achieves a high recognition accuracy of 99.90% and 99.75% in our private CSB dataset and the public Northeastern University (NEU) dataset, respectively. Moreover, the proposed SurfNetv2 model achieves a real-time computing speed of about 199.38 fps when processing images with a resolution of 128 × 128 pixels. Therefore, the proposed CNN model has great potential for real-time automatic surface defect recognition applications.

Image segmentation of activated sludge phase contrast images using phase stretch transform.

Activated sludge (AS) is a biological treatment process that is employed in wastewater treatment plants. Filamentous bacteria in AS plays an important role in the settling ability of the sludge. Proper settling of the sludge is essential for normal functionality of the wastewater plants, where filamentous bulking is always a persistent problem preventing sludge from settling. The performance of AS plants is conventionally monitored by physico-chemical procedures. An alternative way of monitoring the AS in wastewater treatment process is to use image processing and analysis. Good performance of the image segmentation algorithms is important to quantify flocs and filaments in AS. In this article, an algorithm is proposed to perform segmentation of filaments in the phase contrast images using phase stretch transform. Different values of strength (S) and warp (W) are tested to obtain optimum segmentation results and decrease the halo and shade-off artefacts encountered in phase contrast microscopy. The performance of the algorithm is assessed using DICE coefficient, accuracy, false positive rate (FPR), false negative rate (FNR) and Rand index (RI). Sixty-one gold approximations of ground truth images were manually prepared to assess the segmentation results. Thirty-two of them were acquired at 10× magnification and 29 of them were acquired at 20× magnification. The proposed algorithm exhibits better segmentation performance with an average DICE coefficient equal to 52.25%, accuracy 99.74%, FNR 41.8% and FPR 0.14% and RI 99.49%, based on 61 images.

Indoor Scene Point Cloud Registration Algorithm Based on RGB-D Camera Calibration.

With the increasing popularity of RGB-depth (RGB-D) sensor, research on the use of RGB-D sensors to reconstruct three-dimensional (3D) indoor scenes has gained more and more attention. In this paper, an automatic point cloud registration algorithm is proposed to efficiently handle the task of 3D indoor scene reconstruction using pan-tilt platforms on a fixed position. The proposed algorithm aims to align multiple point clouds using extrinsic parameters of the RGB-D camera obtained from every preset pan-tilt control point. A computationally efficient global registration method is proposed based on transformation matrices formed by the offline calibrated extrinsic parameters. Then, a local registration method, which is an optional operation in the proposed algorithm, is employed to refine the preliminary alignment result. Experimental results validate the quality and computational efficiency of the proposed point cloud alignment algorithm by comparing it with two state-of-the-art methods.

Heterogeneity-projection hard-decision color interpolation using spectral-spatial correlation.

This paper presents a novel heterogeneity-projection hard-decision (HPHD) color interpolation procedure for reproduction of Bayer mosaic images. The proposed algorithm aims to estimate the optimal interpolation direction and perform hard-decision interpolation, in which each pixel only needs to be interpolated once. A new heterogeneity-projection scheme based on a novel spectral-spatial correlation concept is proposed to estimate the best interpolation direction directly from the original mosaic image. Using the proposed heterogeneity-projection scheme, a hard-decision rule can be decided before performing the interpolation. The advantage of this scheme is that it provides an efficient way for decision-based algorithms to generate improved results using fewer computations. Compared with three recently reported demosaicing techniques, Gunturk's, Lu's, and Li's methods, the proposed HPHD outperforms all of them in both PSNR values and S-CIELAB deltaEab measures by utilizing 25 natural images from Kodak PhotoCD.