Historical Text Line Segmentation Using Deep Learning Algorithms: Mask-RCNN against U-Net Networks.

Text line segmentation is a necessary preliminary step before most text transcription algorithms are applied. The leading deep learning networks used in this context (ARU-Net, dhSegment, and Doc-UFCN) are based on the U-Net architecture. They are efficient, but fall under the same concept, requiring a post-processing step to perform instance (e.g., text line) segmentation. In the present work, we test the advantages of Mask-RCNN, which is designed to perform instance segmentation directly. This work is the first to directly compare Mask-RCNN- and U-Net-based networks on text segmentation of historical documents, showing the superiority of the former over the latter. Three studies were conducted, one comparing these networks on different historical databases, another comparing Mask-RCNN with Doc-UFCN on a private historical database, and a third comparing the handwritten text recognition (HTR) performance of the tested networks. The results showed that Mask-RCNN outperformed ARU-Net, dhSegment, and Doc-UFCN using relevant line segmentation metrics, that performance evaluation should not focus on the raw masks generated by the networks, that a light mask processing is an efficient and simple solution to improve evaluation, and that Mask-RCNN leads to better HTR performance.

Recognition of intraglomerular histological features with deep learning in protocol transplant biopsies and their association with kidney function and prognosis.

Background: The Banff Classification may not adequately address protocol transplant biopsies categorized as normal in patients experiencing unexplained graft function deterioration. This study seeks to employ convolutional neural networks to automate the segmentation of glomerular cells and capillaries and assess their correlation with transplant function. Methods: A total of 215 patients were categorized into three groups. In the Training cohort, glomerular cells and capillaries from 37 patients were manually annotated to train the networks. The Test cohort (24 patients) compared manual annotations vs automated predictions, while the Application cohort (154 protocol transplant biopsies) examined predicted factors in relation to kidney function and prognosis. Results: In the Test cohort, the networks recognized histological structures with Precision, Recall, F-score and Intersection Over Union exceeding 0.92, 0.85, 0.89 and 0.74, respectively. Univariate analysis revealed associations between the estimated glomerular filtration rate (eGFR) at biopsy and relative endothelial area (r = 0.19, P = .027), endothelial cell density (r = 0.20, P = .017), mean parietal epithelial cell area (r = -0.38, P < .001), parietal epithelial cell density (r = 0.29, P < .001) and mesangial cell density (r = 0.22, P = .010). Multivariate analysis retained only endothelial cell density as associated with eGFR (Beta = 0.13, P = .040). Endothelial cell density (r = -0.22, P = .010) and mean podocyte area (r = 0.21, P = .016) were linked to proteinuria at biopsy. Over 44 ± 29 months, 25 patients (16%) reached the primary composite endpoint (dialysis initiation, or 30% eGFR sustained decline), with relative endothelial area, mean endothelial cell area and parietal epithelial cell density below medians linked to this endpoint [hazard ratios, respectively, of 2.63 (P = .048), 2.60 (P = .039) and 3.23 (P = .019)]. Conclusion: This study automated the measurement of intraglomerular cells and capillaries. Our results suggest that the precise segmentation of endothelial and epithelial cells may serve as a potential future marker for the risk of graft loss.

Automated evaluation with deep learning of total interstitial inflammation and peritubular capillaritis on kidney biopsies.

BACKGROUND: Interstitial inflammation and peritubular capillaritis are observed in many diseases on native and transplant kidney biopsies. A precise and automated evaluation of these histological criteria could help stratify patients' kidney prognoses and facilitate therapeutic management. METHODS: We used a convolutional neural network to evaluate those criteria on kidney biopsies. A total of 423 kidney samples from various diseases were included; 83 kidney samples were used for the neural network training, 106 for comparing manual annotations on limited areas to automated predictions, and 234 to compare automated and visual gradings. RESULTS: The precision, recall and F-score for leukocyte detection were, respectively, 81%, 71% and 76%. Regarding peritubular capillaries detection the precision, recall and F-score were, respectively, 82%, 83% and 82%. There was a strong correlation between the predicted and observed grading of total inflammation, as for the grading of capillaritis (r = 0.89 and r = 0.82, respectively, all P < .0001). The areas under the receiver operating characteristics curves for the prediction of pathologists' Banff total inflammation (ti) and peritubular capillaritis (ptc) scores were respectively all above 0.94 and 0.86. The kappa coefficients between the visual and the neural networks' scores were respectively 0.74, 0.78 and 0.68 for ti ≥1, ti ≥2 and ti ≥3, and 0.62, 0.64 and 0.79 for ptc ≥1, ptc ≥2 and ptc ≥3. In a subgroup of patients with immunoglobulin A nephropathy, the inflammation severity was highly correlated to kidney function at biopsy on univariate and multivariate analyses. CONCLUSION: We developed a tool using deep learning that scores the total inflammation and capillaritis, demonstrating the potential of artificial intelligence in kidney pathology.

Deep learning automation of MEST-C classification in IgA nephropathy.

BACKGROUND: Although the MEST-C classification is among the best prognostic tools in immunoglobulin A nephropathy (IgAN), it has a wide interobserver variability between specialized pathologists and others. Therefore we trained and evaluated a tool using a neural network to automate the MEST-C grading. METHODS: Biopsies of patients with IgAN were divided into three independent groups: the Training cohort (n = 42) to train the network, the Test cohort (n = 66) to compare its pixel segmentation to that made by pathologists and the Application cohort (n = 88) to compare the MEST-C scores computed by the network or by pathologists. RESULTS: In the Test cohort, >73% of pixels were correctly identified by the network as M, E, S or C. In the Application cohort, the neural network area under the receiver operating characteristics curves were 0.88, 0.91, 0.88, 0.94, 0.96, 0.96 and 0.92 to predict M1, E1, S1, T1, T2, C1 and C2, respectively. The kappa coefficients between pathologists and the network assessments were substantial for E, S, T and C scores (kappa scores of 0.68, 0.79, 0.73 and 0.70, respectively) and moderate for M score (kappa score of 0.52). Network S and T scores were associated with the occurrence of the composite survival endpoint (death, dialysis, transplantation or doubling of serum creatinine) [hazard ratios 9.67 (P = .006) and 7.67 (P < .001), respectively]. CONCLUSIONS: This work highlights the possibility of automated recognition and quantification of each element of the MEST-C classification using deep learning methods.

A Cross-Sectional Reproducibility Study of a Standard Camera Sensor Using Artificial Intelligence to Assess Food Items: The FoodIntech Project.

Having a system to measure food consumption is important to establish whether individual nutritional needs are being met in order to act quickly and to minimize the risk of undernutrition. Here, we tested a smartphone-based food consumption assessment system named FoodIntech. FoodIntech, which is based on AI using deep neural networks (DNN), automatically recognizes food items and dishes and calculates food leftovers using an image-based approach, i.e., it does not require human intervention to assess food consumption. This method uses one-input and one-output images by means of the detection and synchronization of a QRcode located on the meal tray. The DNN are then used to process the images and implement food detection, segmentation and recognition. Overall, 22,544 situations analyzed from 149 dishes were used to test the reliability of this method. The reliability of the AI results, based on the central intra-class correlation coefficient values, appeared to be excellent for 39% of the dishes (n = 58 dishes) and good for 19% (n = 28). The implementation of this method is an effective way to improve the recognition of dishes and it is possible, with a sufficient number of photos, to extend the capabilities of the tool to new dishes and foods.

Automatic Evaluation of Histological Prognostic Factors Using Two Consecutive Convolutional Neural Networks on Kidney Samples.

BACKGROUND AND OBJECTIVES: The prognosis of patients undergoing kidney tumor resection or kidney donation is linked to many histologic criteria. These criteria notably include glomerular density, glomerular volume, vascular luminal stenosis, and severity of interstitial fibrosis/tubular atrophy. Automated measurements through a deep-learning approach could save time and provide more precise data. This work aimed to develop a free tool to automatically obtain kidney histologic prognostic features. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: In total, 241 samples of healthy kidney tissue were split into three independent cohorts. The "Training" cohort (n=65) was used to train two convolutional neural networks: one to detect the cortex and a second to segment the kidney structures. The "Test" cohort (n=50) assessed their performance by comparing manually outlined regions of interest to predicted ones. The "Application" cohort (n=126) compared prognostic histologic data obtained manually or through the algorithm on the basis of the combination of the two convolutional neural networks. RESULTS: In the Test cohort, the networks isolated the cortex and segmented the elements of interest with good performances (>90% of the cortex, healthy tubules, glomeruli, and even globally sclerotic glomeruli were detected). In the Application cohort, the expected and predicted prognostic data were significantly correlated. The correlation coefficients r were 0.85 for glomerular volume, 0.51 for glomerular density, 0.75 for interstitial fibrosis, 0.71 for tubular atrophy, and 0.73 for vascular intimal thickness, respectively. The algorithm had a good ability to predict significant (>25%) tubular atrophy and interstitial fibrosis level (receiver operator characteristic curve with an area under the curve, 0.92 and 0.91, respectively) or a significant vascular luminal stenosis (>50%) (area under the curve, 0.85). CONCLUSION: This freely available tool enables the automated segmentation of kidney tissue to obtain prognostic histologic data in a fast, objective, reliable, and reproducible way.

Event-Based Trajectory Prediction Using Spiking Neural Networks.

In recent years, event-based sensors have been combined with spiking neural networks (SNNs) to create a new generation of bio-inspired artificial vision systems. These systems can process spatio-temporal data in real time, and are highly energy efficient. In this study, we used a new hybrid event-based camera in conjunction with a multi-layer spiking neural network trained with a spike-timing-dependent plasticity learning rule. We showed that neurons learn from repeated and correlated spatio-temporal patterns in an unsupervised way and become selective to motion features, such as direction and speed. This motion selectivity can then be used to predict ball trajectory by adding a simple read-out layer composed of polynomial regressions, and trained in a supervised manner. Hence, we show that a SNN receiving inputs from an event-based sensor can extract relevant spatio-temporal patterns to process and predict ball trajectories.

High speed stereovision setup for position and motion estimation of fertilizer particles leaving a centrifugal spreader.

A 3D imaging technique using a high speed binocular stereovision system was developed in combination with corresponding image processing algorithms for accurate determination of the parameters of particles leaving the spinning disks of centrifugal fertilizer spreaders. Validation of the stereo-matching algorithm using a virtual 3D stereovision simulator indicated an error of less than 2 pixels for 90% of the particles. The setup was validated using the cylindrical spread pattern of an experimental spreader. A 2D correlation coefficient of 90% and a Relative Error of 27% was found between the experimental results and the (simulated) spread pattern obtained with the developed setup. In combination with a ballistic flight model, the developed image acquisition and processing algorithms can enable fast determination and evaluation of the spread pattern which can be used as a tool for spreader design and precise machine calibration.

Computational evidence that frequency trajectory theory does not oppose but emerges from age-of-acquisition theory.

According to the age-of-acquisition hypothesis, words acquired early in life are processed faster and more accurately than words acquired later. Connectionist models have begun to explore the influence of the age/order of acquisition of items (and also their frequency of encounter). This study attempts to reconcile two different methodological and theoretical approaches (proposed by Lambon Ralph & Ehsan, 2006 and Zevin & Seidenberg, 2002) to age-limited learning effects. The current simulations extend the findings reported by Zevin and Seidenberg (2002) that have shown that frequency trajectories (FTs) have limited and specific effects on word-reading tasks. Using the methodological framework proposed by Lambon Ralph and Ehsan (2006), which makes it possible to compare word-reading and picture-naming tasks in connectionist networks, we were able to show that FT has a considerable influence on age-limited learning effects in a picture naming task. The findings show that when the input-output mappings are arbitrary (simulating picture naming tasks), the links formed by the network become entrenched as a result of early experience and that subsequent variations in frequency of exposure of the items have only a minor impact. In contrast, when the mappings between input-output are quasi-systematic or systematic (simulating word-reading tasks), the training of new items was generalized and resulted in the suppression of age-limited learning effects. At a theoretical level, we suggest that FT, which simultaneously takes account of time and the level of exposure across time, represents a more precise and modulated measure compared with the order of introduction of the items and may lead to innovative hypotheses in the field of age-limited learning effects.