Improving Joint Learning of Chest X-Ray and Radiology Report by Word Region Alignment.

Self-supervised learning provides an opportunity to explore unlabeled chest X-rays and their associated free-text reports accumulated in clinical routine without manual supervision. This paper proposes a Joint Image Text Representation Learning Network (JoImTeRNet) for pre-training on chest X-ray images and their radiology reports. The model was pre-trained on both the global image-sentence level and the local image region-word level for visual-textual matching. Both are bidirectionally constrained on Cross-Entropy based and ranking-based Triplet Matching Losses. The region-word matching is calculated using the attention mechanism without direct supervision about their mapping. The pre-trained multi-modal representation learning paves the way for downstream tasks concerning image and/or text encoding. We demonstrate the representation learning quality by cross-modality retrievals and multi-label classifications on two datasets: OpenI-IU and MIMIC-CXR. Our code is available at https://github.com/mshaikh2/JoImTeR_MLMI_2021.

Development of Individuality in Children's Handwriting.

Handwriting of children in early grades is studied from the viewpoint of quantitatively measuring the development of handwriting individuality. Handwriting samples of children, in grades 2-4, writing a paragraph of text in both handprinted and cursive, collected over a period of 3 years, were analyzed using two different approaches: (i) characteristics of the word "and" and (ii) entire paragraphs using an automated system. In the first approach, word characteristics were analyzed using statistical measures. In the second approach, pairs of paragraphs were compared. Both types of analysis, single word and complete writing, led to the same conclusions: (i) handwriting of each child remains relatively similar when handwriting has been just learnt and becomes markedly different from grades 3 to 4 and (ii) handwriting of different children becomes progressively more different from grades 2 to 4. The results provide strong support that handwriting becomes more individualistic with child development.

Fast k-nearest neighbor classification using cluster-based trees.

Most fast k-nearest neighbor (k-NN) algorithms exploit metric properties of distance measures for reducing computation cost and a few can work effectively on both metric and nonmetric measures. We propose a cluster-based tree algorithm to accelerate k-NN classification without any presuppositions about the metric form and properties of a dissimilarity measure. A mechanism of early decision making and minimal side-operations for choosing searching paths largely contribute to the efficiency of the algorithm. The algorithm is evaluated through extensive experiments over standard NIST and MNIST databases.

Individuality of handwriting.

Motivated by several rulings in United States courts concerning expert testimony in general, and handwriting testimony in particular, we undertook a study to objectively validate the hypothesis that handwriting is individual. Handwriting samples of 1,500 individuals, representative of the U.S. population with respect to gender, age, ethnic groups, etc., were obtained. Analyzing differences in handwriting was done by using computer algorithms for extracting features from scanned images of handwriting. Attributes characteristic of the handwriting were obtained, e.g., line separation, slant, character shapes, etc. These attributes, which are a subset of attributes used by forensic document examiners (FDEs), were used to quantitatively establish individuality by using machine learning approaches. Using global attributes of handwriting and very few characters in the writing, the ability to determine the writer with a high degree of confidence was established. The work is a step towards providing scientific support for admitting handwriting evidence in court. The mathematical approach and the resulting software also have the promise of aiding the FDE.