EM-COGLOAD: An investigation into age and cognitive load detection using eye tracking and deep learning.

Alzheimer's Disease is the most prevalent neurodegenerative disease, and is a leading cause of disability among the elderly. Eye movement behaviour demonstrates potential as a non-invasive biomarker for Alzheimer's Disease, with changes detectable at an early stage after initial onset. This paper introduces a new publicly available dataset: EM-COGLOAD (available at https://osf.io/zjtdq/, DOI: 10.17605/OSF.IO/ZJTDQ). A dual-task paradigm was used to create effects of declined cognitive performance in 75 healthy adults as they carried out visual tracking tasks. Their eye movement was recorded, and time series classification of the extracted eye movement traces was explored using a range of deep learning techniques. The results of this showed that convolutional neural networks were able to achieve an accuracy of 87.5% when distinguishing between eye movement under low and high cognitive load, and 76% when distinguishing between the oldest and youngest age groups.

Potential pitfalls in analysing a SARS-CoV-2 RT-PCR assay and how to standardise data interpretation.

The emergence of SARS-CoV-2 in December 2019 lead to the rapid implementation of assays for virus detection, with real-time RT-PCR arguably considered the gold-standard. In our laboratory Altona RealStar SARS-Cov-2 RT-PCR kits are used with Applied Biosystems QuantStudio 7 Flex thermocyclers. Real-time PCR data interpretation is potentially complex and time-consuming, particularly for SARS-CoV-2, where the laboratory handles up to 2000 samples each day. To simplify this, an automated system that rapidly interprets the curves, developed by diagnostics.ai was introduced. QuantStudio software provides two methods for interpretation, relative threshold and baseline threshold. Many of our assays are analysed using relative threshold and directly exported into pcr.ai software, however, in some rare cases the QuantStudio software assigns positive results to 'ambiguous' curves, flagged by pcr.ai, requiring manual intervention. Due to the sample numbers processed and the proportionate increase in curves flagged by pcr.ai, the two methods were investigated. An audit was carried out to determine the frequency of these curves, involving 138 samples tested during November 2020, including 97 serial samples from 38 patients and it was determined that the relative threshold method produced unreliable results in many of these cases. In addition, we present a solution to simplify the interpretation and automate the process.

Diabetes mellitus prediction and diagnosis from a data preprocessing and machine learning perspective.

BACKGROUND AND OBJECTIVE: Diabetes mellitus is a metabolic disorder characterized by hyperglycemia, which results from the inadequacy of the body to secrete and respond to insulin. If not properly managed or diagnosed on time, diabetes can pose a risk to vital body organs such as the eyes, kidneys, nerves, heart, and blood vessels and so can be life-threatening. The many years of research in computational diagnosis of diabetes have pointed to machine learning to as a viable solution for the prediction of diabetes. However, the accuracy rate to date suggests that there is still much room for improvement. In this paper, we are proposing a machine learning framework for diabetes prediction and diagnosis using the PIMA Indian dataset and the laboratory of the Medical City Hospital (LMCH) diabetes dataset. We hypothesize that adopting feature selection and missing value imputation methods can scale up the performance of classification models in diabetes prediction and diagnosis. METHODS: In this paper, a robust framework for building a diabetes prediction model to aid in the clinical diagnosis of diabetes is proposed. The framework includes the adoption of Spearman correlation and polynomial regression for feature selection and missing value imputation, respectively, from a perspective that strengthens their performances. Further, different supervised machine learning models, the random forest (RF) model, support vector machine (SVM) model, and our designed twice-growth deep neural network (2GDNN) model are proposed for classification. The models are optimized by tuning the hyperparameters of the models using grid search and repeated stratified k-fold cross-validation and evaluated for their ability to scale to the prediction problem. RESULTS: Through experiments on the PIMA Indian and LMCH diabetes datasets, precision, sensitivity, F1-score, train-accuracy, and test-accuracy scores of 97.34%, 97.24%, 97.26%, 99.01%, 97.25 and 97.28%, 97.33%, 97.27%, 99.57%, 97.33, are achieved with the proposed 2GDNN model, respectively. CONCLUSION: The data preprocessing approaches and the classifiers with hyperparameter optimization proposed within the machine learning framework yield a robust machine learning model that outperforms state-of-the-art results in diabetes mellitus prediction and diagnosis. The source code for the models of the proposed machine learning framework has been made publicly available.

Towards Machine Vision for Insect Welfare Monitoring and Behavioural Insights.

Machine vision has demonstrated its usefulness in the livestock industry in terms of improving welfare in such areas as lameness detection and body condition scoring in dairy cattle. In this article, we present some promising results of applying state of the art object detection and classification techniques to insects, specifically Black Soldier Fly (BSF) and the domestic cricket, with the view of enabling automated processing for insect farming. We also present the low-cost "Insecto" Internet of Things (IoT) device, which provides environmental condition monitoring for temperature, humidity, CO2, air pressure, and volatile organic compound levels together with high resolution image capture. We show that we are able to accurately count and measure size of BSF larvae and also classify the sex of domestic crickets by detecting the presence of the ovipositor. These early results point to future work for enabling automation in the selection of desirable phenotypes for subsequent generations and for providing early alerts should environmental conditions deviate from desired values.

Investigating the sequence variation in the influenza A matrix genes during the 2017-2018 and 2018-2019 seasons in samples from a local population in London.

Recent publications have highlighted the emergence of mutations in the M1 gene of both influenza A H1N1pdm09 and H3N2 subtypes affecting the performance of commercial RT-PCR assays. Respiratory samples from the 2018/2019 season positive by our in-house RT-PCR for influenza A were analysed for the prevalence and impact of any M1 gene mutations. Sequence information was used to re-design primers for our routine assay and their performance assessed. Forty-five samples, consisting of 11 H1N1pdm09 and 34 H3N2 subtypes, together with the NIBSC H1N1 control were sequenced. All samples displayed the core mutations for H1N1 M1(C154T; G174A and G238A) and for H3N2 M1(C153T; C163T and G189T); three of the H1N1pdm09 viruses also showed a small number of point mutations. None of the mutations appeared to affect either the sensitivity or efficiency of the RT-PCR when compared to the re-designed primers. Although the mutations we found agreed with those in the publications cited we did not encounter any problems with our routine diagnostic assay and no improvements were found when the primers were modified to suit those mutations. However, it is likely that the influenza A virus M1 gene will accumulate further mutations that could impact RT-PCR assays and, therefore, it would be prudent to implement routine sequencing of samples during the influenza seasons to ensure no loss in assay performance.

A computer vision approach to improving cattle digestive health by the monitoring of faecal samples.

The digestive health of cows is one of the primary factors that determine their well-being and productivity. Under- and over-feeding are both commonplace in the beef and dairy industry; leading to welfare issues, negative environmental impacts, and economic losses. Unfortunately, digestive health is difficult for farmers to routinely monitor in large farms due to many factors including the need to transport faecal samples to a laboratory for compositional analysis. This paper describes a novel means for monitoring digestive health via a low-cost and easy to use imaging device based on computer vision. The method involves the rapid capture of multiple visible and near-infrared images of faecal samples. A novel three-dimensional analysis algorithm is then applied to objectively score the condition of the sample based on its geometrical features. While there is no universal ground truth for comparison of results, the order of scores matched a qualitative human prediction very closely. The algorithm is also able to detect the presence of undigested fibres and corn kernels using a deep learning approach. Detection rates for corn and fibre in image regions were of the order 90%. These results indicate the potential to develop this system for on-farm, real time monitoring of the digestive health of individual animals, allowing early intervention to effectively adjust feeding strategy.

Using next-generation sequencing of microRNAs to identify host and/or pathogen nucleic acid signatures in samples from children with biliary atresia - a pilot study.

Biliary atresia (BA) is a progressive disease affecting infants resulting in inflammatory obliteration and fibrosis of the extra- and intra-hepatic biliary tree. BA may be grouped into type 1 isolated; type 2 syndromic, where other congenital malformations may be present; type 3 cystic BA, where there is cyst formation within an otherwise obliterated biliary tree; and cytomegalovirus-associated BA. The cause of BA is unclear, with immune dysregulation, inflammation and infection, particularly with cytomegalovirus (CMV), all implicated. In this study a total of 50/67 samples were tested for CMV DNA using quantitative real-time PCR. Ten liver tissue and 8 bile samples from 10 patients representing the range of BA types were also analysed by next-generation sequencing. CMV DNA was found in 8/50 (16 %) patients and a total of 265 differentially expressed microRNAs were identified. No statistically significant differences between the various types of BA were found. However, differences were identified in the expression patterns of 110 microRNAs in bile and liver tissue samples (P<0.05). A small number of bacterial and viral sequences were found, although their relevance to BA remains to be determined. No direct evidence of viral causes of BA were found, although clear evidence of microRNAs associated with hepatocyte and cholangiocyte differentiation together with fibrosis and inflammation were identified. These include miR-30 and the miR-23 cluster (liver and bile duct development) and miR-29, miR-483, miR-181, miR-199 and miR-200 (inflammation and fibrosis).

Optical imaging technology in colonoscopy: Is there a role for photometric stereo?

Colonoscopy screening for the detection and removal of colonic adenomas is central to efforts to reduce the morbidity and mortality of colorectal cancer. However, up to a third of adenomas may be missed at colonoscopy, and the majority of post-colonoscopy colorectal cancers are thought to arise from these. Adenomas have three-dimensional surface topographic features that differentiate them from adjacent normal mucosa. However, these topographic features are not enhanced by white light colonoscopy, and the endoscopist must infer these from two-dimensional cues. This may contribute to the number of missed lesions. A variety of optical imaging technologies have been developed commercially to enhance surface topography. However, existing techniques enhance surface topography indirectly, and in two dimensions, and the evidence does not wholly support their use in routine clinical practice. In this narrative review, co-authored by gastroenterologists and engineers, we summarise the evidence for the impact of established optical imaging technologies on adenoma detection rate, and review the development of photometric stereo (PS) for colonoscopy. PS is a machine vision technique able to capture a dense array of surface normals to render three-dimensional reconstructions of surface topography. This imaging technique has several potential clinical applications in colonoscopy, including adenoma detection, polyp classification, and facilitating polypectomy, an inherently three-dimensional task. However, the development of PS for colonoscopy is at an early stage. We consider the progress that has been made with PS to date and identify the obstacles that need to be overcome prior to clinical application.

A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth.

BACKGROUND: Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared with manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS). RESULTS: We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1,768 images in total). A full protocol is provided, including all software components and an additional test data set. CONCLUSIONS: PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small- and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.

Multispectral imaging for presymptomatic analysis of light leaf spot in oilseed rape.

BACKGROUND: The use of spectral imaging within the plant phenotyping and breeding community has been increasing due its utility as a non-invasive diagnostic tool. However, there is a lack of imaging systems targeted specifically at plant science duties, resulting in low precision for canopy-scale measurements. This study trials a prototype multispectral system designed specifically for plant studies and looks at its use as an early detection system for visually asymptomatic disease phases, in this case Pyrenopeziza brassicae in Brassica napus. The analysis takes advantage of machine learning in the form of feature selection and novelty detection to facilitate the classification. An initial study into recording the morphology of the samples is also included to allow for further improvement to the system performance. RESULTS: The proposed method was able to detect light leaf spot infection with 92% accuracy when imaging entire oilseed rape plants from above, 12 days after inoculation and 13 days before the appearance of visible symptoms. False colour mapping of spectral vegetation indices was used to quantify disease severity and its distribution within the plant canopy. In addition, the structure of the plant was recorded using photometric stereo, with the output influencing regions used for diagnosis. The shape of the plants was also recorded using photometric stereo, which allowed for reconstruction of the leaf angle and surface texture, although further work is needed to improve the fidelity due to uneven lighting distributions, to allow for reflectance compensation. CONCLUSIONS: The ability of active multispectral imaging has been demonstrated along with the improvement in time taken to detect light leaf spot at a high accuracy. The importance of capturing structural information is outlined, with its effect on reflectance and thus classification illustrated. The system could be used in plant breeding to enhance the selection of resistant cultivars, with its early and quantitative capability.

Innovative 3D and 2D machine vision methods for analysis of plants and crops in the field.

Machine vision systems offer great potential for automating crop control, harvesting, fruit picking, and a range of other agricultural tasks. However, most of the reported research on machine vision in agriculture involves a 2D approach, where the utility of the resulting data is often limited by effects such as parallax, perspective, occlusion and changes in background light - particularly when operating in the field. The 3D approach to plant and crop analysis described in this paper offers potential to obviate many of these difficulties by utilising the richer information that 3D data can generate. The methodologies presented, such as four-light photometric stereo, also provide advanced functionalities, such as an ability to robustly recover 3D surface texture from plants at very high resolution. This offers potential for enabling, for example, reliable detection of the meristem (the part of the plant where growth can take place), to within a few mm, for directed weeding (with all the associated cost and ecological benefits) as well as offering new capabilities for plant phenotyping. The considerable challenges associated with robust and reliable utilisation of machine vision in the field are also considered and practical solutions are described. Two projects are used to illustrate the proposed approaches: a four-light photometric stereo apparatus able to recover plant textures at high-resolution (even in direct sunlight), and a 3D system able to measure potato sizes in-the-field to an accuracy of within 10%, for extended periods and in a range of environmental conditions. The potential benefits of the proposed 3D methods are discussed, both in terms of the advanced capabilities attainable and the widespread potential uptake facilitated by their low cost.

Photometric stereo for three-dimensional leaf venation extraction.

Leaf venation extraction studies have been strongly discouraged by considerable challenges posed by venation architectures that are complex, diverse and subtle. Additionally, unpredictable local leaf curvatures, undesirable ambient illuminations, and abnormal conditions of leaves may coexist with other complications. While leaf venation extraction has high potential for assisting with plant phenotyping, speciation and modelling, its investigations to date have been confined to colour image acquisition and processing which are commonly confounded by the aforementioned biotic and abiotic variations. To bridge the gaps in this area, we have designed a 3D imaging system for leaf venation extraction, which can overcome dark or bright ambient illumination and can allow for 3D data reconstruction in high resolution. We further propose a novel leaf venation extraction algorithm that can obtain illumination-independent surface normal features by performing Photometric Stereo reconstruction as well as local shape measures by fusing the decoupled shape index and curvedness features. In addition, this algorithm can determine venation polarity - whether veins are raised above or recessed into a leaf. Tests on both sides of different leaf species with varied venation architectures show that the proposed method is accurate in extracting the primary, secondary and even tertiary veins. It also proves to be robust against leaf diseases which can cause dramatic changes in colour. The effectiveness of this algorithm in determining venation polarity is verified by it correctly recognising raised or recessed veins in nine different experiments.

Functional and genetic predisposition to rhinovirus lower respiratory tract infections in prematurely born infants.

Term born infants are predisposed to human rhinovirus (HRV) lower respiratory tract infections (LRTI) by reduced neonatal lung function and genetic susceptibility. Our aim was to investigate whether prematurely born infants were similarly predisposed to HRV LRTIs or any other viral LRTIs. Infants born less than 36 weeks of gestational age were recruited. Prior to neonatal/maternity unit discharge, lung function (functional residual capacity by helium gas dilution and multiple breath washout, lung clearance index and compliance (Crs), and resistance (Rrs) of the respiratory system) was assessed and DNA samples assessed for eight single nucleotide polymorphisms (SNPs) in seven genes: ADAM33, IL10, MMP16 NFκB1A,SFTPC, VDR, and NOS2A. Infants were prospectively followed until 1 year corrected age. Nasopharyngeal aspirates (NPAs) were sent whenever an infant developed a LRTI and tested for 13 viruses. One hundred and thirty-nine infants were included in the analysis. Infants who developed HRV LRTIs had reduced Crs (1.6 versus 1.2 mL/cmH2O/kg, p = 0.044) at 36 weeks postmenstrual age. A SNP in the gene coding for the vitamin D receptor was associated with the development of HRV LRTIs and any viral LRTIs (p = 0.02). CONCLUSION: Prematurely born infants may have both a functional and genetic predisposition to HRV LRTIs. What is Known: • Term born infants are predisposed to rhinovirus lower respiratory tract (HRV LRTIs) infection by reduced neonatal lung function. • Term born infants requiring hospitalisation due to HRV bronchiolitis were more likely to have single nucleotide polymorphism (SNP) in the IL-10 gene. What is New: • Prematurely born infants who developed a HRV LRTI had lower C rs before maternity unit discharge. • A SNP in the gene coding for the vitamin D receptor was associated with the development of HRV LRTIs and overall respiratory viral LRTIs in prematurely born infants.

Analysis of Epstein-Barr virus and cellular gene expression during the early phases of Epstein-Barr virus lytic induction.

In order to develop novel host/pathogen real-time PCR assays for routine diagnostic use, early gene expression patterns from both Epstein-Barr virus (EBV) and Raji cells were examined after inducing the lytic life cycle using 12-O-tetradecanoyl-13-phorbol ester and sodium butyrate. Real-time PCR identified several highly induced (>90-fold) EBV lytic genes over a 48 h time course during the lytic induction phase. Latent genes were induced at low levels during this phase. The cellular response to lytic viral replication is poorly understood. Whole human genome microarray analysis identified 113 cellular genes regulated twofold or more by EBV, including 63 upregulated and 46 downregulated genes, over a 24 h time course post-induction. The most upregulated gene was CHI3L1, a chitinase-3-like 1 protein (18.1-fold; P<0.0084), and the most downregulated gene was TYMS, a thymidylate synthetase (-7.6-fold). Gene Ontology enrichment analysis using MetaCore software revealed cell cycle (core), cell cycle (role of anaphase-promoting complex) in cell cycle regulation) and lymphatic diseases as the most significantly represented biological network processes, canonical pathways and disease biomarkers, respectively. Chemotaxis, DNA damage and inflammation (IL-4 signalling) together with lymphoproliferative disorders and non-Hodgkin's lymphoma were significantly represented biological processes and disease biomarkers.

Baseline cytokine profiling identifies novel risk factors for invasive fungal disease among haematology patients undergoing intensive chemotherapy or haematopoietic stem cell transplantation.

BACKGROUND: Invasive fungal disease (IFD) is a disease of immunocompromised hosts. Cytokines are important mediators of innate and adaptive immune system. The aim of this study was to identify cytokine profiles that correlate with increased risk of IFD. METHODS: We prospectively enrolled 172 adult haematology patients undergoing intensive chemotherapy, immunosuppressive therapy, and haematopoietic stem cell transplantation. Pro-inflammatory cytokine profiling using 30-plex Luminex assay was performed at baseline and during treatment. Nine single nucleotide polymorphisms (TLR1, TLR2, TLR3, TLR4.1, TLR4.2, TLR6, CLEC7A, CARD9, and INFG) were investigated among transplant recipients and donors. FINDINGS: The incidence of IFD in this cohort was 16.9% (29/172). Median baseline serum concentrations of IL-15, IL-2R, CCL2, and MIP-1α were significantly higher whilst IL-4 was lower in patients with proven/probable IFD compared to those with no evidence of IFD. Baseline high IL-2R and CCL2 were associated with increased risk of IFD in the multivariate analysis (adjusted hazard ratio 2.3 [95% CI 1.1-5.1; P = 0.037], and hazard ratio 2.7 [95% CI 1.2-6.1; P = 0.016], respectively). However, these differences were not significant in follow up measurements. Similarly, no significant independent prognostic value was associated with baseline cytokine profile. INTERPRETATION: High baseline IL-2R and CCL2 concentrations were independent indicators of the risk of developing IFD and could be used to identify patients for enhanced prophylaxis and early antifungal therapy.

Triazole antifungals used for prophylaxis and treatment of invasive fungal disease in adult haematology patients: Trough serum concentrations in relation to outcome.

Triazole antifungal drugs are widely used for the prophylaxis and treatment of invasive fungal disease (IFD). Efficacy may depend on attaining minimum effective plasma concentrations. The aim of this study was to ascertain the proportion of samples in which the recommended concentrations were achieved in patients given these drugs in relation to outcome. In-patients prescribed standard doses of fluconazole, itraconazole solution, posaconazole suspension, or oral voriconazole for at least one week were studied. Pre-dose serum triazole concentrations were measured using validated methods. There were 359 samples from 90 patients. The median (range) number of samples per patient was 3 (1-13), and the median (range) fluconazole, itraconazole, posaconazole (prophylaxis), posaconazole (treatment), and voriconazole serum concentrations were 5.64 (0.11-18), 0.57 (0-5.3), 0.31 (0.02-2.5), 0.65 (0.02-2.5), and 0.95 (0.10-5.4) mg/l, respectively. The number of samples in which the recommended pre-dose concentrations were achieved was 98 (54%), 9 (20%), 2 (18%), and 29 (49%) for itraconazole, posaconazole (>0.7 mg/l prophylaxis), posaconazole (treatment), and voriconazole, respectively. No significant differences were detected in the median triazole trough concentrations between patients with proven/probable IFD compared to those with no evidence of IFD. However, itraconazole was not detected in 10 samples (7 patients). The small number of patients who achieved the recommended trough posaconazole concentrations may explain the high rate of break-through IFD observed in patients prescribed this drug. Except for fluconazole, the number of patients prescribed standard doses of triazoles who achieved recommended trough triazole concentrations was low. The prospective use of serum triazole measurements assay may have improved outcomes with itraconazole, posaconazole, and with voriconazole.

Eye center localization and gaze gesture recognition for human-computer interaction.

This paper introduces an unsupervised modular approach for accurate and real-time eye center localization in images and videos, thus allowing a coarse-to-fine, global-to-regional scheme. The trajectories of eye centers in consecutive frames, i.e., gaze gestures, are further analyzed, recognized, and employed to boost the human-computer interaction (HCI) experience. This modular approach makes use of isophote and gradient features to estimate the eye center locations. A selective oriented gradient filter has been specifically designed to remove strong gradients from eyebrows, eye corners, and shadows, which sabotage most eye center localization methods. A real-world implementation utilizing these algorithms has been designed in the form of an interactive advertising billboard to demonstrate the effectiveness of our method for HCI. The eye center localization algorithm has been compared with 10 other algorithms on the BioID database and six other algorithms on the GI4E database. It outperforms all the other algorithms in comparison in terms of localization accuracy. Further tests on the extended Yale Face Database b and self-collected data have proved this algorithm to be robust against moderate head poses and poor illumination conditions. The interactive advertising billboard has manifested outstanding usability and effectiveness in our tests and shows great potential for benefiting a wide range of real-world HCI applications.

Gender recognition from facial images: two or three dimensions?

This paper seeks to compare encoded features from both two-dimensional (2D) and three-dimensional (3D) face images in order to achieve automatic gender recognition with high accuracy and robustness. The Fisher vector encoding method is employed to produce 2D, 3D, and fused features with escalated discriminative power. For 3D face analysis, a two-source photometric stereo (PS) method is introduced that enables 3D surface reconstructions with accurate details as well as desirable efficiency. Moreover, a 2D+3D imaging device, taking the two-source PS method as its core, has been developed that can simultaneously gather color images for 2D evaluations and PS images for 3D analysis. This system inherits the superior reconstruction accuracy from the standard (three or more light) PS method but simplifies the reconstruction algorithm as well as the hardware design by only requiring two light sources. It also offers great potential for facilitating human computer interaction by being accurate, cheap, efficient, and nonintrusive. Ten types of low-level 2D and 3D features have been experimented with and encoded for Fisher vector gender recognition. Evaluations of the Fisher vector encoding method have been performed on the FERET database, Color FERET database, LFW database, and FRGCv2 database, yielding 97.7%, 98.0%, 92.5%, and 96.7% accuracy, respectively. In addition, the comparison of 2D and 3D features has been drawn from a self-collected dataset, which is constructed with the aid of the 2D+3D imaging device in a series of data capture experiments. With a variety of experiments and evaluations, it can be proved that the Fisher vector encoding method outperforms most state-of-the-art gender recognition methods. It has also been observed that 3D features reconstructed by the two-source PS method are able to further boost the Fisher vector gender recognition performance, i.e., up to a 6% increase on the self-collected database.

Combination of 3D skin surface texture features and 2D ABCD features for improved melanoma diagnosis.

Two-dimensional asymmetry, border irregularity, colour variegation and diameter (ABCD) features are important indicators currently used for computer-assisted diagnosis of malignant melanoma (MM); however, they often prove to be insufficient to make a convincing diagnosis. Previous work has demonstrated that 3D skin surface normal features in the form of tilt and slant pattern disruptions are promising new features independent from the existing 2D ABCD features. This work investigates that whether improved lesion classification can be achieved by combining the 3D features with the 2D ABCD features. Experiments using a nonlinear support vector machine classifier show that many combinations of the 2D ABCD features and the 3D features can give substantially better classification accuracy than using (1) single features and (2) many combinations of the 2D ABCD features. The best 2D and 3D feature combination includes the overall 3D skin surface disruption, the asymmetry and all the three colour channel features. It gives an overall 87.8 % successful classification, which is better than the best single feature with 78.0 % and the best 2D feature combination with 83.1 %. These demonstrate that (1) the 3D features have additive values to improve the existing lesion classification and (2) combining the 3D feature with all the 2D features does not lead to the best lesion classification. The two ABCD features not selected by the best 2D and 3D combination, namely (1) the border feature and (2) the diameter feature, were also studied in separate experiments. It found that inclusion of either feature in the 2D and 3D combination can successfully classify 3 out of 4 lesion groups. The only one group not accurately classified by either feature can be classified satisfactorily by the other. In both cases, they have shown better classification performances than those without the 3D feature in the combinations. This further demonstrates that (1) the 3D feature can be used to improve the existing 2D-based diagnosis and (2) including the 3D feature with subsets of the 2D features can be used in distinguishing different benign lesion classes from MM. It is envisaged that classification performance may be further improved if different 2D and 3D feature subsets demonstrated in this study are used in different stages to target different benign lesion classes in future studies.