Unifying Fourteen Post-Hoc Attribution Methods With Taylor Interactions.

Various attribution methods have been developed to explain deep neural networks (DNNs) by inferring the attribution/importance/contribution score of each input variable to the final output. However, existing attribution methods are often built upon different heuristics. There remains a lack of a unified theoretical understanding of why these methods are effective and how they are related. Furthermore, there is still no universally accepted criterion to compare whether one attribution method is preferable over another. In this paper, we resort to Taylor interactions and for the first time, we discover that fourteen existing attribution methods, which define attributions based on fully different heuristics, actually share the same core mechanism. Specifically, we prove that attribution scores of input variables estimated by the fourteen attribution methods can all be mathematically reformulated as a weighted allocation of two typical types of effects, i.e., independent effects of each input variable and interaction effects between input variables. The essential difference among these attribution methods lies in the weights of allocating different effects. Inspired by these insights, we propose three principles for fairly allocating the effects, which serve as new criteria to evaluate the faithfulness of attribution methods. In summary, this study can be considered as a new unified perspective to revisit fourteen attribution methods, which theoretically clarifies essential similarities and differences among these methods. Besides, the proposed new principles enable people to make a direct and fair comparison among different methods under the unified perspective.

Differentiated Explanation of Deep Neural Networks With Skewed Distributions.

Over the last decade, deep neural networks (DNNs) are regarded as black-box methods, and their decisions are criticized for the lack of explainability. Existing attempts based on local explanations offer each input a visual saliency map, where the supporting features that contribute to the decision are emphasized with high relevance scores. In this paper, we improve the saliency map based on differentiated explanations, of which the saliency map not only distinguishes the supporting features from backgrounds but also shows the different degrees of importance of the various parts within the supporting features. To do this, we propose to learn a differentiated relevance estimator called DRE, where a carefully-designed distribution controller is introduced to guide the relevance scores towards right-skewed distributions. DRE can be directly optimized under pure classification losses, enabling higher faithfulness of explanations and avoiding non-trivial hyper-parameter tuning. The experimental results on three real-world datasets demonstrate that our differentiated explanations significantly improve the faithfulness with high explainability. Our code and trained models are available at https://github.com/fuweijie/DRE.

Subarchitecture Ensemble Pruning in Neural Architecture Search.

Neural architecture search (NAS) is gaining more and more attention in recent years because of its flexibility and remarkable capability to reduce the burden of neural network design. To achieve better performance, however, the searching process usually costs massive computations that might not be affordable for researchers and practitioners. Although recent attempts have employed ensemble learning methods to mitigate the enormous computational cost, however, they neglect a key property of ensemble methods, namely diversity, which leads to collecting more similar subarchitectures with potential redundancy in the final design. To tackle this problem, we propose a pruning method for NAS ensembles called " subarchitecture ensemble pruning in neural architecture search (SAEP)." It targets to leverage diversity and to achieve subensemble architectures at a smaller size with comparable performance to ensemble architectures that are not pruned. Three possible solutions are proposed to decide which subarchitectures to prune during the searching process. Experimental results exhibit the effectiveness of the proposed method by largely reducing the number of subarchitectures without degrading the performance.

Edible Bird's Nest Attenuates Menopause-Related Bone Degeneration in Rats via Increaing Bone Estrogen-Receptor Expression.

OBJECTIVE: To investigate the mechanistic basis for the attenuation of bone degeneration by edible bird's nest (EBN) in ovariectomized rats. METHODS: Forty-two female Sprage-Dawley rats were randomized into 7 groups (6 in each group). The ovariectomized (OVX) and OVX + 6%, 3%, and 1.5% EBN and OVX +estrogen groups were given standard rat chow alone, standard rat chow +6%, 3%, and 1.5% EBN, or standard rat chow +estrogen therapy (0.2mg/kg per day), respectively. The sham-operation group was surgically opened without removing the ovaries. The control group did not have any surgical intervention. After 12 weeks of intervention, blood samples were taken for serum estrogen, osteocalcin, and osteoprotegerin, as well as the measurement of magnesium, calcium abd zinc concentrations. While femurs were removed from the surrounding muscles to measure bone mass density using the X-ray edge detection technique, then collected for histology and estrogen receptor (ER) immunohistochemistry. RESULTS: Ovariectomy altered serum estrogen levels resulting in increased food intake and weight gain, while estrogen and EBN supplementation attenuated these changes. Ovariectomy also reduced bone ER expression and density, and the production of osteopcalcin and osteorotegerin, which are important pro-osteoplastic hormones that promote bone mineraliztion and density. Conversely, estrogen and EBN increased serum estrogen levels leading to increased bone ER expression, pro-osteoplastic hormone production and bone density (all P<0.05). CONCLUSION: EBN could be used as a safe alternative to hormone replacement therapys for managing menopausal complications like bone degeneration.

A deep learning approach for identifying cancer survivors living with post-traumatic stress disorder on Twitter.

BACKGROUND: Emotions after surviving cancer can be complicated. The survivors may have gained new strength to continue life, but some of them may begin to deal with complicated feelings and emotional stress due to trauma and fear of cancer recurrence. The widespread use of Twitter for socializing has been the alternative medium for data collection compared to traditional studies of mental health, which primarily depend on information taken from medical staff with their consent. These social media data, to a certain extent, reflect the users' psychological state. However, Twitter also contains a mix of noisy and genuine tweets. The process of manually identifying genuine tweets is expensive and time-consuming. METHODS: We stream the data using cancer as a keyword to filter the tweets with cancer-free and use post-traumatic stress disorder (PTSD) related keywords to reduce the time spent on the annotation task. Convolutional Neural Network (CNN) learns the representations of the input to identify cancer survivors with PTSD. RESULTS: The results present that the proposed CNN can effectively identify cancer survivors with PTSD. The experiments on real-world datasets show that our model outperforms the baselines and correctly classifies the new tweets. CONCLUSIONS: PTSD is one of the severe anxiety disorders that could affect individuals who are exposed to traumatic events, including cancer. Cancer survivors are at risk of short-term or long-term effects on physical and psycho-social well-being. Therefore, the evaluation and treatment of PTSD are essential parts of cancer survivorship care. It will act as an alarming system by detecting the PTSD presence based on users' postings on Twitter.

Identification of Cancer Survivors Living with PTSD on Social Media.

The trauma of cancer often leaves survivors with PTSD. Tweets posted on Twitter usually reflect the users' psychological state, which is convenient for data collection. However, Twitter also contains a mix of noisy and genuine tweets. The process of manually identifying genuine tweets is expensive and time-consuming. Thus, we propose a knowledge transfer technique to filter out unrelated tweets. Our experiments show that our model outperforms the baselines.