Interpreting vision and language generative models with semantic visual priors.

When applied to Image-to-text models, explainability methods have two challenges. First, they often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence. This makes explanations expensive to compute and unable to comprehensively explain the model's output. Second, for models with visual inputs, explainability methods such as SHAP typically consider superpixels as features. Since superpixels do not correspond to semantically meaningful regions of an image, this makes explanations harder to interpret. We develop a framework based on SHAP, that allows for generating comprehensive, meaningful explanations leveraging the meaning representation of the output sequence as a whole. Moreover, by exploiting semantic priors in the visual backbone, we extract an arbitrary number of features that allows the efficient computation of Shapley values on large-scale models, generating at the same time highly meaningful visual explanations. We demonstrate that our method generates semantically more expressive explanations than traditional methods at a lower compute cost and that it can be generalized to a large family of vision-language models.

Unreliability of the %VO2 reserve versus %heart rate reserve relationship for aerobic effort relative intensity assessment in chronic heart failure patients on or off beta-blocking therapy.

BACKGROUND: Percentage of heart rate reserve (%HRR) has been shown to be equivalent to percentage of VO2 reserve (%VO2R) in normal individuals, but it is not clear whether this is also true for chronic heart failure (CHF) patients. DESIGN: This study aimed to evaluate the reliability of the %VO2R versus %HRR relationship for aerobic effort relative intensity assessment in CHF patients. METHODS: We studied 388 patients with CHF, of whom 213 were on carvedilol (CHF on betab) and 175 were off the drug (CHF off betab), and 40 normal individuals (N). %VO2R and %HRR values were evaluated at matched steps during cardiopulmonary exercise testing. The %VO2R versus %HRR regression was determined for each group (CHF, CHF on betab, CHF off betab, N) as a whole and as the mean of individual regressions. RESULTS: %VO2R strongly correlated with %HRR in both N and CHF, at both group and mean individual regressions, with slope and y-intercept values significantly lower and higher, respectively, in CHF than N. The 95% prediction interval of %VO2R for a given %HRR value was 24% points in N but rose to 41 in CHF, and the mean regression line coincided with identity line in N but not in CHF. These results were independent of beta-blockade. CONCLUSIONS: In CHF patients the %VO2R versus %HRR relationship is unreliable for assessment of aerobic effort relative intensity, because of a large prediction interval of %VO2R and lack of coincidence with the identity line, independently of beta-blocking therapy. This implies that the %VO2R versus %HRR relationship should be determined directly in each patient.