Your browser doesn't support javascript.
loading
Mostrar: 20 | 50 | 100
Resultados 1 - 1 de 1
Filtrar
Mais filtros










Base de dados
Intervalo de ano de publicação
1.
Neural Netw ; 173: 106204, 2024 May.
Artigo em Inglês | MEDLINE | ID: mdl-38412738

RESUMO

We explore element-wise convex combinations of two permutation-aligned neural network parameter vectors ΘA and ΘB of size d. We conduct extensive experiments by examining various distributions of such model combinations parametrized by elements of the hypercube [0,1]d and its vicinity. Our findings reveal that broad regions of the hypercube form surfaces of low loss values, indicating that the notion of linear mode connectivity extends to a more general phenomenon which we call mode combinability. We also make several novel observations regarding linear mode connectivity and model re-basin. We demonstrate a transitivity property: two models re-based to a common third model are also linear mode connected, and a robustness property: even with significant perturbations of the neuron matchings the resulting combinations continue to form a working model. Moreover, we analyze the functional and weight similarity of model combinations and show that such combinations are non-vacuous in the sense that there are significant functional differences between the resulting models.


Assuntos
Redes Neurais de Computação , Neurônios , Vias Neurais/fisiologia , Imageamento por Ressonância Magnética , Encéfalo
SELEÇÃO DE REFERÊNCIAS
DETALHE DA PESQUISA
...