Alessandro Ingrosso, researcher at the Donders Institute for Neuroscience, has developed a new mathematical method in collaboration with colleagues in two Italian research institutions, which enables the prediction of the effectiveness of transfer learning in neural networks.
The paper is published in the journal Physical Review Letters.
The problem of limited data
AI systems are typically trained with large amounts of data. Consider an image recognition system trained to identify a dog in a picture. By training this network with large quantities of dog photos, the network learns to identify a dog with high accuracy in new images.
However, for some applications, sufficient training data is not available. “In the medical sector, for example, when diagnosing cancer via ultrasound, there aren’t enough examples available to properly train a neural network,” explains Ingrosso.
“This leads to ‘overfitting,’ where the network fails at generalizing on new, unseen cases. The number of false positives and false negatives can then become very large.”
To solve this problem, the researchers focused on “transfer learning,” a technique where knowledge from a network trained on a large dataset (the “source”) is transferred to a new network (the “target”) that is trained on limited data.
“We studied the simplest model for this transfer learning approach and developed a mathematical theory for networks with one hidden layer,” says Ingrosso.
Innovative combination of analytical methods
In this theory, the researchers combined two different analytical methods: the recently developed “Kernel Renormalization” approach and the classical “Franz-Parisi” formalism from the theory of Spin Glasses. Merging such tools makes it possible to work with specific, real datasets instead of statistical models or approximations.
Ingrosso adds, “Our new method can directly and accurately predict how effective the target network will be in generalizing data when it adopts knowledge from the source network.”
This research provides important new insights for the effective training of AI systems in domains where data is scarce, such as medical diagnostics and other specialized applications.
More information:
Alessandro Ingrosso et al, Statistical Mechanics of Transfer Learning in Fully Connected Networks in the Proportional Limit, Physical Review Letters (2025). DOI: 10.1103/PhysRevLett.134.177301. On arXiv: DOI: 10.48550/arxiv.2407.07168
Radboud University
Citation:
Scientists develop new mathematical model for transfer learning in neural networks (2025, May 7)
retrieved 7 May 2025
from https://techxplore.com/news/2025-05-scientists-mathematical-neural-networks.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
