Exploring connections of spectral analysis and transfer learning in medical imaging. Academic Article uri icon

Overview

abstract

  • In this paper, we use spectral analysis to investigate transfer learning and study model sensitivity to frequency shortcuts in medical imaging. By analyzing the power spectrum density of both pre-trained and fine-tuned model gradients, as well as artificially generated frequency shortcuts, we observe notable differences in learning priorities between models pre-trained on natural vs medical images, which generally persist during fine-tuning. We find that when a model's learning priority aligns with the power spectrum density of an artifact, it results in overfitting to that artifact. Based on these observations, we show that source data editing can alter the model's resistance to shortcut learning. Code available at: https://github.com/YCL92/Shortcut-PSD.

publication date

  • April 11, 2025

Identity

PubMed Central ID

  • PMC12201968

Scopus Document Identifier

  • 105004584334

Digital Object Identifier (DOI)

  • 10.1117/12.3047670

PubMed ID

  • 40575597

Additional Document Info

volume

  • 13406