Logo image
Back
Flew Over Learning Trap: Learn Unlearnable Samples by Progressive Staged Training
Preprint   Open access

Flew Over Learning Trap: Learn Unlearnable Samples by Progressive Staged Training

Pucheng Dang, Xing Hu, Kaidi Xu, Jinhao Duan, Di Huang, Husheng Han, Rui Zhang, Zidong Du, Qi Guo and Yunji Chen
arXiv.org
03 Jun 2023
DOI: https://doi.org/10.48550/arxiv.2306.02064
url
https://doi.org/10.48550/arxiv.2306.02064View
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute,  Open

Abstract

Computer Science - Computer Vision and Pattern Recognition
Unlearning techniques are proposed to prevent third parties from exploiting unauthorized data, which generate unlearnable samples by adding imperceptible perturbations to data for public publishing. These unlearnable samples effectively misguide model training to learn perturbation features but ignore image semantic features. We make the in-depth analysis and observe that models can learn both image features and perturbation features of unlearnable samples at an early stage, but rapidly go to the overfitting stage since the shallow layers tend to overfit on perturbation features and make models fall into overfitting quickly. Based on the observations, we propose Progressive Staged Training to effectively prevent models from overfitting in learning perturbation features. We evaluated our method on multiple model architectures over diverse datasets, e.g., CIFAR-10, CIFAR-100, and ImageNet-mini. Our method circumvents the unlearnability of all state-of-the-art methods in the literature and provides a reliable baseline for further evaluation of unlearnable techniques.

Metrics

7 Record Views

Details

Logo image