Offsite-Tuning: Transfer Learning without Full Model

Transfer learning is important for foundation models to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning large foundation models is computation-intensive and impractical for most downstream users. In this paper, we propose Offsite-Tuning, a privacy preserving and efficient transfer learning framework that can adapt billion-parameter foundation models to downstream data without access to the full model. 2023: Guangxuan Xiao, Ji Lin, Song Han https://arxiv.org/pdf/2302.04870v1.pdf