Paper accepted at ICCV 2025: “Disrupting Model Merging”
Model merging combines several fine-tuned models into one without additional training. It also lets a free-rider cheaply inherit the specialized capabilities of someone else’s model. Existing approaches such as watermarking or fingerprinting can only detect merging after the fact; they cannot prevent it.
This work proposes the first proactive defense against model merging. Two operations — rearranging MLP parameters and scaling attention heads — push the model out of the shared basin in parameter space, so that performance degrades sharply when it is merged with another model while its behavior stays unchanged when used on its own. The approach is validated on image classification, image generation, and text classification.
[Paper]
Junhao Wei, Zhe Yu, Jun Sakuma, “Disrupting Model Merging: A Parameter-Level Defense Without Sacrificing Accuracy,” Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV 2025), pp. 17698-17707, October 2025.