Locking Pretrained Weights via Deep Low-Rank Residual Distillation

Introduced DLR-Lock, a method to protect pretrained model weights by replacing MLPs with Deep Low-Rank Residual Networks (DLR-Nets). DLR-Nets increase activation memory linearly with depth during backpropagation, creating an optimization landscape mismatch for standard fine-tuning. This defense is trained via module-wise distillation and has been validated on LLMs to withstand adaptive attackers while preserving model capabilities.