TechPrufer
Get started
Apple
Apple logoApple · Feature trail

Weight Locking via Deep Low-Rank Residual Distillation

1 post · Aug 2026

What this trail covers

This post introduces DLR-Lock, a novel defense mechanism for open-weight language models that prevents unauthorized modification of pretrained weights. DLR-Lock replaces standard MLPs with Deep Low-Rank Residual Networks (DLR-Nets), which exploit the inference-training asymmetry of automatic differentiation. This defense forces increased activation memory during backpropagation and creates architectural mismatches that complicate fine-tuning, effectively withstanding adaptive attackers while preserving original model capabilities. Experiments on LLMs validate its efficacy.

All Apple trails
  1. Locking Pretrained Weights via Deep Low-Rank Residual DistillationIntroduced DLR-Lock, a method to protect pretrained model weights by replacing MLPs with Deep Low-Rank Residual Networks (DLR-Nets). DLR-Nets increase activation memory linearly with depth during backpropagation, creating an optimization landscape mismatch for standard fine-tuning. This defense is trained via module-wise distillation and has been validated on LLMs to withstand adaptive attackers while preserving model capabilities.