Constraining CLIPDraw
May 27, 2022

Summary
This post details an engineer's journey adapting the CLIPDraw model to constrain its output to the center of the canvas. The core technical contributions include: 1. Understanding and debugging PyTorch gradient flow issues when implementing custom loss functions. 2. Implementing vectorized operations on tensors for efficient distance calculations from the canvas center. 3. Iteratively refining the loss function to balance prompt similarity with spatial constraints, including introducing a differentiable approximation of a piecewise function to penalize points outside a defined radius, moving from a procedural to a differentiable approach.