Image Generation Optimization
Constraining CLIPDraw

Constraining CLIPDraw

5/27/2022

What this post added

This post details an engineer's journey adapting the CLIPDraw model to constrain its output to the center of the canvas. The core technical contributions include: 1. Understanding and debugging PyTorch gradient flow issues when implementing custom loss functions. 2. Implementing vectorized operations on tensors for efficient distance calculations from the canvas center. 3. Iteratively refining the loss function to balance prompt similarity with spatial constraints, including introducing a differentiable approximation of a piecewise function to penalize points outside a defined radius, moving from a procedural to a differentiable approach.

Read the original post ↗