Represent k-means as an MLP
Problem Description
Given a set of fixed centroids $q_1, ..., q_k$ and an input vector $x$, demonstrate how the nearest-centroid assignment step of squared-Euclidean k-means can be implemented using a fully connected neural network layer followed by an activation function.
You must:
- Utilize the identity $||x - q_j||^2 = ||x||^2 + ||q_j||^2 - 2 q_j^T x$ to derive the appropriate layer weights and biases.
- Explain how a low-temperature softmax function can approximate a one-hot cluster assignment.
- Describe how to implement the softmax function stably when the temperature is very small.
AI interview prep
Core dilemma
Bridging the mathematical equivalence between a classical clustering algorithm (k-means) and a neural network component (fully connected layer + activation), while ensuring numerical stability for practical implementation.
2 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Represent k-means as an MLP
Problem Description
Given a set of fixed centroids $q_1, ..., q_k$ and an input vector $x$, demonstrate how the nearest-centroid assignment step of squared-Euclidean k-means can be implemented using a fully connected neural network layer followed by an activation function.
You must:
- Utilize the identity $||x - q_j||^2 = ||x||^2 + ||q_j||^2 - 2 q_j^T x$ to derive the appropriate layer weights and biases.
- Explain how a low-temperature softmax function can approximate a one-hot cluster assignment.
- Describe how to implement the softmax function stably when the temperature is very small.