Implement 2D convolution forward pass
Problem Statement
Implement the forward pass of a 2D convolution (conv2d) operation from scratch, without using any existing deep learning libraries or pre-built convolution operators.
You will be provided with the following inputs:
- Input tensor
x: A 4D tensor with shape(N, C_in, H, W), representing a batch ofNimages, each withC_ininput channels, heightH, and widthW(NCHW layout). - Filter weights
w: A 4D tensor with shape(C_out, C_in, K_h, K_w), representingC_outoutput filters, each operating onC_ininput channels, with kernel heightK_hand kernel widthK_w. - Optional bias
b: A 1D tensor with shape(C_out,). This may beNone, in which case no bias should be added. - Integer strides
s_h, s_w: The stride values for height and width, respectively. - Integer padding
p_h, p_w: The zero-padding values applied to the height and width dimensions of the input tensorx.
Your task is to compute and return the output tensor y with shape (N, C_out, H_out, W_out).
The output dimensions H_out and W_out are calculated as follows:
H_out = floor((H + 2*p_h - K_h) / s_h) + 1
W_out = floor((W + 2*p_w - K_w) / s_w) + 1
Each element of the output tensor y is computed using the following formula:
y[n, c_out, i, j] = sum(x_pad[n, c_in, i*s_h + u, j*s_w + v] * w[c_out, c_in, u, v]) + b[c_out]
where the sum is over c_in from 0 to C_in-1, u from 0 to K_h-1, and v from 0 to K_w-1. x_pad refers to the input tensor x after being zero-padded by p_h and p_w along its height and width dimensions, respectively.
Requirements
- The function must return
yas a dense numeric array or tensor. - You are strictly prohibited from using any existing convolution operators or deep learning library functions that perform convolution.
- Your implementation must correctly handle edge cases, including when the bias
bisNone, when kernels are non-square (K_h != K_w), and when strides or padding values are different (s_h != s_worp_h != p_w).
Constraints
- The input sizes will be sufficiently small such that an implementation with a time complexity of
O(N * C_out * C_in * H_out * W_out * K_h * K_w)will pass within typical interview time limits. - All input tensors (
x,w,b) will contain floating-point numbers.
Example
Given an input tensor x of shape (1, 3, 32, 32), filter weights w of shape (8, 3, 3, 3), a stride of (1, 1), and padding of (1, 1), the expected output tensor y will have a shape of (1, 8, 32, 32).
AI interview prep
Core dilemma
Correctly translating the mathematical definition of 2D convolution into an efficient and bug-free nested loop implementation, managing complex multi-dimensional array indexing, zero-padding, and stride logic without relying on high-level libraries.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Implement 2D convolution forward pass
Problem Statement
Implement the forward pass of a 2D convolution (conv2d) operation from scratch, without using any existing deep learning libraries or pre-built convolution operators.
You will be provided with the following inputs:
- Input tensor
x: A 4D tensor with shape(N, C_in, H, W), representing a batch ofNimages, each withC_ininput channels, heightH, and widthW(NCHW layout). - Filter weights
w: A 4D tensor with shape(C_out, C_in, K_h, K_w), representingC_outoutput filters, each operating onC_ininput channels, with kernel heightK_hand kernel widthK_w. - Optional bias
b: A 1D tensor with shape(C_out,). This may beNone, in which case no bias should be added. - Integer strides
s_h, s_w: The stride values for height and width, respectively. - Integer padding
p_h, p_w: The zero-padding values applied to the height and width dimensions of the input tensorx.
Your task is to compute and return the output tensor y with shape (N, C_out, H_out, W_out).
The output dimensions H_out and W_out are calculated as follows:
H_out = floor((H + 2*p_h - K_h) / s_h) + 1
W_out = floor((W + 2*p_w - K_w) / s_w) + 1
Each element of the output tensor y is computed using the following formula:
y[n, c_out, i, j] = sum(x_pad[n, c_in, i*s_h + u, j*s_w + v] * w[c_out, c_in, u, v]) + b[c_out]
where the sum is over c_in from 0 to C_in-1, u from 0 to K_h-1, and v from 0 to K_w-1. x_pad refers to the input tensor x after being zero-padded by p_h and p_w along its height and width dimensions, respectively.
Requirements
- The function must return
yas a dense numeric array or tensor. - You are strictly prohibited from using any existing convolution operators or deep learning library functions that perform convolution.
- Your implementation must correctly handle edge cases, including when the bias
bisNone, when kernels are non-square (K_h != K_w), and when strides or padding values are different (s_h != s_worp_h != p_w).
Constraints
- The input sizes will be sufficiently small such that an implementation with a time complexity of
O(N * C_out * C_in * H_out * W_out * K_h * K_w)will pass within typical interview time limits. - All input tensors (
x,w,b) will contain floating-point numbers.
Example
Given an input tensor x of shape (1, 3, 32, 32), filter weights w of shape (8, 3, 3, 3), a stride of (1, 1), and padding of (1, 1), the expected output tensor y will have a shape of (1, 8, 32, 32).