A Workaround for Non-Determinism in TensorFlow
May 24, 2017Two Sigma
Summary
This post details a workaround for non-determinism in TensorFlow's GPU operations, specifically addressing the `reduce_sum` operation. The author demonstrates how to replace non-deterministic `reduce_sum` with a deterministic matrix multiplication approach, and how to adapt neural network architectures (e.g., by augmenting biases into the weight matrix) to leverage this deterministic operation for repeatable training results.