Two Sigma

A Workaround for Non-Determinism in TensorFlow

May 24, 2017Two Sigma
A Workaround for Non-Determinism in TensorFlow
SummaryAI, from the source post

This post details a workaround for non-determinism in TensorFlow's GPU operations, specifically addressing the `reduce_sum` operation. The author demonstrates how to replace non-deterministic `reduce_sum` with a deterministic matrix multiplication approach, and how to adapt neural network architectures (e.g., by augmenting biases into the weight matrix) to leverage this deterministic operation for repeatable training results.