Nyströmformer: Approximating self-attention in linear time and memory via the Nyström method

This post details the Nyströmformer model, which approximates self-attention in linear time and memory. It leverages the Nyström method to reduce the computational complexity of self-attention mechanisms in transformer models, making them more efficient for processing long sequences. The model is available as `uw-madison/nystromformer-1024` for fill-mask tasks.