Uncovering Layer-Dependent Activation Sparsity Patterns in ReLU Transformers

Arxiv Papers - Uncovering Layer-Dependent Activation Sparsity Patterns in ReLU Transformers