Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

Arxiv Papers - Reducing Transformer Key-Value Cache Size with Cross-Layer Attention