文章摘要
文章目录1. Context Extension1.1 Rotary Position Embedding (RoPE)1.2 LongLoRA2. Evaluation of Long-Context LLMs2.1 The Lost in the Middle Phenomenon2.2 Long-Context Benchmarks: NIAH, LongBench3. Efficient Attention Mechanisms3.1 KV Cache3.2 StreamingLLM and…