arXiv Artificial Intelligence· Hyunwoo Yu, Jongbeom Lee, Hyunbin Kim, Sungjun Park, Jaemin Choi, Jincheol Yang, Seunghun Moon, Bongjoon Hyun, Kyomin Sohn, Kyeongbo Kong, Suk-Ju Kang·· 3 小时前AI 评分38
OpenAttention：用尾部代理实现 1% 读取的稀疏注意力
OpenAttention: One Percent Is Enough with Tail Proxies
AI 导读
OpenAttention 用聚类、ErrorGate 与 top-k 选择加计数感知均值代理来近似稠密注意力分布，在 Llama-3.1-8B 的 RULER-32K HARD 上以 1% 读取保留 99% 稠密准确率。
来源：arXiv Artificial Intelligence · arxiv.org