DeepSeek announces NSA for ultra-fast long context training and inference

DeepSeek announces NSA for ultra-fast long-context training and inference. According to it, NSA is a hardware-consistent and natively trainable sparse attention mechanism for ultra-fast long-context training and inference.

With an optimized design for modern hardware, NSA accelerates inference while reducing pre-training costs without compromising performance.

It matches or outperforms fully attention models on general benchmarks, long-context tasks, and instruction-based reasoning.

Editor in charge: Liu Mingliang