DeepSeek: DeepSeek V3

DeepSeek-V3 is a Mixture-of-Experts (MoE) model from the DeepSeek team with 671B total parameters and 37B activated per token, building on the instruction-following and coding abilities of prior versions. It adopts Multi-head Latent Attention (MLA) and the DeepSeekMoE architecture, with an auxiliary-loss-free load-balancing strategy and a multi-token prediction objective. Pre-trained on nearly 15 trillion tokens, it reports 87.1% on MMLU, 89.3% on GSM8K, and 65.2% on HumanEval, with results that rival leading closed-source models.