Key Findings
Baidu released ERNIE 5.5 Flash Base on November 5, 2026, providing a pre-training checkpoint for developers to use in reinforcement-learning research and domain adaptation. The model is a mixture-of-experts (MoE) architecture with 280 billion total parameters, of which 14 billion are activated per token. In benchmarks, ERNIE leads in MMLU and HumanEval+ results, though it trails Hy3 in FullStackBench and DeepSeek in InfiniteBench.
Technical Details
The architecture consists of 43 layers with a hidden size of 4,096 and 64 attention heads. To optimize long-context processing, 37 layers employ Heavily Compressed Attention, while six Multi-head Latent Attention layers use an indexer to preserve token-level access to full context. Except for the first layer, all layers use MoE blocks with 512 routed experts (10 activated) and one shared expert operating in a 2,048-dimensional latent space.
Training involved 33.5 trillion tokens using the Muon optimizer and FP8 mixed precision. While training context was extended to 128K tokens, inference and fine-tuning support up to 1 million tokens. The training corpus included multilingual web data, software development records (commits, pull requests, CI-verified fixes), and synthetic retrieval data. For deployment, Baidu provides the FlashMask repository for Hopper and Blackwell GPUs, reporting forward pass speed gains of up to 175% over FlashAttention-4 MaskMod on SM100 hardware.
Background & Context
The model is available in FP8 and BF16 safetensors formats and is compatible with SGLang and vLLM. NVIDIA’s Megatron Bridge provides a supervised fine-tuning workflow; the full-model recipe requires 16 nodes with four GB300 GPUs per node and is validated on Blackwell GB200 and GB300 systems.
Significance & Outlook
The release is governed by a custom license that excludes the European Union, United Kingdom, and South Korea. Commercial use is free for entities with annual revenue at or below $50 million (requiring “Powered by ERNIE” attribution), while higher revenue requires separate written authorization. Additionally, the license prohibits using the model or its outputs to improve unrelated AI systems, restricting distillation and synthetic-data workflows.
Source: https://www.unite.ai/baidu-releases-ernie-5-5-flash-base-for-further-training/
Get our weekly technology intelligence — free
Receive an infographic that lets you judge at a glance whether each field’s analysis report is worth reading.
Subscribe Free — Weekly Tech Intelligence
By subscribing, you’ll receive Troy-Technical’s weekly technology intelligence newsletter.
- Your email and selected fields are used only to deliver the newsletter.
- We never share your information with third parties.
- You can unsubscribe anytime via the link in each email.
See our Privacy Policy for details.
Takes about a minute · Unsubscribe anytime
