Engineering

Agents-A1

❖ Agents-A1 is a 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling the agent horizon. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities.

VeRL-Omni

❖ VeRL-Omni is a general RL training framework focused on multimodal generative models, built on top of verl (Volcano Engine Reinforcement Learning). It is designed to provide easy, fast, and stable RL training for diffusion and omni-modality models.

❖ DeepSeek Sparse Attention (DSA) was a breakthrough for long-context models, reducing core attention complexity from O(L²) to O(Lk), but its lightning indexer remained an expensive O(L²) tax at every layer. IndexCache eliminates up to 75% of indexer computations in DSA through cross-layer index reuse — achieving up to 1.82× prefill speedup and 1.48× decode speedup with negligible quality degradation.

❖ This is the introduction of Attention Residuals (AttnRes), a drop-in replacement for standard residual connections in Transformers that enables each layer to selectively aggregate earlier representations via learned, input-dependent attention over depth.

MTP in SGLang

❖ SGLang is the first and only open-source serving framework to support Multiple Token Prediction (MTP) in combination with Large-Scale Expert Parallelism (EP) and Prefill-Decode disaggregation. This integration delivers up to 60% higher output throughput through a new decoding paradigm, better parallelism, and more efficient resource utilization without sacrificing generation quality.

❖ slime is an LLM post-training framework for RL scaling, providing two core capabilities: High-Performance Training – Supports efficient training in various modes by connecting Megatron with SGLang; Flexible Data Generation – Enables arbitrary training data generation workflows through custom data generation interfaces and server-based engines.

Large-Scale Expert Parallelism

❖ DeepSeek is a popular open-source large language model (LLM) praised for its strong performance. However, its large size and unique architecture, which uses Multi-head Latent Attention (MLA) and Mixture of Experts (MoE), require an advanced system for efficient serving at scale. In this blog, we explain how we match DeepSeek’s inference system performance using prefill-decode disaggregation and large-scale expert parallelism (EP) with SGLang.

vLLM V1

❖ We are thrilled to announce the alpha release of vLLM V1, a major upgrade to vLLM’s core architecture. Based on lessons we learned over the past 1.5 years of vLLM development, we revisited key design decisions, consolidated various features, and simplified the codebase to enhance flexibility and scalability. V1 already achieves state-of-the-art performance and is set to gain even more optimizations.