en.Wedoany.com Reported - NVIDIA recently open-sourced the Nemotron-Labs-TwoTower model, which enhances the text generation efficiency of large models through a dual-tower architecture. In tests, the model retained 98.7% of the quality performance of the autoregressive baseline model while achieving a 2.42x improvement in actual generation throughput. The relevant weights have been made available on the Hugging Face platform in the United States.
The technical focus of TwoTower is to separate the two tasks of "understanding context" and "generating new tokens" in traditional language models. Most autoregressive models generate tokens sequentially, one after another, where the next token cannot enter a stable output process until the previous token is completed. While this approach yields high quality, it often creates a speed bottleneck in scenarios involving long texts, code generation, agent tasks, and high-concurrency services. NVIDIA's proposed TwoTower architecture splits the model into a context tower and a denoising tower: the context tower is responsible for reading prompts and existing text, maintaining the autoregressive model's ability to understand semantics, logic, and context; the denoising tower is responsible for iteratively correcting noise blocks, allowing text generation to proceed in a more parallel manner.
From a model structure perspective, Nemotron-Labs-TwoTower is built upon Nemotron-3-Nano-30B-A3B, consisting of two model towers each approximately 30B in scale, employing a hybrid Mamba-Transformer MoE architecture. The context tower remains frozen, primarily undertaking a "read-only" context representation task; the denoising tower is trained separately, utilizing bidirectional block attention and layer-by-layer cross-attention to read semantic information from the context tower, then completing block-level generation and denoising. This design avoids a single network having to simultaneously handle both context representation and iterative denoising, and also allows the capabilities of the original pre-trained autoregressive model to be reused, eliminating the need to train a diffusion language model from scratch.
In terms of training and efficiency, the value of TwoTower extends beyond inference speed improvement. According to the paper, the model was trained on approximately 2.1 trillion tokens and adapted based on an open-weight 30B hybrid architecture model. Addressing the common issue of degraded semantic understanding in traditional diffusion language models, TwoTower retains a frozen autoregressive context tower to keep the existing model's grasp of linguistic knowledge and long context within the system, while the denoising tower specifically addresses the parallel generation problem. In other words, it does not simply sacrifice quality for speed but redesigns the generation process through a division of model tasks.
A 2.42x speedup in token generation has direct implications for developers and enterprise deployments. As large model applications enter domains like agents, AI programming, customer service Q&A, knowledge base generation, and long document processing, output speed affects user wait times, service concurrency capabilities, and inference costs. A single agent task may involve multiple rounds of planning, tool calls, code generation, result verification, and explanation output; the slower the token generation speed, the longer the overall task duration. If a model can increase throughput while maintaining near-original quality, the same hardware can handle more requests, or response times can be reduced for the same request volume.
This open-source release also means TwoTower is no longer just a lab architecture. The Hugging Face page in the United States shows that Nemotron-Labs-TwoTower-30B-A3B-Base-BF16 can be loaded via Transformers, and deployed using vLLM, SGLang, Docker Model Runner, and other methods. For researchers, open weights facilitate experiment replication, comparison of decoding modes, and exploration of diffusion language model performance in long text generation; for engineering teams, the open-source model can be integrated into local test environments to evaluate inference costs, GPU memory usage, service throughput, and quality stability.
However, TwoTower does not mean all text tasks will unconditionally experience a 2.42x improvement. Actual speed is affected by hardware configuration, batch size, context length, decoding strategy, deployment framework, and task type. For short Q&A, low concurrency, or scenarios with strong dependencies on generation order, the benefits may be less pronounced than for long text generation, code completion, and multi-step agent outputs. The model retaining 98.7% quality also implies some capability loss, especially in tasks requiring mathematics, code, or rigorous reasoning; developers need to validate it against their own business data.
NVIDIA's open-sourcing of TwoTower reflects that competition in large models is moving from parameter scale expansion into the deep waters of generation efficiency, inference costs, and engineering deployment capabilities. Faster token generation speeds can bring AI applications closer to real-time interaction; open weights and support for mainstream deployment frameworks allow enterprises to test new architectures on their own hardware and business processes. The role of TwoTower is not to replace all autoregressive models, but to provide a new generation pathway for long texts, high concurrency, agents, and local inference scenarios: retaining the context understanding capabilities of existing models while using a diffusion-based denoising mechanism to address the speed bottleneck of token-by-token generation.
