Nvidia Develops Nemotron 4, Largest Model Expected to Reach Trillion-Parameter Scale

en.Wedoany.com Reported - On August 11, multiple employees involved in Nvidia's Nemotron 4 project disclosed that the U.S. chip company is developing a new family of open artificial intelligence models, with the largest model expected to have at least 1 trillion parameters. Nvidia reiterated the same day that it is advancing Nemotron 4 development but did not confirm the aforementioned parameter scale or release timeline.

Nemotron 4 has not yet completed final training, and Nvidia has not set an official release date. Employees involved in the project expect the series could be ready as early as late fall 2026, with the development goal of competing with major open models worldwide.

Nvidia's current largest model in the Nemotron 3 series is Nemotron 3 Ultra, which employs a mixture-of-experts and Mamba-Attention architecture, featuring 550 billion total parameters and activating 55 billion parameters per token. The model was pretrained on 20 trillion text tokens and supports a context window of up to 1 million tokens.

Nvidia also released Nemotron 3.5 Lightning on August 11. The model has 30 billion total parameters and 3 billion activated parameters, and can be used for agentic tasks such as code review, tool calling, and security alert monitoring. It is already commercially available.