NVIDIA Releases Nemotron 3 Super: 120B Parameter Open-Source Hybrid Mamba-Attention MoE Model Delivers 5x Higher Throughput for Agentic AI

The gap between proprietary boundary models and more transparent open source models is closing faster than ever. NVIDIA has officially pulled back the curtain Nemotron 3 Superan incredible 120 billion parameter model built specifically for multi-agent applications.

Released today, Nemotron 3 Super it sits nicely between the lightweight parameter of the 30 billion Nemotron 3 Nano and the highly anticipated parameter of the 500 billion Nemotoron 3 Ultra coming later in 2026. Delivering up to 7x higher productivity and double the accuracy of its previous generation, this model is a major step forward for engineers who refuse to compromise between intelligence efficiency.

The ‘Five Wonders’ of Nemotron 3 Super

The unprecedented performance of the Nemotron 3 Super is driven by five major technological achievements:

• Hybrid MoE Architecture: The model intelligently combines Mamba layers with good memory and Transformer layers with high precision. By turning on only a fraction of the parameters to generate each token, it achieves a 4x increase in cache efficiency over KV and SSM.
• Multi-Token Prediction (MTP): The model can predict multiple future tokens simultaneously, resulting in 3x faster reasoning times for complex reasoning tasks.
• 1 Million Content Window: Boasting a context length 7x greater than the previous generation, developers can drop large technical reports or entire codebases directly into model memory, eliminating the need for re-consultation in multi-step workflows.
• Hidden MoE: This allows the model to compress information and enable four experts to work at the same computational cost as one. Without this innovation, the model would need to be 35 times larger to achieve the same levels of accuracy.
• NeMo RL Gym Integration: Using interactive reinforcement learning pipelines, the model learns from dynamic feedback loops rather than static text, doubling its intelligence index.

All of this works out, leading to incredible efficiency in terms of output tokens per GPU

Why Nemotron 3 Super is the Ultimate Multi-Agent AI Engine?

The Nemotron 3 Super is not just an ordinary large tongue model; it is uniquely positioned as a logic engine designed to plan, verify, and execute complex operations within a comprehensive system of specialized models. Here’s exactly why its design makes it a game changer for multi-agent workflows:

• Top Performance for Deep Reasons: The model’s 7x higher performance expands its search space. Because it can process and generate tokens faster, it can explore more trajectories and explore better responses. This allows developers to use deep thinking on the same computing budget, which is important for building complex, autonomous agents.
• Zero “Re-Consultation” in Long Distance Flows: In multi-agent systems, agents are constantly passing context back and forth. A 1 million token context window allows the model to store large amounts of state, such as entire codes or long, multi-step histories of an agent, directly in its memory. This eliminates the delay and cost of forcing the model to re-process the context every single step.
• Agent-specific training areas: Instead of relying only on static text data sets, the model pipeline has been expanded to cover more than 15 learning scenarios. By training in dynamic simulation loops (such as dedicated software engineering agent environments and advanced search tools), Nemotron 3 Super has learned the correct ways to complete an autonomous task.
• Advanced Tool Calling Capabilities: In real-world multi-agent applications, models need to act, not just respond textually. Out of the box, the Nemotron 3 Super has proven to be highly capable in tooling, able to successfully navigate large pools of available tasks—such as dynamically selecting from over 100 different tools in complex cybersecurity workflows.

Open Sourced and Training Scale

NVIDIA doesn’t just release weights; they completely open source the entire model stack, which includes training datasets, libraries, and reinforcement learning environments.

Because of this level of transparency, Performance Analysis places the Nemotron 3 Super squarely in the ‘most attractive quadrant,’ noting that it achieves high aperture scores while maintaining leading accuracy alongside proprietary models. The foundation of this intelligence comes from a completely redesigned pipeline trained with 10 trillion selected tokens, supplemented by an additional 9 to 10 billion tokens focused strictly on advanced coding and reasoning tasks.

Developer Control: Introducing ‘Consulting Budgets‘

While the raw parameter calculations and benchmark scores are impressive, the NVIDIA team understands that real-world business developers need precise control over latency, user experience, and compute costs. To solve the age-old problem of intelligence-versus-speed, the Nemotron 3 Super introduces a highly flexible Ways of Thinking directly through its API, it puts an unprecedented level of granular control in the hands of developers.

Instead of forcing a one-size-fits-all output, developers can adjust dynamically how hard the model ‘thinks’ based on the specific task it performs:

• Full Consultation (Automatic): The model is released to use its maximum power, exploring deep search spaces and multi-step processes to solve the most complex, agent-based problems.
• ‘Thinking Budget’: This is a complete game changer for latency sensitive applications. Developers can explicitly disable the model’s reasoning time or calculate the clearance. By setting a tight thinking budget, the model intelligently optimizes its internal search space to deliver the best possible answer. to that exact limit.
• ‘Low Effort Mode’: Not all information requires in-depth, multi-agent analysis. If the user just needs a simple, short answer (such as a general summary or a basic Q&A) without a deep thinking head, this modification turns the Nemotron 3 Super into a fast answerer, saving huge amounts of computation and time.

The ‘Golden’ Configuration

Tuning logic models can often be a frustrating process of trial and error, but the NVIDIA team has completely eliminated it in this release. To bring out the best performance in every area everything of these changing ways, NVIDIA recommends global settings of Temperature 1.0 and Top P 0.95.

According to the NVIDIA team, locking in these precise hyperparameter settings ensures that the model maintains the perfect mathematical balance of creative testing and logical accuracy, whether it is working in a delayed low-effort mode or an unblocked deep-dive.

Real World Applications and Availability

The Nemotron 3 Super is already proving its potential in all business applications:

• Software Development: It handles pull requests at a low level and surpasses the best proprietary localization models, effectively finding the line of code that causes the bug.
• Cybersecurity: The model is at the forefront of complex navigation of ISV security operations with logic driving advanced tools.
• Sovereign AI: Organizations around the world in regions such as India, Vietnam, South Korea, and Europe use the Nemotron Architecture to build specialized, localized models for specific regions and regulatory frameworks.

The Nemotron 3 Super is released in BF16, FP8, and NVFP4 ratings, with NVFP4 required to run the model on the DGX Spark.

See Models at A Hugging Face. You can find details at Research Paper again Technology/Engineering Blog.

_{Thanks to the NVIDIA AI team for the thought leadership/Resources for this article. The NVIDIA AI team supported and sponsored this content/article.}

Jean-marc is a successful AI business executive .He leads and accelerates the development of AI-powered solutions and started a computer vision company in 2006. He is a well-known speaker at AI conferences and has an MBA from Stanford.