Research
30 billion total parameters. 3 billion active at any given moment. A hybrid Mamba-Transformer-MoE architecture that delivers 3.3x higher throughput than comparable models. Here's what the benchmarks actually say, where it leads, and where it falls short.
March 2026 | Dry Ground AI Research
30B
Total Parameters
31.6B with embeddings
3B
Active Parameters
Per token during inference
1M
Context Window
Native, not extended
3.3x
Throughput
Faster than Qwen3-30B-A3B
~30GB
VRAM (FP8)
Fits on a single A100
Hybrid
Architecture
Mamba-2 + Transformer + MoE
Architecture
Nemotron 3 Nano is not a standard transformer. It is a hybrid that combines three different architectural approaches into a single 52-layer backbone. Each layer type handles a different class of problem, and the combination delivers capabilities that none of them achieve alone.
The result: a model that processes 1M tokens natively, runs on a single GPU, and generates output at 126 tokens per second on an H200.
State-space models that scale linearly with sequence length. When you are processing a 256K-token context window, this is the difference between "fits on one GPU" and "needs a cluster."
Pure Mamba can struggle with finding one specific fact buried in 100K tokens of noise. These attention layers provide high-fidelity retrieval when it matters most.
Each token activates only 6 experts plus 1 shared expert out of 128 total. 30B parameters worth of knowledge, 3B of compute per token. The efficiency gains compound fast at scale.
Strengths
Against open models under 30B MoE, Nemotron 3 Nano takes the top spot on long-context retrieval, math reasoning with tools, and instruction following.
| Benchmark | Score | Why it matters |
|---|---|---|
| AIME 2025 (with tools) | 99.2 | Math reasoning with tool access. Near-perfect. |
| RULER @ 256K tokens | 92.9% | Long-context retrieval accuracy. Best in class. |
| RULER @ 1M tokens | 86.3% | Still retrieving accurately at 1M tokens. Most models fall apart here. |
| Arena Hard v2 | 67.7 | Head-to-head instruction following. |
| IFBench | 71.5 | Complex multi-constraint instruction following. |
Comparison
Two models, nearly identical on paper. Roughly 30B total parameters, roughly 3B active per token, both targeting single-GPU deployment, both open-weight. But the architectures are fundamentally different, and so are the results.
The SWE-bench gap is the one that stings. Fixing 39% of real GitHub issues versus 69% is not a minor difference. For automated code repair workflows, that is a different capability tier entirely. But benchmarks measure a model in isolation. In production, you rarely use one model alone.
| Benchmark | Nemotron 3 Nano | Qwen3.5-35B-A3B | Gap |
|---|---|---|---|
| MMLU-Pro | 78.3 | 85.3 | -7.0 |
| GPQA Diamond | 73.0 | 84.2 | -11.2 |
| SWE-bench Verified | 38.8 | 69.2 | -30.4 |
| AIME 2025 (with tools) | 99.2 | - | - |
| RULER @ 1M tokens | 86.3% | 77.5% | +8.8 |
| Throughput (single H200) | 126 tok/s | ~38 tok/s | 3.3x |
Performance
On identical hardware (single H200, 8K input / 16K output), Nemotron 3 Nano delivers 3.3x higher throughput than Qwen3-30B-A3B. That is not marketing. That is architectural. Mamba-2's linear complexity versus attention's quadratic cost shows up every single inference call.
For agentic workloads with dozens of sequential LLM calls per task, this compounds. A 3.3x speed advantage on every call in a 20-step agent chain means your workflow finishes in a third of the time. Or you serve 3x the concurrent users on the same hardware.
NVIDIA designed this model for exactly this use case. Multi-agent systems generate up to 15x the tokens of standard chat interactions. Every token-per-second matters.
3.3x
Throughput advantage
vs Qwen3-30B-A3B
126
Tokens per second
Single H200 GPU
15x
Token multiplier
Multi-agent vs standard chat
Strategy
Nemotron 3 Nano is not meant to do everything. It is meant to do most things fast and cheap, while a bigger model handles the hard stuff. This tiered approach is how production AI systems actually work. No one runs GPT-5 for every API call. The economics do not support it. The latency does not support it. And most tasks do not require it.
Nano
30B total / 3B active
Routing, simple tasks, high-volume work
Fast, cheap, good enough for 80% of agent actions.
Super
120B total / 12B active
Complex reasoning, multi-step planning
Slower, more expensive, but significantly more capable for hard tasks.
Proprietary
Varies
Expert-level work
When you need frontier-class performance on a specific task.
Feature
Nemotron 3 Nano supports configurable thinking budgets. You can toggle reasoning on or off and set a maximum token budget for the model's chain-of-thought process.
Not every request needs deep reasoning. A routing decision in a multi-agent system does not need 2,000 tokens of internal deliberation. A classification task does not need chain-of-thought. Controlling the reasoning budget reduces latency and cost on tasks that do not benefit from extended thinking.
This is a practical feature that most benchmarks do not capture, but operators notice immediately.
Deployment
NVIDIA provides official FP8 weights and TensorRT-LLM optimization out of the box. Framework support includes vLLM, SGLang, and TensorRT-LLM. The Mamba-2 architecture is well-supported across NVIDIA's stack, though non-NVIDIA inference frameworks are still catching up.
| Configuration | VRAM Required | Compatible Hardware |
|---|---|---|
| BF16 (full precision) | ~60GB | A100 80GB, H100, H200 |
| FP8 (NVIDIA official) | ~30GB | A100 40GB+, A40, RTX 6000 |
| Q4 (quantized) | ~18GB | RTX 4090, RTX 5090 |
Assessment
Takeaway
The 3B-active-parameter class is becoming one of the most interesting spaces in open-weight AI. Models like Nemotron 3 Nano and Qwen3.5-35B-A3B prove that you do not need 70B+ active parameters to build production-grade AI systems. The MoE architecture has changed the math entirely.
For enterprises evaluating self-hosted options, the question is not "which model is best." It is "which model is best for each layer of my system." The answer is almost certainly a combination.
NVIDIA is betting that speed and efficiency win the infrastructure layer while quality wins the reasoning layer. Based on these benchmarks, that bet looks well-placed.
We run our own GPU infrastructure, deploy our own models, and build systems that work in production. If self-hosted inference is on your radar, let's talk.
Book a ConsultationWe use cookies to improve your experience. Cookie Policy · Privacy Policy