Nvidia's Nemotron 3.5 Bolt puts a 120B-class agent inside a 24GB GPU — and on laptops shipping tomorrow
Three new open-weight Nemotron models span data center to desktop, giving small B2B teams a credible path to always-on local sales agents without per-query cloud costs.
Nvidia released three open-weight Nemotron 3.5 models on Oct. 15, and the structural news isn’t the flagship. It’s the compression. A 74-billion-parameter mixture-of-experts model, Bolt 74B-A9B, now fits inside a 24GB consumer GPU through 2-bit quantization, activating 9B parameters per step. The portable version, Bolt E4B, ships Oct. 23 and targets a 4GB memory footprint. RTX Spark laptops land Oct. 16.
What that means in practice: the agentic workload (prospect research, document parsing, CRM updates, drafting outreach) that justified a cloud API bill a quarter ago now runs on a workstation a five-person team already owns.
The flagship is still impressive. Nemotron 3.5 Super VL is a 120B multimodal MoE activating 12B parameters per token, trained on roughly 26 trillion tokens with about 3.8 million post-training samples drawn from around 100 open reinforcement-learning environments. Nvidia claims up to 5x higher single-GPU throughput than peer open models at roughly one-fifth the parameter count of Nemotron 3 Ultra. The benchmarks back the compression story: 63.44% on Hermes Bench versus Ultra’s 56.33%, 87.05% on LangChain behavioral evaluations against Ultra’s 84.3%, and 80.8% on CharXiv chart reasoning (84% when the model can execute Python).
Enterprise post-training partners are already quoting the implied economics. Scale AI projects a 40x inference cost reduction and 5x latency reduction for Skydance’s vendor-credit finance agent. Ramp expects up to 50% higher throughput and 10x lower latency. AI21 reports a 40% accuracy lift for Alice’s threat-intelligence work. On 16 B200 GPUs with Dynamo serving at 50 tokens per second per user, KV-cache-aware routing delivers up to 8x more concurrent users and an 85% cut in time to first token.
The Nemotron Coalition framing (Mistral AI, Thinking Machines Lab, Perplexity, Black Forest Labs) and the AI4Bharat partnership at IIT Madras, which produced a 2.8-trillion-token dataset across 22 Indian languages with 10 billion Indic tokens added during continued pretraining, signal the real positioning. Open weights distributed through Hugging Face, OpenRouter, and ModelScope; Atlassian, CrowdStrike, Dassault Systèmes, and Salesforce listed as deployment references; Kari Briski framing the release on Nvidia’s blog. This is a bid to own the agentic default before the cloud-API incumbents price in local alternatives.
The hardware floor just moved. The next one moves Oct. 23.
Sources
- https://blogs.nvidia.com/blog/nemotron-3-5-super-vl/
- https://www.techrepublic.com/article/news-nvidia-nemotron-3-5-super-vl-bolt-ai-models/
- https://the-decoder.com/nvidias-nemotron-3-5-super-vl-is-an-open-model-built-to-run-enterprise-ai-agents-nonstop/
- https://siliconangle.com/2026/10/15/nvidia-expands-nemotron-llm-family-with-hyper-efficient-models-for-agentic-systems/
- https://www.unite.ai/nvidia-releases-open-nemotron-3-5-super-vl-and-bolt-edge-models/
— END —