NVIDIA Releases Nemotron 3.5 Bolt, a 74B Local Model That Fits in 24GB - 2-Bit Quantization, but Slower Than Smaller 4-Bit MoE Models

On October 15, 2026, NVIDIA released the open model Nemotron 3.5 Super VL (120B) and Bolt 74B-A9B 24GB for local use. Bolt fits in 24GB of memory through 2-bit quantization and can be used commercially. In NVIDIA's own measurements on the same RTX Spark machine, it generates more slowly than Qwen3.6 35B-A3B and Gemma 4 26B-A4B at 4 bits.

NVIDIA Nemotron open-weight local-LLM quantization RTX Spark

On October 15, 2026, NVIDIA announced the open model Nemotron 3.5 Super VL along with two models for local use, Nemotron 3.5 Bolt 74B-A9B 24GB and Bolt E4B 4GB¹. Super VL and Bolt 74B-A9B can be downloaded now, and Bolt E4B is scheduled for October 23. NVIDIA describes Super VL as a 120B-parameter model for enterprises, and Bolt 74B-A9B as a local model that, through what it calls “native 2-bit quantization,” needs only 24GB of memory and can run next to other apps.

For anyone running LLMs on their own hardware, the story is that a 70B-class model now fits in 24GB, and the question is what that costs. Our guide to choosing hardware for local LLMs uses the widely adopted 4-bit (Q4) format as its baseline and lists file sizes of 43-47GB and recommended memory of 48-64GB or more for the 70-72B class. Bolt gets the model itself down to 24GB by going to 2 bits. Even so, the speed comparison NVIDIA itself published shows Bolt generating more slowly than Qwen3.6 35B-A3B and Gemma 4 26B-A4B at 4 bits on the same machine³.

Bolt 74B-A9B: A Parent Model Compressed, Then Taken Down to 2 Bits

According to the model card, Bolt 74B-A9B has 74B total parameters and 9B active parameters, and is a Mixture of Experts (MoE) design that combines Mamba-2 and Transformer layers². It takes text, images and video as input, supports a context length of up to 256K tokens, and Japanese is among its supported languages. The license is OpenMDW-1.1, and the model can be used for both commercial and non-commercial purposes.

NVIDIA built it by compressing the 120B Super VL (12B active) down to 74B, then quantizing most of the weights to 2 bits². The technical blog sets two design targets, fitting in 24GB and decoding at no less than 80 TPS, and concludes that within a fixed memory budget, compressing a large model works better than training a small one from scratch³. NVIDIA says it quantized only the weights because inference on RTX Spark and DGX Spark is mostly limited by weight bandwidth. The memory bandwidth we covered in our comparison of Apple silicon and NVIDIA GPU memory bandwidth is again named here as a factor in speed.

For how to read figures like “74B” and “9B,” see What Are LLM Parameters?. The basics of quantization are covered in Model Distillation and Quantization Explained.

Higher Scores on Many Tests, but Slower Than Smaller 4-Bit MoE Models

The benchmark table on the model card puts Bolt (2-bit) next to Qwen3.6-35B-A3B and Gemma4 26B-A4B, both in BF16². On Terminal-Bench 2.1, for example, Bolt scores 50.98 against 44.38 for Qwen3.6 and 37.22 for Gemma4. On PinchBench, however, Qwen3.6 edges out Bolt, 88.07 to 87.82, and on AA-Omniscience accuracy Gemma4 scores 22.17 to Bolt’s 16.90. Bolt does not come out ahead on every test.

For speed, the technical blog publishes results measured with llama.cpp on a single RTX Spark machine (Surface Laptop Ultra, 128GB)³.

Model (quantization)	Generation speed (TPS)	Agent task replay time (s)
Nemotron 3.5 Bolt 74B-A9B (2-bit)	83.99	600.80
Qwen3.6 35B-A3B (4-bit)	96.63	453.97
Gemma 4 26B-A4B (4-bit)	100.56	480.68
Qwen3.8 27B (4-bit)	28.26	1043.28

Generation speed is from AIPerf SPEED-Bench-Coding and replay time is from AA AgentPerf Local; a shorter replay time is better. Bolt is far quicker than Qwen3.8 27B, but it trails Qwen3.6 35B-A3B and Gemma 4 26B-A4B on both generation speed and the time to finish a full sequence of tasks. NVIDIA positions Bolt as an option that balances accuracy and speed. Which matters more for a given workload, the accuracy gap or the waiting time, is likely to decide the choice.

Where It Runs, and What to Check First

NVIDIA says Bolt 74B-A9B is available on Hugging Face and works with vLLM, llama.cpp, Ollama, LM Studio and Unsloth¹. It was built for laptops with RTX Spark, and the company says RTX PRO workstations, DGX Station, DGX Spark and GeForce RTX PCs can run it as well. The technical blog says the quantized model is optimized for llama.cpp on RTX Spark and for vLLM on DGX Spark³.

The software integration section of the model card, however, lists vLLM as the runtime, Blackwell-generation hardware (SM120 and SM121) as supported, and Linux as the operating system². NVIDIA’s materials do not confirm whether the model runs with llama.cpp on older GeForce generations. None of the three documents, the announcement blog, the model card or the technical blog, mention Apple silicon. Before trying it on existing hardware, check support on the side of the tool you plan to use.

Super VL: A 120B Model Meant to Be Post-Trained on Company Data

Super VL, released at the same time, is a 120B-parameter model. NVIDIA says it reaches accuracy comparable to its biggest reasoning model while using about a fifth as many parameters¹. NVIDIA also claims that, on the Artificial Analysis Intelligence Index leaderboard, it delivers up to 5x the throughput of comparable models. NVIDIA has released the weights, datasets and training recipes, and the model is available on Hugging Face, OpenRouter, ModelScope and NVIDIA NIM.

NVIDIA lists Atlassian, CrowdStrike, Perplexity and Salesforce among the companies building Super VL into their products and operations¹. It promotes the ability to post-train the model on an organization’s own data and host it on its own infrastructure as an advantage over closed frontier models.

With Super VL and Bolt, the Nemotron family now spans Lightning for high-throughput work, Ultra for the most difficult reasoning work, and these two new models¹. For readers weighing local setups, there is now a 70B-class option that fits in 24GB, and NVIDIA’s own numbers make it possible to compare its speed against smaller 4-bit MoE models that fit in the same memory.