Nvidia squeezes a 74B agentic model into 24GB for your laptop

Nvidia has made its move in its catalog of open models. The company has announced new additions to its Nemotron family: on the one hand, Nemotron 3.5 Super VL, a 120 billion parameter vision-language model designed for enterprise agents; on the other, two smaller Bolt models designed to run coding, research, and conversational tasks locally. The most striking is the Nemotron 3.5 Bolt 74B-A9B, which Nvidia has managed to compress down to 24 GB of memory, allowing it to run on a laptop.

The focus of the three launches is clear: speed and efficiency. The two larger models are already available, while the 4 GB Bolt E4B, the smallest in the family, will arrive in the coming weeks.

What Nemotron 3.5 Super VL brings

The star model of the announcement is Super VL, a 120 billion parameter model that combines reasoning and tool use with understanding of images and documents. According to Kari Briski, VP of open models and software at Nvidia, the goal is to offer leading accuracy in multimodal agentic tasks —instruction following, tool use, multi-step enterprise workflows, and visual understanding— comparable to the company's largest reasoning model with roughly one-fifth of the parameters.

Super VL is built on Nemotron 3 Super, released in March. Both share the same 120 billion total parameters, with 12 billion active during inference, and context windows of up to one million tokens. The key difference is that the new model adds support for image and video input, though not for output.

Mixed results against the competition

When its benchmarks are compared with other open models of similar size, the results are uneven. In Nvidia's own evaluations, Super VL beats Alibaba's Qwen3.5-122B-A10B in scientific reasoning and terminal tasks, but falls slightly below it in SWE-bench Verified and document understanding. The five tests used are GPQA-Diamond, Terminal-Bench 2.1, SWE-bench Verified, IFBench, and MMLongBench-Doc.

On Terminal-Bench, Super VL scores 66.7 versus 48.7 for Qwen3.5, but trails in SWE-bench Verified (71.1 versus 71.9). The Qwen model has 122 billion total parameters and 10 billion active, compared with 120 billion and 12 billion for Super VL. The more recent Qwen3.8-Flash-Next scores higher on all five tests, although its additional embedding and prediction components raise the total to about 180 billion parameters. Inkling-Small, from Thinking Machines, with 276 billion total and 12 billion active, also trails in terminal tasks but beats Super VL in instruction following.

Nvidia also reports that Super VL beats Ultra on Hermes Bench and on LangChain behavior evaluations, with 63.44% versus 56.33% and 87.05% versus 84.3%, respectively. Overall, the focus is clearly on speed: Nvidia reports four times faster completion of 10,000 SWE-bench Pro tasks on four B200 GPUs, although its chart shows lower accuracy than Qwen3.8-Flash-Next and DeepSeek V4 Flash.

How it was built

The Nemotron coalition keeps the hybrid architecture of the 3.5 family, which combines Mamba-2 layers, attention, and mixture of experts, but adds a vision encoder. Nvidia continued training the language model through distillation from Nemotron 3 Ultra before additional training for software engineering, vision, and agentic tasks. To speed up text generation, it uses LiLiCorr, a speculative decoding technique in which a smaller auxiliary model drafts several tokens ahead and Super VL verifies them as a group. Nvidia claims this speeds up token generation on DGX Spark by up to 2.3 times compared with decoding without speculation.

For document workflows, Super VL focuses on optical character recognition, tables, charts, and structured information. On CharXiv, a benchmark for reasoning over charts, Nvidia reports 80.8% accuracy versus 63% for Nano Omni. Interpreting Python code raises Super VL's result to 84%.

Bolt: 74B in 24 GB

If the interest is in running models locally, the new Bolt models are probably the most relevant launch, and their benchmarks tend to beat most comparable models. The Nemotron 3.5 Bolt 74B-A9B 24GB derives from Super VL. Its hybrid mixture-of-experts architecture activates 9 billion of its 74 billion parameters at a time. In Nvidia's tests, Bolt scores 51.0 on Terminal-Bench 2.1, versus 44.4 for Qwen3.6-35B-A3B and 37.2 for Gemma4 26B-A4B, both tested in BF16. Bolt uses mixed precision, including two-bit experts.

To reach that size, Nvidia applied two-bit quantization that reduces the memory footprint of the 74B model to 24 GB. The company says it is optimizing the model for RTX Spark and DGX Spark systems, with support for llama.cpp, Ollama, LM Studio, and vLLM. A second, even smaller version, Nemotron 3.5 Bolt E4B 4GB, is planned for the end of the month. With it, Nvidia is targeting conversational and physical agents on GeForce RTX, Jetson, and Spark hardware, including device assistants and interactive characters that need multi-turn tool use and awareness of application state.

Availability and license

The new models are already available for download on Hugging Face under the OpenMDW 1.1 license, which allows commercial use and modification.

What it means for an SMB

The news has two readings for an SMB's IT team. The first is strategic: a 74B agentic model that fits in 24 GB opens the door to running coding assistants and automations directly on the developer's laptop or on a desktop machine with a GPU, without depending on an external API or sending sensitive data to the cloud. For sectors with confidentiality requirements —legal, healthcare, industry— this can be a real change in how AI pilots are approached.

The second is caution. The benchmarks are mixed, and Nvidia's own data show that speed comes at the cost of some accuracy compared with competitors such as Qwen3.8-Flash-Next. In addition, two-bit quantization is aggressive: it is advisable to validate real behavior on each business's specific tasks before integrating it into a critical workflow.

ForgeNEX recommendations

• Start with a narrow use case: test generation, summarizing technical documentation, or assistance with terminal tasks. Measure accuracy and time before expanding.
• Try it locally with Ollama or LM Studio on a machine with a sufficient GPU. It is the fastest way to evaluate without committing infrastructure.
• Review the OpenMDW 1.1 license with your legal team if you are going to distribute the model or integrate it into a product.
• Don't rule out the cloud: for one-off workloads or peaks, a managed model may still be more cost-effective than maintaining dedicated hardware.
• Strengthen control over what runs on devices. If your employees start downloading local models, you need visibility. We already saw how Cloudflare released Void to control vibe-coding apps: here is the analysis.

The trend is clear: generative AI is shifting from the data center to the desktop. For an SMB, that lowers the barrier to entry and improves privacy, but it also multiplies the points where you have to govern what runs and with what data. As with any technology adoption, the key is to test with judgment and measure before scaling.

Source: The New Stack. Analysis and adaptation: ForgeNEX.