NVIDIA Nemotron 3.5 Lightning 30B: A 3B-Active Model Built for Long-Running AI Agents

The next generation of AI systems is increasingly being designed around agents rather than simple chatbots.

Instead of answering one prompt and stopping, an agent may need to inspect files, call tools, search through information, write code, execute commands, reason through intermediate steps, and continue working for minutes or even hours.

That is the environment NVIDIA is targeting with Nemotron 3.5 Lightning 30B-A3B-NVFP4.

The model combines 30 billion total parameters with only 3 billion active parameters per token, using a hybrid architecture built from Mamba-2, Mixture-of-Experts layers, and selected attention layers. NVIDIA also gives it a massive context window of up to 1 million tokens and supports several speculative decoding approaches intended to make inference faster.

Rather than competing purely on chatbot benchmarks, Nemotron 3.5 Lightning is positioned as a workhorse for autonomous agents, coding systems, RAG applications, sub-agents, and other AI workflows that need to perform many steps efficiently.

View the model on Hugging Face

30 Billion Parameters, But Only 3 Billion Are Active

One of the most interesting aspects of Nemotron 3.5 Lightning is the difference between its total parameter count and its active parameter count.

The model contains 30 billion parameters in total, but only around 3 billion parameters are active for a given token.

That is possible because the architecture uses a Mixture-of-Experts, or MoE, design.

Instead of sending every token through every parameter in the network, the model can route computation through selected expert components. This allows the system to have a much larger overall parameter pool without paying the full computational cost of a dense 30B model on every token.

This distinction matters when thinking about performance.

A model advertised as “30B” can sound extremely expensive to run, but a 3B-active architecture can make the practical compute profile very different from a conventional dense 30B model.

That is one of the reasons Nemotron 3.5 Lightning is particularly interesting for agentic workloads where the model may need to generate large amounts of text over extended sessions.

A Hybrid Architecture Instead of a Conventional Transformer

Nemotron 3.5 Lightning is not simply another Transformer with a larger context window.

NVIDIA describes its architecture as a hybrid combination of Mamba-2, Mixture-of-Experts, and Attention layers.

The Mamba component is designed around efficient sequence processing, while attention remains valuable for tasks that benefit from direct token-to-token interactions. The MoE architecture adds conditional computation on top of that.

The result is an architecture built around a different question:

How can a model process extremely long sequences and repeated agent interactions without requiring every part of the network to perform the same amount of work?

That question becomes increasingly important as AI applications move beyond short conversations.

An ordinary chatbot might generate a few hundred tokens.

An autonomous coding agent can generate plans, inspect files, execute commands, analyze errors, modify source code, run tests, and repeat the process many times.

The architecture therefore matters just as much as the raw parameter count.

The 1 Million Token Context Window Changes the Game

Nemotron 3.5 Lightning supports a context length of up to 1 million tokens.

That is an enormous amount of information for a single model session.

For ordinary chat, users may never need anything close to that much context. For agents, however, long context can become extremely useful.

Imagine an AI coding assistant that needs to understand an entire software repository.

Instead of repeatedly throwing away older information and retrieving small sections of the project, a long-context system can potentially keep much more of the relevant repository, documentation, logs, tool outputs, and previous reasoning inside a single context.

The same idea applies to large research projects.

An agent could process a long collection of technical documents, maintain extensive tool traces, inspect generated reports, and keep more information available while working through a problem.

NVIDIA’s model card validates the 1M-token context on several deployment configurations, including H100 and GB200 setups, while also providing local-oriented deployment paths. :contentReference[oaicite:1]{index=1}

This Model Is Built Around Agents

The intended use case is one of the clearest differences between Nemotron 3.5 Lightning and many general-purpose language models.

NVIDIA specifically positions it for long-running autonomous agents, sub-agent workloads, and efficient inference on personal hardware.

That means the model makes sense in systems where the language model is only one component of a larger loop.

For example:

User request → planning → tool call → observation → reasoning → another tool call → code modification → test execution → error analysis → final response.

An agent might perform dozens of these interactions during a single task.

A model optimized for that environment needs more than good conversational ability. It needs reliable instruction following, coding performance, tool-use capabilities, and efficient generation.

That is where Nemotron 3.5 Lightning is trying to compete.

Strong Coding and Agentic Results

NVIDIA reports a number of benchmark results for the official NVFP4 checkpoint.

On SWE-bench Verified, the NVFP4 model records a score of 52.80 in NVIDIA’s evaluation setup.

It scores 23.46 on Terminal-Bench 2.1, 83.43 on PinchBench, and 36.81 on BrowseComp.

For general and reasoning evaluations, NVIDIA reports 81.62 on MMLU Pro, 75.57 on GPQA Diamond, and 31.38 on SciCode.

The model also records 72.88 on IFBench, which measures instruction-following behavior, while NVIDIA reports 49.19 on AA-LCR for its long-context evaluation.

These numbers are NVIDIA’s own measurements under its published evaluation harnesses, so they should be treated as reference results rather than universal rankings across every benchmark setup. :contentReference[oaicite:2]{index=2}

Still, the results show why NVIDIA is emphasizing agentic workloads rather than presenting Nemotron 3.5 Lightning as simply another chatbot.

NVFP4 Is a Major Part of the Story

The model you are looking at is specifically the NVFP4 version.

That is important because NVIDIA has designed the checkpoint around low-precision inference and modern GPU hardware.

The model is optimized for NVIDIA hardware including Blackwell systems such as GB200 and DGX Spark, as well as Hopper GPUs such as the H100 and H200. NVIDIA also documents W4A16 deployment paths for Ampere hardware such as the A100. :contentReference[oaicite:3]{index=3}

This means the NVFP4 checkpoint is not simply a smaller downloadable copy of the model.

Its precision format is part of the strategy for making a large MoE model practical to serve efficiently.

For developers operating AI infrastructure, lower precision can mean lower memory requirements and more efficient computation, although actual performance depends heavily on the GPU, runtime, concurrency, context length, and serving configuration.

Speculative Decoding Makes Lightning More Than a Name

The “Lightning” branding is not just marketing.

NVIDIA ships Nemotron 3.5 Lightning alongside several speculative decoding approaches designed to accelerate generation.

These include DSpark, DFlash, and Multi-Token Prediction (MTP).

DSpark is particularly interesting for DGX Spark and lower-concurrency, latency-sensitive workloads. DFlash uses a lightweight block-diffusion approach for generating draft token blocks, while MTP trains the network to predict several future tokens rather than just the next token.

The basic idea behind speculative decoding is straightforward.

Instead of having the main model generate every token completely sequentially, a smaller or specialized mechanism proposes multiple tokens ahead of time. The main model can then verify those predictions.

When the predictions are accepted, generation can move forward more quickly.

For an agent that generates thousands of tokens during a long session, even moderate improvements in generation efficiency can have a major impact on the total user experience.

NVIDIA explicitly recommends DSpark for DGX Spark and latency-sensitive low-concurrency serving, while no speculative decoding is recommended for maximum-throughput batch workloads on H100 systems. :contentReference[oaicite:4]{index=4}

Long-Running Agents Need More Than a Big Context Window

It would be easy to look at the 1M-token context window and conclude that Nemotron 3.5 Lightning is simply a “long-context model.”

That misses the bigger picture.

The interesting combination is:

long context + MoE + Mamba + tool use + coding + speculative decoding + agent-oriented training and evaluation.

Each component solves a different part of the problem.

Long context helps the model retain more information.

MoE reduces the amount of active computation relative to a dense model with the same total parameter count.

Mamba contributes another sequence-processing mechanism.

Tool integration allows the model to interact with external systems.

Coding benchmarks measure its ability to work with real software tasks.

Speculative decoding targets the latency problem.

Together, these features make the model much more suitable for applications where the AI is expected to keep working rather than simply produce one response.

A Strong Candidate for AI Coding Systems

Nemotron 3.5 Lightning also has an obvious place in AI coding.

Software agents increasingly need to do more than autocomplete individual lines.

They need to understand repositories, inspect dependencies, modify multiple files, run tests, interpret compiler output, reason about failures, and repeat the process.

A 1M-token context window can be especially useful for this kind of work because large repositories can produce enormous amounts of context.

The model also supports tool-call oriented serving configurations. NVIDIA’s deployment examples include tool-call parsing and automatic tool choice, which are important capabilities for modern coding agents.

That makes the model a potential foundation for systems that operate more like autonomous software engineers than traditional coding assistants. :contentReference[oaicite:5]{index=5}

RAG Applications Can Also Benefit

Retrieval-augmented generation is another natural use case.

A standard RAG system retrieves several relevant documents and gives them to the language model.

That works well for small questions, but large enterprise workloads can involve enormous collections of information.

The combination of a long context window and agentic behavior means Nemotron 3.5 Lightning can be used in more complex retrieval pipelines.

For example, an agent might:

Retrieve documents.

Compare multiple sources.

Identify missing information.

Run another search.

Inspect structured data.

Call an external tool.

Re-evaluate the answer.

Then produce a final response with supporting evidence.

This is fundamentally different from a simple “search five chunks and answer” RAG pipeline.

The model can become the reasoning component inside a larger information-processing system.

Running It Locally Is Possible, But Hardware Matters

The phrase “local AI” needs some qualification here.

Nemotron 3.5 Lightning is not a tiny model designed for ordinary CPUs or entry-level laptop GPUs.

The official hardware recommendations target NVIDIA GPUs, with specific validation for systems including DGX Spark, H100, H200, GB200, and other NVIDIA platforms.

The model card also documents local deployment options through tools such as llama.cpp, Ollama, LM Studio, and Jan, while noting that usable context length can vary based on available VRAM. :contentReference[oaicite:6]{index=6}

So this is better described as practical local or private deployment on sufficiently powerful NVIDIA hardware, rather than a lightweight model that can run comfortably on any consumer laptop.

That distinction is important for developers deciding what hardware to buy.

Transformers, vLLM and SGLang Support

Developers already using modern open-source inference frameworks will find a familiar ecosystem around the model.

The Hugging Face checkpoint includes instructions for Transformers, while NVIDIA provides deployment recipes for vLLM, SGLang, and TensorRT-LLM.

For example, the Hugging Face model card provides a straightforward Transformers workflow and vLLM serving instructions.

This means developers can move from experimentation to API-style serving without having to build an inference stack completely from scratch. :contentReference[oaicite:7]{index=7}

That is particularly useful for teams building internal AI systems because the same model can fit into different deployment environments depending on throughput and latency requirements.

Six Languages Plus Coding Languages

Nemotron 3.5 Lightning is primarily positioned around English and programming languages, but NVIDIA also lists support for Spanish, French, German, Italian, and Japanese.

That makes the model more versatile for multilingual agent applications than an English-only coding model.

For companies building global AI products, this opens the possibility of combining software-engineering capabilities with multilingual user interactions.

It also makes the model more interesting for international development teams where the engineering workflow may involve both English-heavy code and other human languages. :contentReference[oaicite:8]{index=8}

More Than 20 Trillion Training Tokens

The scale of the training process is another major part of the model.

NVIDIA states that Nemotron 3.5 Lightning was pretrained using more than 20 trillion tokens and incorporated code, mathematics, science, and general knowledge data.

The model also uses Multi-Token Prediction as part of its design.

MTP is interesting because the model is trained to predict multiple future tokens, providing additional training signals beyond the standard next-token objective.

This is especially relevant when the final goal is fast generation because the training strategy itself is connected to the inference strategy.

NVIDIA describes the pretraining pipeline as using an NVFP4 recipe and the Megatron-LM training stack. :contentReference[oaicite:9]{index=9}

The Model Is Not for Every Use Case

Despite all of these capabilities, Nemotron 3.5 Lightning is not automatically the best model for every AI application.

Its hardware requirements are substantial compared with truly lightweight models.

The NVFP4 format is especially relevant to NVIDIA-oriented deployment, meaning hardware compatibility and inference-stack configuration should be considered before choosing it.

The 1M context window is also not free.

As context grows, memory consumption for KV caches and the overall serving requirements can become much larger, especially when serving multiple concurrent users.

NVIDIA explicitly notes that reducing the maximum model length can provide more KV-cache headroom when concurrency becomes a concern. :contentReference[oaicite:10]{index=10}

So the model’s headline specifications should not be interpreted as meaning that every workload should run at a full million tokens.

Commercial Use and Licensing

The model is released under the OpenMDW License Agreement, version 1.1.

NVIDIA states that the model is ready for commercial use, but developers and organizations should still review the specific license terms before deploying it in a commercial product.

This is especially important for companies building hosted AI services, enterprise agents, or products that redistribute model weights.

The Hugging Face model card and NVIDIA documentation provide the governing license information and should be treated as the authoritative source for deployment decisions. :contentReference[oaicite:11]{index=11}

Where Nemotron 3.5 Lightning Makes the Most Sense

Nemotron 3.5 Lightning is especially interesting when an application needs the model to continuously interact with tools and information.

For example, you could build a software engineering agent that reads a repository, plans a change, edits source files, runs tests, reads logs, and fixes the resulting errors.

You could build a research agent that processes a large collection of documents while maintaining a very large working context.

You could build a private enterprise assistant that combines RAG, internal APIs, databases, and workflow automation.

You could build a sub-agent that handles one specialized task inside a larger multi-agent architecture.

You could even use it as a general reasoning engine behind a chatbot, provided the hardware and serving requirements fit the application.

The common theme is simple:

The model becomes more interesting as the number of steps in the workflow increases.

Five Practical Things Developers Can Build

1. Autonomous coding agents: Give an agent access to repositories, terminals, test runners, Git, and documentation so it can work through software tasks with multiple iterations.

2. Long-context research agents: Keep large amounts of documents, notes, logs, and retrieved information inside a single working context.

3. Private enterprise assistants: Combine the model with internal databases, APIs, RAG systems, and business tools without relying entirely on an external model API.

4. Multi-agent systems: Use Nemotron as a strong sub-agent for coding, research, planning, or tool execution inside a larger orchestration system.

5. High-throughput AI services: Deploy the NVFP4 checkpoint with NVIDIA-focused inference stacks such as vLLM or TensorRT-LLM and tune the serving configuration around the available hardware.

Why Nemotron 3.5 Lightning Is Important

The biggest story here is not that NVIDIA created another 30B language model.

It is the way NVIDIA is combining several ideas into a single system.

A 30B total / 3B active MoE model.

A hybrid Mamba-2 and attention architecture.

A 1M-token context window.

NVFP4 quantization.

Multi-Token Prediction.

Multiple speculative decoding strategies.

Strong coding and agentic benchmark performance.

And deployment options targeted at both data-center and local NVIDIA hardware.

That combination reflects where the AI industry is heading.

The next important AI systems will not necessarily be models that simply answer questions better.

They will be systems that can keep working.

They will inspect information, call tools, write code, execute tasks, remember context, recover from errors, and continue until the job is finished.

Nemotron 3.5 Lightning is clearly designed around that future.

Final Verdict

NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4 is one of the more interesting open-weight models for developers building serious AI agents in 2026.

Its 30B total / 3B active architecture gives it a compelling efficiency story.

Its 1M-token context window makes it highly relevant to long-running workflows.

Its coding and agentic benchmark results show that it is not merely optimized for conversation.

And its combination with speculative decoding and NVIDIA’s modern inference stack makes the model particularly attractive for production-oriented deployments.

It is not a tiny model and it is not a drop-in replacement for every LLM.

But for developers building coding agents, autonomous workflows, RAG systems, private AI infrastructure, and long-context applications, Nemotron 3.5 Lightning is a model worth paying attention to.

The larger trend may be even more important.

AI models are moving from systems that simply generate answers toward systems that can actually do work.

Nemotron 3.5 Lightning is built for exactly that transition.