NVIDIA said its DSpark feature raised Nemotron-3.5 Lightning’s average output rate on a 64GB DGX Spark, with the highest result reaching 126.9 tokens per second on HumanEval prompts.
The HumanEval rate compared with 73.4 tokens per second when DSpark was off. On SpeedBench 1K, the mean rose to 108.5 tokens per second from 73.3; on SpeedBench 2K, it reached 105.3 from 72.8.
The faster output rates came with small increases in median time to first token. That measure rose by 2.0 milliseconds on SpeedBench 1K, 2.1 milliseconds on SpeedBench 2K and 0.6 milliseconds on HumanEval prompts.
NVIDIA’s benchmark settings used NVFP4 and the model’s thinking mode. Each run had a batch size of one, a single concurrent streaming request and an output cap of 128 tokens. DSpark generated three draft tokens.
NVIDIA’s tutorial runs TensorRT Edge-LLM’s local server at 127.0.0.1:8000, where compatible clients can send requests. Preparation involves obtaining roughly 21GB of model data, after which the inference engines are built. Later launches can use a cached engine when its configuration matches. For inference runs with DSpark, memory use across the entire system reached a 27.9GB peak, operating system included.
The walkthrough uses TensorRT Edge-LLM 0.11.1 and TensorRT 10.16.1.11. NVIDIA lists CUDA 13, TensorRT 10.16.1.11 or a newer 10.x release, and Python 3.10 to 3.12 among the requirements.
