A warehouse simulation sustained 30 frames a second during a two-minute test after developers used an AI agent to build and troubleshoot it, NVIDIA said in an Oct. 29 blog.
Performance fell when developers doubled the scene again, the company said. NVIDIA, which supplies the simulation libraries used, published “Into the Omniverse: How Developers Turn Ideas Into Simulations With Frontier AI Agents” with examples spanning robotics, industrial equipment and vehicles.
The company described development projects rather than a study covering a common observation period. Developers directed agents through written instructions, reviewed their work and requested changes, using Omniverse libraries for physics, rendering and simulated sensors.
Bhavik Sanganee, a principal engineer at NVIDIA, used GPT-6 Astra to build the warehouse simulation and investigate slowdowns, the company said. Running physics calculations, image rendering and video preparation concurrently restored performance after an initial expansion pushed it below target.
The test contained 80 conveyors, 40 robot arms and 800 boxes, NVIDIA said. It ran at full HD resolution, and the arms completed 640 successful box placements. Further expansion exposed a slowdown in physics calculations and robot control.
Other examples showed limits on what simulated success established, the company said. Developers converted an ABB Robotics YuMi robot model into a simulation and configured its joints and contact surfaces. Both arms and grippers completed four cube pick-and-place cycles.
Those tests used estimated physical properties that developers had not calibrated against the real robot, NVIDIA said. The results demonstrated behavior under the simulated conditions.
A cable-handling project also required further work before physical testing, the company said. Oyindamola Omotuyi, NVIDIA’s robotics technical marketing manager, directed Astra to build a simulation and a learning system for routing a cable and inserting its USB-C connector.
The controller received information directly from the simulation, NVIDIA said. Next steps included testing a grasp held by friction and using information available from real cameras and sensors.
For sensor validation, Ashley Reid, who works on NVIDIA’s RTX sensor validation, guided Astra and Claude Fable 5 agents through about three days of comparisons with recorded data, the company said. Agents built two digital twins, virtual representations of real environments, and improved two existing ones.
The agents changed objects, geometry and materials, with acceptance tied to camera and lidar measurements, NVIDIA said. Lidar measures distance using laser light. The company said measured differences could guide improvements to simulated scenes.
In another experiment, a simulated humanoid cleared a hurdle in 64 of 100 trials, NVIDIA said. Tae Kim, who leads its Omniverse engineering and product work, directed the project using sports videos and written instructions. The trials provided feedback for improving timing and control.
