Editor’s note: This post is part of Into the Omniverse , a series focused on how developers, 3D practitioners, and enterprises can transform their workflows using the latest advances in OpenUSD and NVIDIA Omniverse .
Vision AI agents are becoming a practical way to automatically turn video data from the physical world into operational intelligence in factories, cities, warehouses and transportation systems.
That shift is accelerating as more AI workloads move closer to where data is generated. Gartner projects that more than two-thirds of enterprise-managed data will be created and processed outside the data center or cloud by 2028, and that over two-thirds of all enterprises globally will deploy edge AI by 2029, up from 10% in 2025 (1) .
But more edge data doesn’t automatically create more intelligence. As much as 90% of existing edge data goes unprocessed, according to the same Gartner report. Turning that data into useful action requires vision AI agents that can understand video, adapt to real-world conditions and connect insights to operational workflows. These agents often run near cameras, machines and sensors, where models must meet latency, power, cost and connectivity requirements while adapting to site-specific conditions. To build those agents, developers need repeatable ways to generate training data, fine-tune models and deploy agentic video applications across edge and cloud environments.
NVIDIA Metropolis agent skills and blueprints give developers reusable workflows to build, operate and optimize vision AI agents across that lifecycle.
For the simulation and synthetic data side of that work, Universal Scene Description, or OpenUSD , provides a common framework for describing, composing and reusing 3D worlds. Built on OpenUSD, NVIDIA Omniverse libraries help teams build simulation, synthetic data generation and digital twin workflows that model real-world environments and expand scenario coverage across conditions such as lighting, weather, traffic patterns, camera angles, occlusion and rare events.
As organizations move toward autonomous vision agents, three challenges often come up:
Accuracy Plateaus With Data Gaps: Vision AI agents need to spot rare defects, abnormal events and changing environments. In manufacturing, for example, an inspection model may perform well on common scratches or dents but struggle with a new hairline crack not represented in the training data.
Lack of Fine-Tuning Expertise: Once teams identify a performance gap, improving the model is rarely a simple handoff. Fine-tuning requires labeled datasets, training configuration, experiment tracking, evaluation and decisions about whether there’s improvement for the target use case. Many organizations building vision AI agents don’t have large in-house machine learning teams to manage that process quickly, especially across many sites, products or camera views.
Complex, Time-Consuming Agent Assembly Workflows: Deploying a vision AI agent requires more than running inference. Developers have to stitch together video pipelines, AI models, metadata, embeddings, indexing, search, alerts, reporting and system integrations. Customizing that workflow for a specific environment adds significant time and requires specialized expertise. Without OpenUSD’s shared scene description layer, teams often rebuild 3D environments from scratch each time conditions or deployment sites change.
NVIDIA agent skills and blueprints — used alongside NVIDIA Omniverse for OpenUSD-based simulation and synthetic data generation, NVIDIA Metropolis for model development and video AI deployment — give developers reusable starting points for key parts of those workflows:
The Defect Image Generation skill helps create synthetic defect data.
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://blogs.nvidia.com/blog/vision-ai-agent-skills-omniverse-metropolis/#primary
- https://blogs.nvidia.com/blog/author/estherlee/
- https://blogs.nvidia.com/blog/vision-ai-agent-skills-omniverse-metropolis/#disqus_thread
- AI researchers trick chatbots into sharing how to make cocaine as long as they believe a user is wearing a green shirt — 'CoT Forgery' exploit spurs LLMs to div
- Possible AMD RX 7900 XTX engineering sample with red PCB surfaces — prototype came with no backplate & custom VBIOS but matches RX 7900 GRE specs
- NVIDIA Unlocks AI Compute at Scale, Inviting Partners to Power the AI Infrastructure Buildout
- Oomwoo is a new open-source robot vacuum you can 3D print yourself, sidesteps cloud security risks by running fully offline — project combines Raspberry Pi, 2D
- Grab this epic Razer Wolverine V3 controller for a record-low Amazon price, now just $64.99 — big 46% saving on this esports-friendly gamepad for your PC or con
Informational only. No financial advice. Do your own research.