Video Textures in Kit 110.4 and a Standalone ovrtx Renderer

An .mp4 is now just another texture. Here is how to use it in Omniverse Kit 110.4 and in a headless ovrtx render service, using one small demo scene.

A digital twin of a factory has screens on it: HMI panels, andon boards, a camera feed on the wall of the control room. Until now, putting moving video on those surfaces in Omniverse meant writing custom code. You swapped image files every frame, or built a dynamic texture provider by hand.

With Kit 110.4 and the RTX renderer behind it, that is no longer needed. You point a material’s texture input at an .mp4 file, the same way you would point it at a .png. The renderer decodes the video and shows the current frame on the surface.

This article walks through one demo scene, video_studio.usda, in two places:

1. Kit 110.4. Open the scene, press play, and watch the video on the surfaces.
2. Standalone ovrtx. Render the same scene headless from Python, to a PNG, GIF or MP4, with no Kit app running. This is how we run it at SyncTwin, in a FastAPI render service.

The second path has a few traps that cost us a day. On Windows the renderer can hang, and video can drift out of sync with the animation. Both are covered below, with the fixes.

The big picture

The scene file is the same in both renderers. Kit handles the video for you. A standalone ovrtx renderer needs two extra steps: the decoder plugins on Windows, and a renderer clock that you drive yourself.

Part 1: Video textures in Kit 110.4

The demo video

The demo clip is textures/test_video.mp4: 10 seconds, 1280 × 720, 30 fps, about 4 MB. It shows a running timecode in one corner. That makes it easy to see which video frame is on screen at any moment, which helps a lot when you check sync later.

Any H.264 .mp4 works the same way. A short clip with visible timing is a better first test than a real camera feed.

The material: an .mp4 where an image would go

The scene uses a standard OmniPBR material. The only new thing is the file extension on the texture inputs:

def Material "VideoMaterial"
{
    token outputs:mdl:surface.connect = </World/Looks/VideoMaterial/Shader.outputs:out>     def Shader "Shader"
    {
        uniform token info:implementationSource = "sourceAsset"
        uniform asset info:mdl:sourceAsset = @OmniPBR.mdl@
        uniform token info:mdl:sourceAsset:subIdentifier = "OmniPBR"
        # Kit 110.4: a video file, referenced like any image texture
        asset inputs:diffuse_texture = @./textures/test_video.mp4@ (
            colorSpace = "auto"
        )
        # Emissive, so the screen reads like a display even in a dark scene
        bool inputs:enable_emission = 1
        asset inputs:emissive_color_texture = @./textures/test_video.mp4@ (
            colorSpace = "auto"
        )
        float inputs:emissive_intensity = 1000
        token outputs:out
    }
}

The video goes into both the diffuse and the emissive slot. Diffuse alone looks like a printed poster: it is only as bright as the lights hitting it. With emission on, the screen glows like a real display. It also lights up the dark studio around it a little.

The geometry: meshes need UVs

The scene has two surfaces with this material:

• /World/Screen: a flat, double-sided 16:9 mesh. Its UVs match the 1280 × 720 video, so nothing is stretched.
• /World/Sphere: a UV sphere, to show the video wrapped around a curved surface.

The sphere is a Mesh, not a Sphere prim. An implicit UsdGeom.Sphere has no primvars:st, so a texture has nothing to map onto. This is true of any texture, but it is easy to forget when you want a quick test object.

The rest of the scene is a photo studio: a dark, slightly glossy cyclorama, a warm key light, a cool rim light, and a camera at /World/Studio/Camera that frames both surfaces.

Time: match the timeline to the clip

The stage metadata sets the timeline to the length of the video:

(
    startTimeCode = 0
    endTimeCode = 240
    timeCodesPerSecond = 24
    framesPerSecond = 30
)

240 time codes at 24 per second is 10 seconds, which is the length of the clip. So when the timeline loops, the video loops with it. A render of the whole range also covers the whole clip.

Open it in Kit

Open video_studio.usda in a Kit 110.4 app with the RTX viewport, such as USD Composer or your own app built on the 110.4 templates. Switch to /World/Studio/Camera and press Play. The screen and the sphere show the video, and the timecode in the corner counts up.

You can also author it from the UI. Select the OmniPBR shader of any material, and in the Property panel set Albedo Map (and Emissive Color Map, if you want a glowing display) to the .mp4 file.

Note: video textures need Kit 110.4. If you build your own app from the Kit templates, check that its kit-kernel dependency is on 110.4 or later.

Part 2: The same scene in a standalone ovrtx renderer

ovrtx is NVIDIA’s RTX renderer as a Python package, without Kit. You give it a USD stage and a render product, call step(), and read back the pixels. We use it in a small FastAPI service that renders previews and animations of SyncTwin scenes on a GPU server. The versions below are ovrtx==0.6.0 and ovstage==0.3.0.

The good news: the USD file does not change. The same video_studio.usda that plays in Kit renders in ovrtx, video and all.

Step 1: Install ovrtx from NVIDIA’s index

The ovrtx package on PyPI is only a stub. The real wheels come from NVIDIA's index:

pip install --extra-index-url https://pypi.nvidia.com ovrtx==0.6.0.1791947691 ovstage==0.3.0.1791986420

RTX needs Vulkan, not just CUDA. In Docker, give the container the graphics GPU capability as well as compute.

Step 2 (Windows only): Give ovrtx its GStreamer plugins

This is the step that cost us the most time. ovrtx decodes video through GStreamer, and it loads the plugins from one place only, inside its own package:

<site-packages>/ovrtx/bin/plugins/gpu.foundation/gstreamer-1.0/

The Linux wheels ship that folder. The Windows wheel does not. And without it, ovrtx does not report an error. The first step() on a scene with a video texture simply blocks forever, and takes the whole process with it.

Pointing GStreamer somewhere else does not help. GST_PLUGIN_PATH finds the plugins, but Windows then refuses to load their DLL dependencies, because ovrtx restricts the DLL search path. Adding the folder to PATH does not change that.

What works is copying the folder into the installed package. Kit already has a matching set: the omni.gpu_foundation extension carries it at bin/deps/gstreamer-1.0. In a Kit build you find it here:

_build/windows-x86_64/release/extscache/omni.gpu_foundation-*/bin/deps/gstreamer-1.0

We keep a copy in our repo (with Git LFS) and copy it into ovrtx when the service starts, before the renderer is created. Reinstalling ovrtx removes the folder, so doing it on every start is simpler than remembering:

import shutil
from pathlib import Path
import ovrtx
def ensure_gstreamer_plugins(vendored: Path) -> None:
    target = Path(ovrtx.__file__).parent / "bin" / "plugins" / "gpu.foundation" / "gstreamer-1.0"
    if target.is_dir():
        return
    # Copy beside the target, then rename, so a copy cut short
    # never looks like a complete install on the next start.
    partial = target.with_name("gstreamer-1.0.partial")
    shutil.rmtree(partial, ignore_errors=True)
    shutil.copytree(vendored, partial)
    partial.rename(target)

ensure_gstreamer_plugins(Path("third_party/gstreamer/windows-x86_64/gstreamer-1.0"))
renderer = ovrtx.Renderer()  # only after the plugins are in place

Two more Windows details:

• Path length. Windows will not load a plugin whose full path is longer than 260 characters. GStreamer reports this only in its own debug log, so to ovrtx the plugin is just missing, and the render hangs again. Keep your virtualenv at a short path.
• Licences. The folder contains LGPL GStreamer plugins and some NVIDIA proprietary DLLs (DeepStream, CUDA NPP) that come with Kit. Check the Omniverse licence before you ship them outside your company.

Step 3: Render a still

ovrtx renders a RenderProduct that is part of the stage. The pattern that works reliably is to compose a small root layer that sublayers your scene and adds the product, bound to the scene's camera:

from pathlib import Path
import numpy as np
import ovrtx
import ovstage
from PIL import Image
# A layer opened from a string has no folder of its own, so the sublayer
# path must be absolute (forward slashes on Windows too).
scene = Path("video_studio.usda").resolve().as_posix()
LAYER = f'''#usda 1.0
(
    subLayers = [@{scene}@]
)

def Scope "Render"
{{
    def RenderProduct "Product"
    {{
        rel camera = </World/Studio/Camera>
        int2 resolution = (1280, 720)
        rel orderedVars = <LdrColor>
        def RenderVar "LdrColor"
        {{
            string sourceName = "LdrColor"
        }}
    }}
}}
'''
renderer = ovrtx.Renderer()
stage = ovstage.Stage(name="demo")
renderer.attach_ovstage(stage)
ordinal = 1
ovstage.population.open_usd_from_string(stage, LAYER, ordinal=ordinal)
stage.advance_write_floor(ordinal).wait()  
# seal the publication
products = {"/Render/Product"}
for _ in range(40):  # warmup: shaders, texture streaming, convergence
    renderer.step(render_products=products, delta_time=1 / 60, ordinal=ordinal)
result = renderer.step(render_products=products, delta_time=1 / 60, ordinal=ordinal)
frame = result["/Render/Product"].frames[0]
var = frame.render_vars["/Render/Product/LdrColor"].map(device=ovrtx.Device.CPU)
Image.fromarray(np.array(np.from_dlpack(var))).save("still.png")

Some things to know:

• Seal every publication. Each change to the stage goes in under a new ordinal, and you must call advance_write_floor(ordinal) before you step with it. In ovstage 0.3 that call is asynchronous, so .wait() on it. Otherwise step() fails with "ordinal N exceeds ovstage write floor".
• Warm up. The first frames are noisy and textures are still streaming in. Around 40 steps gave us clean stills.
• Copy the pixels. The mapped buffer belongs to the renderer. Copy it with np.array(...) if you keep it past the next step, as you will in an animation.
• Render var keys. In ovrtx 0.5 and later, render_vars is keyed by the full prim path, such as /Render/Product/LdrColor. In 0.4 it was the bare name LdrColor.

Step 4: Render the animation to GIF or MP4

A still shows one video frame. To see the video play, you render a series of frames and encode them. For each output frame you move the stage to that time, warm up a few steps, and capture:

seconds = frame_index / fps
ordinal += 1
ovstage.population.update_from_usd_time(stage, ordinal, seconds)
stage.advance_write_floor(ordinal).wait()
# ...then step and capture as before

That is enough for time-sampled USD attributes, such as a cube moving along a path. For video it is not enough. Our first animated renders of the studio scene showed the video running ahead of the timeline. It was far ahead, and further with every frame.

Video plays on the renderer’s clock, not on USD time

The reason took a while to find. update_from_usd_time moves the USD stage. But a video texture does not read USD time. It plays on ovrtx's own simulation clock, and every step(delta_time=...) moves that clock forward.

So with 8 warmup steps per frame at delta_time=1/60, each output frame moves the video forward by 8/60 s, about 0.13 s. At 30 fps output, the frames are only 1/30 s (0.033 s) apart. The video runs four times too fast.

The fix is to drive the renderer clock yourself:

1. Before the first frame, call renderer.reset(t0). This puts the clock at the first frame's time.
2. On the first warmup step of each later frame, pass delta_time equal to the gap to that frame's time.
3. On every other step, warmup and capture alike, pass delta_time=0. The clock stands still while the image converges.

def iter_frames(renderer, stage, times, products, warmup=40, frame_warmup=8):
    clock = None
    for index, seconds in enumerate(times):
        ordinal = stage.set_time(seconds)        # update_from_usd_time + seal
        if clock is None:
            clock = max(seconds, 0.0)            # the clock cannot go below zero
            renderer.reset(clock)
        advance = max(seconds - clock, 0.0)
        clock += advance
        steps = warmup if index == 0 else frame_warmup
        deltas = [advance] + [0.0] * steps       # move once, then hold still
        for delta in deltas:
            result = renderer.step(render_products=products, delta_time=delta, ordinal=ordinal)
        yield read_ldr_color(result)             # copy the pixels, as in step 3

With this, the renderer clock and the USD time stay in step. The timecode in the demo video matches the timeline in every frame.

We test this with the demo clip. The test renders the studio scene at 1 s and 2 s, crops the screen from each frame, and compares it with every frame of the source .mp4. The closest match must be within 4 frames of 30 and 60. The decoder trails by a frame or so, so an exact match is too strict.

Encode the frames

For MP4 we pipe the RGB frames into imageio-ffmpeg (H.264, yuv420p, so it plays everywhere). For GIF we use Pillow. A GIF keeps every frame in memory until it is written, about one byte per pixel per frame, so keep GIFs short and small.

In our service, both come out of one command:

uv run render-cli --usd tests/data/video_studio.usda \
    --camera /World/Studio/Camera --width 640 --height 360 --fps 30 -o studio.mp4

With no --start or --end, it renders the scene's whole time range, from startTimeCode to endTimeCode. For the studio scene that is the full 10-second clip. In Git Bash, prefix the command with MSYS_NO_PATHCONV=1, so the /World/... camera path is not rewritten into a Windows path.

Troubleshooting checklist

Wrap-up

Video textures are a small change in USD, with a big effect on how real a digital twin feels. In Kit 110.4 it really is a single change: put an .mp4 where the image goes. In a standalone ovrtx renderer the USD stays the same. You add two things on your side: the GStreamer plugins on Windows, and a renderer clock that follows your animation.

The demo scene is a good test bed for both. Its timecode overlay shows sync problems at a glance, and its timeline matches the clip, so a render with no time range covers the whole video.

At SyncTwin we use this to show live-looking machine HMIs and camera feeds in rendered previews of factory layouts, from a render service with no Kit app running. If you try it, I’d like to hear what you put on your screens.

Here are source code and sample files for this article: https://github.com/perfectproducts/openusd_studierstube/tree/main/examples/video_textures

Recent frontier models from OpenAI and Anthropic have made significant advances in 3D modeling. I put both to the test using the same prompts and reference data to see how their agentic 3D modeling capabilities compare.

The model: ABENE VHF-680 CNC Milling Machine

I’ve provided an image and a PDF document with the technical description of CNC milling machine ABENE VHF-680.

This machine presents a challenging test case due to its intricate geometry, including transparent components and protruding elements like the HMI.

The Setup

GPT-6 Astra was run through ChatGPT using the Computer Use plugin to control Blender 5.2, while Claude Opus 5.5 was run through Claude Code using the Blender MCP Addon . Both also used direct python scripting.

First shot

Initial Prompt:

Use Blender to rebuild this machine (image attached), here is more info on the machine : https://www.yumpu.com/de/document/view/1352354/abene-vhf-680-cnc .

Both models had an initial shot, it took a few minutes to build the model in Blender.

The initial result looked like this:

Second Shot

The quality didn’t match my expectations so I gave it another prompt, which both took about 15 minutes:

Refinement prompt:

try again, try to be more accurate, also use pbr materials, we want to use it in a high end visualization

Both results are quite nice, although its interesting how details have been modelled differently — it seems that the models have used different aspects of reference data to model the details — GPT seems to have used the images from the PDF document while Opus seems to have focused on the reference image.

Conclusion

Both models successfully reconstructed the machine in 3D using Blender and were able to further improve their results based on the second prompt.

Compared with diffusion-based GenAI approaches, this agentic workflow has an important advantage: the resulting 3D models are structured in a meaningful way, with individual components such as the housing, control unit, and drive represented as separate, editable elements. This makes it possible to iteratively refine selected parts of the model rather than having to regenerate the entire asset.

At this point, however, we have a 3D model rather than a true digital twin. To get there, the models would also need to reconstruct the machine’s functional mechanisms and joints — for example, the movement of the milling axes and the opening and closing of the doors.

Given the current rate of improvement in frontier models, this next step may not be that far away. Exciting times!

We recently looked into agent-based workflows for digital twins in our OpenUSD study group, the OpenUSD Studierstube.
The challenge: take a vanilla JT file of a robot and see whether AI agents can turn it into a SimReady model, with all physics descriptions and joints.

Turns out it actually worked, and here is the step-by-step description. Note that the starting point was plain geometry: a flat list of part groups, not even a hierarchical model of the robot’s kinematic chain.

This guide takes a JT file of a KUKA KR 270 R2700 ultra to a SimReady OpenUSD asset: PBR materials, colliders, mass and physics materials, six driven joints, validated and rendered. An AI coding agent (Claude Code) drives NVIDIA’s omniverse-cad-to-simready skill; you follow the steps below.

SimReady means an OpenUSD asset that behaves correctly in simulation, not just looks right: correct units, physically based materials, collision shapes, mass, physics materials and, for robots, articulation. A profile of requirements defines it, and a validator checks it.

Starting point: a flat JT file — kr270r2700ultra.jt

This is the complete JT file after conversion to OpenUSD. It has nine sibling part groups with 42 meshes and cryptic part names. There is no parent–child chain, no rigid body, no joint, and no material beyond flat CAD colours.

Nothing here says which group is the base, which one moves, or about which axis. Working that out is most of the job.

What you need

• An NVIDIA GPU and Docker with GPU support (Docker Desktop with WSL2 on Windows; the containers reserve 8 CPUs).
• Python 3.12 and uv.
• A coding agent that can load skills (I used Claude Code), with the omniverse-cad-to-simready skill installed.
• An API key for a vision-language model provider. OpenAI worked for me; see “What worked and what didn’t” for the others.
• The CAD file, and the manufacturer’s datasheet for the robot.

The pipeline you will run:

Each stage is a reference script in the skill. Every stage writes a JSON and a Markdown report, and hands its output_usd_path to the next stage. You can call the scripts one by one, or ask the agent to wrap them in a resumable runner as I did (cad2simready.py, which takes --from and --to for any stage).

Step 1: Start the services

1. Put your provider key in ~/.omniverse-cad-to-simready/secrets.env, for example OPENAI_API_KEY=.... Never commit it.
2. Start the renderer. On Linux, the skill deploys OVRTX in Docker. On Windows, run a native OVRTX render service plus an adapter that serves the Content Agents rendering API on port 8001, as described in How to Use OVRTX Rendering from WSL2 Containers with a Windows Wrapper.
3. Deploy the Material and Physics agents. On Windows, deploy them from WSL2, because preflight refuses a managed deployment on Windows.
4. Check the endpoints before you start the pipeline.

python ovrtx_adapter.py --backend http://127.0.0.1:8000 --port 8001        # Windows only
wsl -d Ubuntu -- bash -l ./deploy_content_agents_wsl.sh openai gpt-5.5    # material + physics agents
curl http://localhost:8001/health; curl http://localhost:8100/health; curl http://localhost:8200/health

Ports used: OVRTX render service 8000, render API adapter 8001, Material Agent 8100, Physics Agent 8200.

After a reboot, start Docker Desktop and the adapter again. The containers come back on their own.

Step 2: Give the pipeline context

The material and physics agents only see renders of parts, so tell them what the object is. Run the context stage, then have the agent research the product and write pipeline/01_context/asset-context-research.json. The file holds the identity, cited sources, and material and physics hints, each marked as evidence or inference.

Its material_physics_prompt is passed to every agent downstream. Mine read, in short: KUKA KR 270 R2700 ultra, about 1170 kg, cast metal links in glossy KUKA orange, dark grey motor and gearbox covers, chromed counterbalance rod, black rubber cable harness, steel or cast-aluminium density, metal friction about 0.5, low restitution.

You can also pass your own text with --prompt "...".

Step 3: Convert and assign materials and physics

Run everything up to and including assignment:

py -3.12 cad2simready.py testfiles\kr270r2700ultra.jt --to assign --assign-timeout 7200

• Convert uses NVIDIA’s usd-convert-cad and keeps the CAD assembly tree. For the KR 270: 42 meshes in millimetres, Z up.
• Minimum gate checks that the stage opens and has a default prim, up-axis and units, before anything expensive runs.
• Material Agent renders each mesh, classifies it with a vision-language model and binds a SimReady library material.
• Physics Agent adds a collider, density and physics material to every mesh, plus a mass on the asset.

Set --assign-timeout well above the 1800 s default; the material session alone took about 32 minutes here. If the client times out but the session finishes, reuse it with --resume-material-session <id> instead of paying for it again.

Check the result in pipeline/04_assign/: the material and physics OpenUSD file, an HTML report, and one prediction per mesh with its reasoning.

Step 4: Add the joints

JT carries no joints, so the robot is still one solid. Build the articulation from the datasheet:

1. Map links to axes. Inspect the CAD tree with bounds. For the KR 270, the groups k1…k7 are the chain from base to flange, and k9/k10 are the counterbalance.
2. Take ranges and speeds from the datasheet. A1 ±185°, A2 -140/-5°, A3 -120/+155°, A4 ±350°, A5 ±122.5°, A6 ±350°, at 105 to 175 °/s (KUKA product page).
3. Find the pivots. Fit circles to the round axis caps, the wrist hub and the flange. Check the result against the datasheet reach (2694.5 mm fitted vs 2696 mm).
4. Note the CAD pose. KUKA’s home pose here is A2 = -90°, A3 = +90°, so the joint limits in OpenUSD are shifted by that.
5. Write it all to robot-kinematics.json and author the rig as an OpenUSD layer over the physics file:

python author_robot_drives.py 04_assign\physics\kr270r2700ultra_material_physics.usd - `
    04_joints\kr270r2700ultra_articulated.usda `
    --kinematics 01_context\robot-kinematics.json --report 04_joints\drives.json

The layer replaces the single root rigid body with one body per link. It adds six revolute joints with limits, angular drives with the datasheet speed limits, a fixed base, the counterbalance fixed to its links, and the articulation root.

Step 5: Conform and validate

Pick a SimReady profile, then let the pipeline author what it can and check the rest:

py -3.12 cad2simready.py testfiles\kr270r2700ultra.jt --from conform --profile Prop-Robotics-Neutral

• simready-conform-profile authors the profile's features. Here that was core metadata, units, and multi-body rigid bodies.
• The validators then run: the Omniverse Asset Validator (asset, geometry, physics) and simready-validate for the profile.
• If a failing requirement is auto-repairable, the pipeline conforms again and re-validates.
• Read the result in pipeline/08_validate_rerun/*.md. Each failing requirement is listed by ID.

If the profile includes grasp physics (GSP.001), render the asset, pick two grasp points, and rerun:

py -3.12 cad2simready.py testfiles\kr270r2700ultra.jt --from conform --grasp-point x1,y1,z1 --grasp-point x2,y2,z2

Step 6: Render and check the report

The render stage makes a 1024 × 1024 thumbnail with OVRTX, auto-framed from the asset bounds. It also inspects the pixels, so an all-black or uniform image fails the stage. The report stage collects everything into omniverse-cad-to-simready-report.md: each stage's status, the validation gates, the final OpenUSD file, and the reason for anything blocked.

Finally, open the articulated OpenUSD file in a UsdPhysics-aware simulator, press play, and drive the joint targets.

The final structure

The same nine groups are now rigid bodies with mass, connected by six revolute joints and three fixed joints under one articulation root. Every one of the 42 meshes also carries a SimReady material, a collider, a density and a physics material.

And this is the result in USD Composer. Joint A2 is selected: it rotates about Y, connects the rotating column k2 to the link arm k3, and is limited to -50° to 85°.

The kinematic chain lives in the joints, not in the prim nesting: k1…k7 are still siblings, and the joints connect them base to flange. A2 and A3 limits are shifted from the datasheet because the CAD’s zero pose is KUKA’s home pose (A2 = -90°, A3 = +90°). Each revolute joint also has an angular drive and the datasheet speed limit.

A word on the masses. The link masses are estimates, not manufacturer data. The Physics Agent put the whole robot at 746 kg; public sources give about 1170 kg. That total was split across the links by mesh volume × predicted density. This treats hollow castings as solid, can’t see motors and gearboxes the CAD doesn’t model, and inherits the agent’s density guesses (the chromed counterbalance rod came out as plastic). Centres of mass and inertias follow from the same assumptions. The result is good enough to make the asset move plausibly, but check it against the manufacturer’s datasheet or mass data if they are available before using it for dynamics, payload or controller tuning.

What worked and what didn’t

• Platform. Worked: Windows 11 with Docker Desktop and WSL2, agents deployed from WSL. Didn’t: managed deployment from Windows (preflight refuses it), and OVRTX in Docker on WSL2 (no NVIDIA Vulkan/GL driver), so the renderer runs natively with an adapter.
• Context. Worked: agent web research with cited sources, and one prompt reused by every agent.
• Conversion. Worked: usd-convert-cad kept the assembly tree, units and scale. Didn’t: the empty ax* axis frames from JT carry no usable orientation.
• Materials and physics. Worked: the OpenAI backend turned 42 meshes into 5 SimReady materials and 14 physics materials, with colliders and densities. Didn’t: NVIDIA-hosted gemma-4–31b-it (529 overloaded, then “At most 4 image(s)”), an Anthropic account without credit, and the client timeout at the 1800 s default.
• Joints. Worked: datasheet values plus circle-fitted pivots gave 6 revolute joints with drives, and the validator was clean. Didn’t: the Joint Agent (research preview) authored 0 joints in 5 runs. It listed the root prim with a null segment, the CAD repeats child names (e1vc under 5 links), and there was a single root rigid body. A custom prompt-library entry fixed the structure step, but no candidate became ready.
• Conform. Worked: automatic repairs for UN.007, NP.006 and RB.MB.001. Didn’t: GSP.001 grasp points need a human, and 18 dangling over prims fail the asset gate.
• Render. Worked: OVRTX thumbnail with a pixel check.

The short version: the skills automate the tedious parts well, and let the agent do conversion, per-mesh material and physics classification and profile repairs. Anything kinematic still needs real data about the product, and the datasheet was worth more than any model.

Resources on SimReady

Start with the overview and the specification, then use SimReady Foundation to look up what a failing requirement ID means.

• Example: simready_robot: this article’s files in the OpenUSD Studierstube repo (converted, material + physics and articulated OpenUSD files, kinematics spec, reports, pipeline scripts).
• SimReady Overview: NVIDIA’s entry point: what SimReady is, simulation needs, asset creation, glossary.
• The SimReady Specification: how assets are made simulation-ready through metadata and core OpenUSD schemas.
• SimReady FAQ: common questions about the standard.
• SimReady 3D Art Asset Creation: modeling, UV, material and physics best practices when building assets by hand.
• Physics Best Practices: colliders, mass in kg, friction and restitution with UsdPhysics.
• SimReady Foundation: requirements, capabilities, features and profiles (e.g. Prop-Robotics-Neutral, Robot-Body-Runnable), plus validators and conform skills.
• Omniverse Asset Validator: rule-based OpenUSD validation with auto-fixes, as a Kit UI or a standalone CLI.
• NVIDIA Agent Skills: official skill catalog for Claude Code, Codex and other agents, including omniverse-cad-to-simready.
• USD Content Agents: Material and Physics agents (beta); Texture, Joint, Geometry and Validation agents (research preview).
• usd-convert-cad: the CAD-to-OpenUSD converter used by the convert stage.
• NVIDIA Agent Toolkit announcement: the 20 July 2026 launch of the CAD-to-SimReady skills.

Install the skill with npx skills add https://github.com/nvidia/skills --skill omniverse-cad-to-simready.

Sources

• NVIDIA usd-convert-cad
• KUKA KR 270 R2700 ultra product page (technical data, opened 21 Sep 2026)

Acknowledgements

Thanks to Ben Völzke for providing the KUKA JT file used in this article.

This article was written with the help of Claude Opus 5 (Anthropic) in Claude Code, which also drove the pipeline as the coding agent described above. The Content Agents used OpenAI models for their own predictions: gpt-5.5 for materials and physics, gpt-4.1 and gpt-4.1-mini in the Joint Agent.

NVIDIA’s OpenUSD Content Agents render every part of your asset before a vision-language model decides its material, physics or joints. They expect an OVRTX rendering service next to them in Docker. On Linux that works out of the box. On Windows with Docker Desktop and WSL2, the OVRTX container dies at startup.

While building a SimReady robot asset from CAD, we got around this by keeping the agents in Docker and running the renderer natively on Windows. A 340-line Python adapter makes the native renderer look exactly like the containerised one. This article shows how to set it up and how the adapter works.

Why OVRTX fails in Docker on WSL2

WSL2 doesn’t pass the GPU through the way native Linux does. The Windows NVIDIA driver exposes a paravirtualised GPU to WSL, with CUDA, DirectX 12 and OptiX libraries mounted from the host (/usr/lib/wsl/lib). There is no NVIDIA Vulkan or OpenGL driver in that set.

OVRTX is an RTX renderer and needs the NVIDIA graphics stack, not just CUDA. The upstream container ships an NVIDIA Vulkan ICD file and expects the driver behind it. On WSL2 there is none, so the renderer daemon exits during initialisation. CUDA-only containers, including the agents themselves, run fine.

The practical conclusion: keep everything that only needs CUDA or the network in Docker, and move the renderer to the Windows host, where the full NVIDIA driver is available.

The architecture

Three pieces, two of them on the Windows host:

• Content Agents run in Docker on WSL2. They speak the Content Agents ovrtx-rendering-api: an OpenUSD scene as a data URI or URL, a list of cameras and frames, and base64 PNGs back.
• The adapter (ovrtx_adapter.py) runs on Windows, port 8001. It serves that API and translates each request to the job API below.
• The OVRTX render service runs on Windows, port 8000. It is a job queue around NVIDIA ovrtx: one OpenUSD file path and one camera per job, a PNG as the result.

The containers reach the host through http://host.docker.internal:8001. The agents don't know anything changed: they see the same /health and /render endpoints the upstream OVRTX container would serve.

Setting it up

1. Run an OVRTX render service on Windows. You need a native process that owns the GPU and renders an OpenUSD file from a camera. Ours is a small FastAPI service around NVIDIA’s ovrtx Python renderer on port 8000. My previous article, How to Build a Simple OVRTX Render Service on Windows, walks through building it; the code is in examples/ovrtx_render_service. A single worker thread holds the one Renderer, because it is single-threaded and loading a scene is expensive. The API has four endpoints:

• POST /render: submit {usd_path, camera_path, width, height, force_reload, priority} and get a job_id back.
• GET /jobs/{id}: job status (queued, running, done, failed or cancelled).
• GET /jobs/{id}/result: the rendered PNG.
• GET /health: {status, worker_alive, pending, running}.

2. Start the adapter. Get ovrtx_adapter.py and run it with Python 3.10 or newer; it uses only the standard library. OpenUSD (pxr, for example pip install usd-core) is needed only for animated cameras.

python ovrtx_adapter.py --backend http://127.0.0.1:8000 --port 8001

It listens on 0.0.0.0:8001, so the WSL2 containers can reach it. Allow the port in Windows Firewall if you are prompted.

3. Point the containers at the host. The agents read the render endpoint from RENDER_ENDPOINT. Set it before deploying and make sure host.docker.internal resolves:

services:
  joint-agent-service:
    environment:
      - JA_RENDER_BACKEND=remote
      - RENDER_ENDPOINT=${RENDER_ENDPOINT:-http://host.docker.internal:8001}
    extra_hosts:
      - "host.docker.internal:host-gateway"

For the Material and Physics agents, export RENDER_ENDPOINT=http://host.docker.internal:8001 in the WSL shell before running the skill's deploy step.

4. Verify. From Windows, and from inside a container (if its image has curl; otherwise use python -c with urllib):

curl http://127.0.0.1:8001/health
docker exec content-material-agent-service curl -s http://host.docker.internal:8001/health

A healthy answer reports "status": "healthy", "renderer": "ovrtx" and the backend's own health under backend_health. Then run any agent session and watch the adapter log rendered 1 frame(s) x 4 camera(s) in …s.

Inside the adapter

The adapter is a standard-library ThreadingHTTPServer with two endpoints. Its job is to close the gap between two contracts:

• Scene. The agents send a url: a data URI, http(s) URL, file URL or path, holding a plain OpenUSD layer or a zip/USDZ bundle. The job API wants a usd_path on the local disk.
• Cameras. The agents send a list of camera prims; the job API renders one camera per job.
• Frames. The agents can ask for a frame_range; the job API only renders the stage's default time.
• Result. The agents expect synchronous JSON with images[frame][camera]["images"] as base64 PNGs; the job API returns a job id to poll, then the PNG as a download.

A /render request goes through five steps:

1. Materialise the payload. Decode the data URI (or download or copy the file) and hash the bytes. Store it once per hash under the work directory; unzip bundles safely, rejecting entries that escape the folder, and pick the root layer.
2. Bake animated cameras. If the request asks for frames other than 0, write one small layer that sublayers the scene and adds a static copy of each camera per frame, with its transform and lens attributes evaluated at that time.
3. Render serially. Under one lock, submit one job per (frame, camera) and poll /jobs/{id} until it finishes. Download the PNG and check it really is one.
4. Answer in-band. Return {status: "success", images}, or {status: "exception", error} with HTTP 200, as the upstream contract expects.
5. Clean up the per-request work folder, unless --keep is set for debugging.

Three design decisions matter for performance:

• Content-addressed cache. The agents resend the same multi-megabyte scene for every view of every part. Storing it by hash gives the backend a stable file path. The backend only reloads the scene (force_reload) when that path changes, so dozens of views share one expensive scene load.
• One layer for all frames. Baking every frame into sibling cameras of a single layer, instead of a layer per frame, avoids a full scene reload per image.
• A deep listen queue. The Joint Agent fires dozens of concurrent /render calls. The stdlib default backlog of 5 refuses most of them, so the server raises request_queue_size to 512. Requests simply wait for the render lock.

Limitations and gotchas

• RGB only. The job API returns colour images. Requested sensors such as depth or segmentation come back as empty strings, with a warning in the response. The Material Agent only asks for RGB. Check your agent before relying on the others.
• One render at a time. The renderer is single-threaded, so the adapter serialises everything. A 42-part robot took the Material Agent about 32 minutes. Raise the agents’ client timeouts accordingly; the adapter’s own per-job timeout is 1800 s (--job-timeout).
• The cache only grows. Materialised scenes stay in %TEMP%\ovrtx_adapter\cache by content hash. Delete that folder now and then.
• Background masks are ignored. apply_background_mask has no equivalent in the job API.
• Nothing restarts by itself. After a reboot, Docker Desktop brings the agent containers back, but the render service and the adapter are plain Windows processes. Start both again, or register them as scheduled tasks. A stopped Docker daemon or adapter looks like an agent that never finishes, so check /health first.
• Ports. Port 8000 is taken by the render service, so move any container that defaults to 8000 (the Joint Agent here) to another host port, and let 8001 through the Windows firewall.

Wrapping up

If a GPU service in Docker on WSL2 needs graphics rather than just CUDA, you don’t have to move the whole stack to Linux. Run the graphics part natively on Windows, and put a thin adapter in front that speaks the API the containers already expect. The agents stay unchanged, and the adapter’s caching and frame baking keep the native renderer efficient.

The same pattern works for any containerised client whose upstream renderer can’t start under WSL2. Match the contract, translate the payload, serialise the GPU.

Resources

The adapter, with a README covering options, API and limitations: examples/ovrtx_wsl_adapter.

• NVIDIA ovrtx (docs): C and Python library for Omniverse RTX rendering and sensor simulation of OpenUSD scenes.
• USD Content Agents: the Material, Physics and Joint agents, and the upstream ovrtx-rendering-api the adapter mimics.
• CUDA on WSL User Guide: what the NVIDIA driver supports in WSL2 (CUDA, DirectX and DirectML); OpenGL-based applications won’t work.
• Docker Desktop GPU support: GPU paravirtualisation for containers on the WSL2 backend.

This article and the adapter were written with the help of Claude Opus 5 (Anthropic) in Claude Code.

NVIDIA’s ovrtx puts Omniverse RTX rendering into a Python package: open a USD stage, step the renderer, get a PNG. That is perfect for a script. But once several tools need renders, such as AI agents in Docker, a web front end or a batch pipeline, you want one process that owns the GPU and takes requests over HTTP.

This article builds that process on Windows in about 250 lines of Python: a FastAPI service with a job queue and a single render worker. Clients submit an OpenUSD file and a camera, poll the job, and download the PNG.

What you need

• Windows 10 or 11 with an NVIDIA RTX GPU and a current driver.
• Python 3.10 to 3.13. I use uv to manage the environment.
• The ovrtx and ovstage wheels from PyPI, plus fastapi, uvicorn, numpy and pillow.
• An OpenUSD scene that contains a camera. The service renders from existing cameras; it doesn’t place its own.

[project]
name = "ovrtx-render-service"
requires-python = ">=3.10,<3.14"
dependencies = [
    "ovrtx==0.5.0.377615",
    "ovstage==0.2.0.377349",
    "fastapi>=0.115",
    "uvicorn>=0.30",
    "numpy>=2.0",
    "pillow>=10.0",
]

Pin the ovrtx and ovstage versions: the API moved between releases. Version 0.4 switched to an “attached” stage, and 0.5 changed how render outputs are keyed. Run everything from PowerShell or cmd. In my setup, the same script failed from Git Bash with failed to parse inline USDA into anonymous layer and worked from PowerShell.

The design

Two facts about ovrtx shape the whole service. The Renderer is single-threaded, and loading a scene is expensive: shader compilation, texture streaming and warm-up frames. So exactly one thread may touch the GPU, and a loaded scene should stay loaded.

The HTTP handlers only read and write the job store. The worker takes the highest-priority job, renders it and writes the PNG to disk. Clients poll until the job is finished; renders take from half a second to a minute, too long to hold a request open.

Step by step

1. Render one image

Start without a service. ovrtx renders render products: a RenderProduct prim binds a camera and a resolution to outputs such as LdrColor. Put the scene, a dome light and the product into one small root layer:

def scene_layer(usd: str) -> str:
    return f'''#usda 1.0
(
    subLayers = [@{asset_path(usd)}@]
)
def DomeLight "RenderSvcSky"
{{
    float inputs:intensity = 1000
}}
'''

def product_layer(camera: str, width: int, height: int) -> str:
    return f'''#usda 1.0
def Scope "RenderSvc"
{{
    def RenderProduct "Product"
    {{
        rel camera = <{camera}>
        int2 resolution = ({width}, {height})
        rel orderedVars = <LdrColor>

 def RenderVar "LdrColor"
        {{
            string sourceName = "LdrColor"
        }}
    }}
}}
'''

Since ovrtx 0.4 the renderer attaches to an ovstage.Stage that you own. Every change to the stage is published under an ordinal, and every step names the ordinal it should see. Seal each ordinal with advance_write_floor, or the step fails:

renderer = ovrtx.Renderer()            # first run compiles shaders
stage = ovstage.Stage("render-one")
renderer.attach_ovstage(stage)

ovstage.population.open_usd_from_string(stage, layer, ordinal=1)
stage.advance_write_floor(1)

for _ in range(40):                    # warm-up: converge and stream textures
    renderer.step(render_products={PRODUCT}, delta_time=1 / 60, ordinal=1)
products = renderer.step(render_products={PRODUCT}, delta_time=1 / 60, ordinal=1)
for product in products.values():
    for frame in product.frames:
        var = frame.render_vars[f"{PRODUCT}/LdrColor"].map(device=ovrtx.Device.CPU)
        pixels = np.from_dlpack(var).copy()   # copy before unmapping
        var.unmap()
        Image.fromarray(pixels).save("render.png")

That is render_one.py. On my machine it takes about 50 s, almost all of it renderer start-up.

2. Keep the scene loaded

In the service, the worker keeps the scene populated and swaps only the product when the next job asks for the same OpenUSD file with another camera. The product must be added as a reference: publishing a product-only layer over the stage replaces the scene instead of adding to it.

def show(self, req) -> int:
    if req.force_reload or self.scene != asset_path(req.usd_path):
        if self.ordinal:
            ovstage.population.reset_usd(self.stage)      # don't merge with the old scene
        ordinal = self.publish()
        ovstage.population.open_usd_from_string(self.stage, scene_layer(req.usd_path), ordinal=ordinal)
        self.stage.advance_write_floor(ordinal)
        self.scene = asset_path(req.usd_path)
    if self.product is not None:
        ovstage.population.remove_usd(self.stage, self.product)
    self.product = ovstage.population.add_usd_reference_from_string(
        self.stage, product_layer(req.camera_path, req.width, req.height), "/")
    ordinal = self.publish()
    ovstage.population.apply_usd_changes(self.stage, ordinal=ordinal)
    self.stage.advance_write_floor(ordinal)
    return ordinal

After show, call renderer.reset() so accumulation starts fresh for the new view, then capture as in step 1. The worker checks a cancel flag between warm-up frames, the only safe point to stop.

3. One worker thread, one job store

The worker is a threading.Thread that creates the renderer inside run(), so every GPU call happens on that thread. The job store is a dictionary of jobs plus a heapq ordered by (−priority, submission number), guarded by a threading.Condition so the worker sleeps until a job arrives.

4. The HTTP layer

FastAPI handlers only touch the store: POST /render validates the request with Pydantic and queues it, GET /jobs/{id} reports status, and GET /jobs/{id}/result returns the PNG with FileResponse. create_app() starts the worker, and uvicorn.run() serves the app on 127.0.0.1:8000.

Run and test

Install and start the service from PowerShell:

uv sync
uv run python render_service.py --port 8000 --output-dir _output

The first start compiles shaders. GET /health reports renderer_ready: true once the worker has its renderer. Then render with the included client, or with any HTTP tool:

uv run python client.py C:\scenes\robot.usda /CameraFront -o front.png

curl -X POST http://127.0.0.1:8000/render -H "Content-Type: application/json" \
  -d '{"usd_path": "C:/scenes/robot.usda", "camera_path": "/CameraFront", "width": 1024, "height": 1024}'
# {"job_id": "f0a4e56b…"}
curl http://127.0.0.1:8000/jobs/f0a4e56b…            # poll until "done"
curl -o front.png http://127.0.0.1:8000/jobs/f0a4e56b…/result

I tested it with a KUKA KR 270 robot (42 meshes, SimReady materials, millimetre units) and three cameras at 1024 × 1024: one perspective and two orthographic. Orthographic cameras need nothing special in the service. Set the projection on the camera, and give the apertures in tenths of a scene unit, so 28000 covers 2.8 m in a millimetre stage:

cam = UsdGeom.Camera.Define(stage, "/CameraTop")
cam.AddTransformOp().Set(Gf.Matrix4d().SetTranslate(Gf.Vec3d(580, -7, 6000)))  # looks down -Z
cam.CreateProjectionAttr("orthographic")
cam.CreateHorizontalApertureAttr(28000.0)   # 2800 mm view width
cam.CreateVerticalApertureAttr(28000.0)
cam.CreateClippingRangeAttr(Gf.Vec2f(1, 20000))

The first job’s time depends on what the renderer has already cached: 1.5 s in a long-running environment, 13 s on the second start of a new one, and about 40 s on the very first.

Gotchas

• The first start is slow. In a fresh environment, creating the renderer took about 3 minutes (shader compilation) and the first job about 40 s. After that the shader cache is warm: renderer start-up drops to well under a minute, a new scene loads in seconds, and more views of it take half a second.
• Seal every ordinal. A publication that isn’t sealed with advance_write_floor makes the next step fail with “ordinal N exceeds ovstage write floor”.
• Reset before reloading. Opening a second scene without reset_usd merges it with the first one.
• Products as references. Publishing a product-only layer over a populated stage replaces the scene and renders black. Add the product with add_usd_reference_from_string, remove the old one, and apply_usd_changes.
• Output keys changed in 0.5. Render outputs are keyed by the full prim path, for example /RenderSvc/Product/LdrColor, not by the bare name.
• Unlit scenes. CAD exports rarely carry lights. The composed layer adds a dome light so they don’t render black.
• Cancellation is cooperative. A running job can only stop between warm-up frames.
• Shell. Start it from PowerShell or cmd, not Git Bash (see above).

Ideas for next steps: prefer queued jobs for the scene that is already loaded, add a WebSocket for status updates instead of polling, and prune old PNGs from the output folder.

Resources

The complete sample (render_one.py, render_service.py, client.py, pyproject.toml) is in examples/ovrtx_render_service.

This article and the sample code were written with the help of Claude Opus 5 (Anthropic) in Claude Code.

OpenUSD as a Digital Twin Substrate

OpenUSD is not a 3D format. It is a composable semantic scene graph for representing, simulating, and linking complex systems across domains. If that framing is new to you, we covered it in depth in our previous article — OpenUSD Is Not a 3D Format — and it is the foundation everything here builds on.

The short version: an OpenUSD stage can carry geometry, physical properties, sensor data, compliance metadata, and lifecycle records in a single composable, layered format. Applied API schemas attach typed, discoverable properties to any prim. Relationships create explicit graph edges between prims. The composition engine — with its LIVERPS arc ordering — assembles data from multiple sources non-destructively.

That infrastructure is mature and widely adopted. The question for industrial digital twins is not whether to use OpenUSD. The question is what semantic content to put in it.

The Asset Administration Shell: Valuable Semantics, Difficult Architecture

The Asset Administration Shell (AAS) was developed within Germany’s Industrie 4.0 initiative as the standardized digital twin framework for manufacturing. Backed by substantial public funding and institutional support from organizations such as Plattform Industrie 4.0 and the Industrial Digital Twin Association (IDTA), it defined a rigorous metamodel for describing industrial assets: their identity, their technical properties, their lifecycle history, their documentation.

In practice, AAS adoption has been limited. Even in Germany, where investment was heaviest, industrial deployments have remained largely confined to research and pilot contexts. Outside Germany, uptake has been close to negligible. The reasons are well understood in the industry: the AAS architectural stack is complex, the tooling ecosystem is thin, and the technology choices have a distinctly academic feel compared to the pragmatic, ecosystem-driven approaches that have succeeded at scale in software infrastructure.

The most useful part of AAS is its Submodel Templates (SMTs).

SMTs are standardized, versioned, semantically typed schemas for recurring industrial concerns: digital nameplates, technical data sheets, handover documentation, carbon footprints, predictive maintenance records, bill of materials. Each is developed by domain working groups and published in the IDTA content hub. In my experience, they vary in maturity and depth — but the strongest ones represent genuine, hard-won agreement across manufacturers, regulators, and system integrators about what information a particular type of object should carry and what it means.

That semantic precision is exactly what a general-purpose scene graph format like OpenUSD needs when it moves into regulated industrial contexts. OpenUSD does not know what a Digital Product Passport requires. AAS Submodel Templates do.

The opportunity is to use the SMTs as semantic source material for OpenUSD schemas — taking the well-defined vocabulary and regulatory grounding of AAS data definitions and expressing them as typed, discoverable, composable OpenUSD applied schemas.

The Identifier Problem That Has to Be Solved First

Before any AAS semantic content can be cleanly encoded in OpenUSD, there is a foundational conflict to resolve.

OpenUSD addresses every object in a scene by a namespace path — a hierarchical string like /Plant/Line3/Pump_CM5_3. These paths are the keys of OpenUSD's composition engine. They follow strict grammar rules. They must not contain characters that external systems routinely use in identifiers: slashes in the middle of a name, leading digits, hyphens used as separators, URI schemes.

Industrial assets carry identifiers from external systems that do not respect OpenUSD’s grammar: product URIs from regulatory registries, serial numbers from MES systems, order codes from ERP systems, document IDs from DMS platforms, facility identifiers under EU regulations. A battery’s product URI under EU Battery Regulation might be https://www.example.com/batteries/CM5-3/2024. An ECLASS-coded semantic identifier looks like 0173-1#02-AAO677#002. Neither can become an OpenUSD prim name without transformation — and transformation means data loss and broken cross-system traceability.

A recent OpenUSD Enhancement Proposal, Separation of Concerns for Identifiers in OpenUSD, led by Aaron Luk and currently a draft submitted to the OpenUSD Alliance, addresses this directly. The principle is simple: OpenUSD namespace paths are for scene composition and hierarchy navigation. External identifiers should be stored alongside them, verbatim, in a standardized mechanism that is independent of OpenUSD grammar and discoverable by tooling.

Two candidate mechanisms are under evaluation — extending the existing assetInfo metadata dictionary with structured sub-dictionaries, or defining typed applied schemas with named properties. The mappings in this work use a sourceId:* attribute namespace (e.g., sourceId:dpp:uriOfTheProduct, sourceId:doc:documentId) as the practical expression of this principle: external identifiers are stored as plain strings on the prim, alongside OpenUSD-native typed attributes that encode the same data in OpenUSD-idiomatic form where applicable.

Without this mechanism, any AAS-to-OpenUSD mapping that involves external identifiers — which is essentially all of them — must choose between data loss and ad-hoc workarounds. The identifier separation is the enabling layer that makes clean mappings possible.

The Mapping Approach

Our mappings follow the OpenUSD Conceptual Data Mapping Template, a community document that provides a consistent structure for cross-format mappings. Each submodel mapping document specifies:

• Directionality: one-way import or full bidirectional round-trip
• High-level concept table: AAS elements mapped to OpenUSD constructs, with round-trip fidelity rated per field (lossless, lossy, partial, or dropped)
• Per-concept drilldowns: encoding rules, fallback strategies, and data transformations
• Illustrative OpenUSD schema definitions: typed API schema classes formalizing the mapping
• Round-trip analysis: explicit documentation of what survives and what is lost in an AAS → OpenUSD → AAS cycle

The general encoding pattern is consistent across all three submodel templates mapped so far:

1. The prim represents the physical or logical asset. Its OpenUSD namespace path is chosen for composition purposes, independent of any AAS identifier.
2. Applied API schemas carry the submodel data as typed, discoverable properties. Tooling can find all prims carrying battery nameplate data by checking for BatteryNameplateAPI — no metadata parsing required.
3. Source identifier attributes store external identifiers verbatim on the prim. They are distinct from the OpenUSD-native attributes that encode the same data in OpenUSD-idiomatic form.
4. Child scope prims handle structured sub-elements with variable cardinality — markings, documents, document versions — using a consistent Concept_NN zero-padded naming convention that preserves AAS list ordering.
5. customData preserves AAS-specific metadata for round-trip fidelity: semantic IDs, language variants, and original AAS idShort values.

The OpenUSD schema hierarchy mirrors the AAS inheritance hierarchy. DppNameplateAPI is the shared base schema covering the full IDTA 02006-3-0 Digital Nameplate fields. Domain-specific schemas compose on top: BatteryNameplateAPI adds the five battery-specific fields from IDTA 02035-1; IndustrialEquipmentAPI adds the industrial equipment extensions. A battery prim carries both DppNameplateAPI and BatteryNameplateAPI — parallel to how the AAS Battery Nameplate SMT extends the Digital Nameplate SMT.

The Three Submodel Templates Mapped

IDTA 02006–3–0 — Digital Nameplate for Industrial Equipment

The Digital Nameplate is the foundational AAS SMT for industrial asset identity. It defines the core fields present in virtually every product passport: product URI, manufacturer name, product designation, order codes, article numbers, serial number, date of manufacture, facility identifier, markings, and asset-specific properties. Domain-specific nameplates (battery, machinery, software) derive from it.

A critical distinction the mapping makes explicit is between type-level and instance-level identifiers:

• Type-level identifiers — ManufacturerProductType, OrderCodeOfManufacturer, ProductArticleNumberOfManufacturer — identify a product model: a class of equipment manufactured to a specification, shared across all units.
• Instance-level identifiers — SerialNumber, URIOfTheProduct, UniqueFacilityIdentifier — identify a specific serialized unit: a particular physical object at a particular location.

This distinction is fundamental in AAS modeling and must survive in the OpenUSD encoding. Downstream systems — maintenance platforms, compliance registries, recycling operators — need to know whether they are looking at a model reference or a unit reference. Both classes of identifiers use the sourceId:dpp:* dual-encoding pattern, but the semantic distinction is preserved in the schema documentation and is available to tooling.

The idta_02006_digital_nameplate_sample.usda file shows a complete encoding for an industrial pump (Pump_MusterAG_CM5_3) including contact information, markings, ATEX explosion safety data, and asset-specific technical properties.

IDTA 02035–1 — Digital Battery Passport (Part 1: Digital Nameplate)

The battery passport nameplate is the Digital Nameplate extended for compliance with EU Battery Regulation (EU) 2023/1542. It adds five fields to the 02006–3–0 base: LifeCycleStage, OperatorIdentifier, ManufacturerIdentifier, EUDeclarationOfConformity, and ResultsOfTestReportsProvingCompliance.

In OpenUSD these are encoded in BatteryNameplateAPI, applied alongside DppNameplateAPI. ManufacturerIdentifier is an opaque external string stored in sourceId:dpp:manufacturerIdentifier. EUDeclarationOfConformity and ResultsOfTestReportsProvingCompliance are arrays of document identifier strings that cross-reference compliance documents in a Handover Documentation submodel — they are stored as sourceId:dpp:* string arrays, and the Handover Documentation mapping defines how to resolve the cross-reference on a shared OpenUSD stage. LifeCycleStage uses an OpenUSD token that maps to an ECLASS IRDI via a lookup table, preserving both OpenUSD-native enumeration semantics and AAS semantic identifier round-trip fidelity.

An important point about scope: the battery passport nameplate covers identity and markings only. The full DPP as defined by the EU Battery Regulation also requires technical data, carbon footprint, state of health, and other submodels. The nameplate mapping is the foundation layer; further submodel mappings build on it using the same schema composition approach.

The idta_02035_battery_passport_sample.usda file shows a complete battery instance (Battery_A12345X75EN).

IDTA 02004–2–0 — Handover Documentation

Handover Documentation is the AAS SMT for transferring product-related documents across organizational boundaries over an asset’s lifecycle: operating manuals, certificates, drawings, compliance declarations. Its central identifier challenge is the DocumentIds SML — an explicit multi-system identifier store where a single document may simultaneously carry IDs from a DMS, an ERP system, a regulatory body, and a certification authority. All of these must survive cross-system transit verbatim.

The mapping defines five coordinated API schemas: HandoverDocumentationAPI as the discoverer, DocumentAPI per document entry with OpenUSD relationships for intra-stage coverage, DocumentVersionAPI for file references and version metadata, DocumentIdAPI for individual document identifiers with sourceId:doc:documentId as the verbatim string, and DocumentClassificationAPI for VDI 2770 / IEC 61355 classifications.

The cross-reference with the battery passport is made explicit: when EU Declaration of Conformity IDs stored in sourceId:dpp:euDeclarationOfConformityIds on a battery prim match docId:documentId values in DocumentId_NN entries on the same stage, the link can be expressed as an OpenUSD relationship — connecting the regulatory compliance requirement on the product prim to the actual document record in the documentation layer.

The idta_02004_handover_documentation_sample.usda demonstrates this with two documents: an EU Declaration of Conformity cross-referenced from the battery passport, and an operating manual with German and English versions linked as translations.

What This Enables and What Comes Next

These three mappings demonstrate the methodology. The AAS SMT library contains many more: Technical Data (IDTA 02003), Carbon Footprint (IDTA 02023), Software Nameplate (IDTA 02007), Predictive Maintenance (IDTA 02017), and others. Each can follow the same pattern: map the SMT fields to typed OpenUSD applied schema properties, encode external identifiers verbatim via sourceId:* attributes, preserve semantic IDs and language metadata in customData, and define child prim structures for variable-cardinality sub-elements.

The result is an OpenUSD stage that can carry the full semantic richness of an AAS-defined asset description — with the geometry, physics, and simulation data that AAS has no vocabulary for — in a single composable, layered format that OpenUSD-native tooling can traverse, validate, and render without an AAS runtime.

OpenUSD does not need to become AAS. But it can carry what AAS Submodel Templates know how to define.

Try the Files

The mapping documents and USDA samples are available in the identifier_separation_of_concerns directory of the OpenUSD proposals repository:

Mapping documents and samples:

• Identifier Separation of Concerns proposal: README.md
• Reusable mapping template: conceptual_data_mapping_template.md
• IDTA 02006–3–0 Digital Nameplate mapping: idta_02006_digital_nameplate_openusd_mapping.md
• IDTA 02006–3–0 industrial pump sample: idta_02006_digital_nameplate_sample.usda
• IDTA 02035–1 Battery Passport mapping: idta_02035_battery_passport_openusd_mapping.md
• IDTA 02035–1 battery unit sample: idta_02035_battery_passport_sample.usda
• IDTA 02004–2–0 Handover Documentation mapping: idta_02004_handover_documentation_openusd_mapping.md
• IDTA 02004–2–0 handover documentation sample: idta_02004_handover_documentation_sample.usda

Official IDTA specifications:

• Digital Nameplate for Industrial Equipment: IDTA 02006–3–0 (PDF)
• Digital Battery Passport Part 1: IDTA 02035–1 (PDF)
• Handover Documentation: IDTA 02004–2–0 (PDF)

The USDA files are valid USD ASCII and can be opened in any OpenUSD-capable tool. Schema classes are defined inline for illustration; in production they would be registered as proper OpenUSD schema plugins.

In Summary

AAS Submodel Templates contain something genuinely useful: semantically precise, industry-vetted definitions for real-world asset data, grounded in regulatory requirements and developed by domain experts. OpenUSD contains something AAS lacks: a proven, high-performance, widely-adopted composition and interchange infrastructure backed by the most significant technology investment in the 3D and simulation space.

The mappings presented here show how to take the valuable part of AAS — its submodel semantics — and express it cleanly in the format that is actually winning. The identifier separation proposal is the enabling mechanism that makes the encodings clean rather than lossy. The bidirectional mapping structure ensures that data can move in both directions without silent loss.

Industrial digital twins need good semantics. They also need infrastructure that the industry will actually use. These two things do not have to be in tension.

The mapping documents in this article are informed by the OpenUSD Enhancement Proposal “Separation of Concerns for Identifiers in USD,” led by Aaron Luk and currently a draft under review by the Alliance for OpenUSD.

The IDTA Submodel Template library is published at https://industrialdigitaltwin.org/en/content-hub/submodels.

This article was written with the assistance of Claude Code (Claude Sonnet 4.6) by Anthropic.

It is easy to understand how the misconception formed.

Universal Scene Description was created at Pixar. Its first public showcase was a film pipeline tool. Its file extensions are .usd, .usda, .usdc, .usdz.
Its ecosystem features DCC tools — Maya, Houdini, Blender — and rendering workflows at Apple, NVIDIA, and Adobe.

If you encountered OpenUSD for the first time through any of those entry points, "3D format" is a reasonable first impression.

But OpenUSD has moved far beyond that origin. Today it is actively adopted in industry, construction, and robotics. NVIDIA’s Omniverse platform has been the primary force pushing OpenUSD into industrial contexts — factory simulation, robotic teleoperation, digital factory twins — and is now central to Physical AI workflows where simulation and the real world must stay in sync.

Global industrial software leaders like Siemens and PTC are adopting OpenUSD as part of their digital twin and PLM strategies. The format that assembled CGI scenes at Pixar is now assembling digital representations of production lines, buildings, and autonomous systems.

So why does the “3D format” label persist? Because the mental model never caught up with the technology.

What OpenUSD Actually Is

OpenUSD is best understood as a composable, layered scene graph and data model for representing complex systems — with geometry, time, relationships, and semantics.

Not just 3D.

It sits closer to:

• A graph-based digital twin substrate
• A scene + data + relationship model
• A composition system for heterogeneous industrial data

The formal definition from the Alliance for OpenUSD captures it well:
a “rich and extensible ecosystem of software and standards for collaboratively constructing animated 3D scenes.”

But notice what that definition does not say: it does not say only 3D scenes.

The word “extensible” is doing a lot of work. The schemas that make OpenUSD meaningful for geometry — UsdGeomMesh, UsdGeomCamera — are just schemas. You can define any schema. You can represent anything.

The Correct Mental Model

Instead of:

❌ OpenUSD = 3D rendering format

The accurate model is:

OpenUSD = Scene Graph + Composition Engine + Typed Data Model + Relationship Layer

Each component matters independently:

Scene graph — a hierarchy of objects. Not meshes. Objects. A battery cell, a compliance record, a robot joint, and a triangle mesh are all equally valid prim candidates.

Composition engine — OpenUSD defines a precise set of composition arcs, remembered by the acronym LIVERPS: Local, Inherits, VariantSets, rElocates, References, Payloads, Specializes. These arcs resolve in a defined strength order, enabling non-destructive assembly of data from multiple sources and teams — overrides, configurations, deferred loading, and structural inheritance all composing cleanly without conflict.

Typed data model — schemas and attributes with declared types. battery:capacity = 75.0 (kWh) is as valid as xformOp:translate = (1, 0, 0).

Relationship layer — named, typed graph edges between prims. A sensor stream relationship, a supplier relationship, a simulation input binding — these are first-class OpenUSD constructs, not hacks.

Take away the geometry schemas and OpenUSD is still a complete, functional system. That is not an accident.

Why It Is Fundamentally a Graph

OpenUSD describes a directed scene graph:

• Nodes → primitives (Prims)
• Edges → relationships (references, inherits, payloads, explicit rel declarations)
• Attributes → typed metadata on nodes

Consider this:

Car
 ├── Body
 │    ├── Material: Aluminum
 │    └── Mass: 120 kg
 ├── Battery
 │    ├── Capacity: 75 kWh
 │    └── Supplier: DID:example:supplier:123
 └── Wheels
      ├── Geometry
      └── SensorData

Nothing in this structure requires rendering. It is a semantic graph of a system — its composition, its properties, its provenance. The Geometry on Wheels is one attribute set among several. The SensorData on Wheels is equally valid OpenUSD.

This is where the mental model shift matters. A 3D format is for describing surfaces. OpenUSD is for describing systems.

Where Semantics Live in OpenUSD

OpenUSD has four distinct semantic mechanisms, and only one of them involves geometry.

Prim structure (hierarchy) defines system decomposition:

product → subsystem → component → part

Custom attributes (typed metadata) carry domain data:

battery:capacity = 75.0
battery:recycledContent = 0.42
battery:cellChemistry = "NMC"

Relationships create graph edges beyond parent-child hierarchy:

rel materialBinding
rel supplier
rel simulationInput
rel sensorStream

Applied schemas are where domain meaning gets encoded formally. Unlike typed schemas that define what a prim is, API schemas define what a prim carries — typed properties and behaviors declared once in a schema definition file, then applied to any prim via apiSchemas metadata.

Domain concepts like these become first-class schema types:

class "BatteryNameplateAPI" (inherits = </APISchemaBase>) { ... }
class "DrivetrainAPI" (inherits = </APISchemaBase>) { ... }
class "SensorBindingAPI" (inherits = </APISchemaBase>) { ... }

Once registered via the usdGenSchema toolchain, these schemas generate typed accessors and make domain properties formally declared and discoverable — laying the groundwork for interoperability, with the degree of validation depending on the tooling built around them.

How This Maps to Digital Product Passports

The EU Battery Regulation (2023/1542) mandates a Digital Product Passport for batteries entering the European market: lifecycle data, material composition, recycled content, carbon footprint, compliance certifications.

This is not a rendering problem. It is a data, structure, and relationship problem.

OpenUSD can represent the structural and relational model of a battery product precisely because it is a graph — not a 3D format. A battery is not a mesh; it is a system. Its hierarchy of subsystems, its typed regulatory attributes, and its relationships to suppliers, certifications, and lifecycle records map directly to native OpenUSD constructs.

What emerges is a clean architectural separation:

Each layer solves a different problem.

OpenUSD does not define meaning — it structures it. It does not establish trust — it carries references to it. It does not handle data exchange — it provides the model being exchanged.

That distinction matters.

Treating OpenUSD as a rendering layer beneath a “real” data architecture leads to unnecessary complexity: duplicated models, fragmented semantics, and brittle integrations. The correct approach is to use OpenUSD as the structural substrate — the system model that everything else attaches to.

Digital Product Passports are not documents. They are graphs of products, properties, and relationships over time.

That is exactly what OpenUSD was built to represent.

In a companion article, we map the EU Digital Battery Passport to OpenUSD applied schemas in full — showing field by field how regulatory data lands cleanly in the scene graph, and what the bidirectional round-trip looks like in practice.

Why OpenUSD Is Especially Powerful for Physical AI

Physical AI — robots, autonomous systems, digital factories — requires a representation layer that can hold:

• Multi-domain models: mechanical, electrical, software, simulation, sensor data, simultaneously
• Time and variants: configurations, states, versions — a robot arm in maintenance mode is a different variant of the same prim hierarchy
• Simulation integration: OpenUSD’s physics schemas (from NVIDIA’s PhysicsSchemas) connect directly to real-time simulation

This is why OpenUSD is emerging as the structural backbone of Physical AI systems. It is not that game engines and robotics frameworks happen to use 3D formats for data exchange. It is that OpenUSD’s composition and schema system provides the right level of abstraction for representing complex, multi-domain physical systems — and geometry is simply one of those domains.

NVIDIA Omniverse is explicit about this: the platform is not a 3D viewer, it is a simulation and collaboration substrate. OpenUSD is its data model. Apple’s visionOS uses OpenUSD as its spatial computing format — not because Apple needed a 3D format, but because OpenUSD’s composition model handles the complexity of layered, multi-source spatial scenes better than anything else available.

The One-Sentence Correction

OpenUSD is not a 3D format — it is a composable semantic scene graph for representing, simulating, and linking complex systems across domains.

Its origins at Pixar gave it excellent geometry support. Its architecture gave it something more general: a principled, extensible, composable model for describing any structured system.

The geometry schemas are an application of OpenUSD, not its definition.

Treating OpenUSD as a 3D format in industrial and Physical AI contexts is a missed opportunity. Existing industrial semantics — AAS Submodel Templates, regulatory data models, PLM schemas — can be mapped directly to OpenUSD applied schemas, making them composable, layerable, and simulation-ready without rebuilding them from scratch. The composition engine does the heavy lifting; you just need to bring the right schemas to it.

Define the schemas. Use the relationships. Trust the composition engine. That is what OpenUSD is for.

This article was written with the assistance of Claude Code (Claude Sonnet 4.6) by Anthropic.

Running Isaac Sim in the cloud often means wrestling with GPU drivers, configs, and networking. With NVIDIA Brev, it’s almost plug-and-play.

I followed the Isaac Sim Brev setup guide and had a fully working streaming setup in about 10 minutes. Here’s the flow:

1️⃣ Create a GPU instance

• Sign in to Brev → Create New Instance
• Choose 1× NVIDIA L40S GPU
• Deploy and wait until it’s ready

2️⃣ Expose the streaming ports

• Open ports 49100 and 47998 for WebRTC streaming

3️⃣ Run Isaac Sim in Docker

docker pull nvcr.io/nvidia/isaac-sim:5.0.0
docker run --name isaac-sim --entrypoint bash -it --runtime=nvidia --gpus all \
  -e "ACCEPT_EULA=Y" -e "PRIVACY_CONSENT=Y" --rm --network=host \
  -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
  -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
  nvcr.io/nvidia/isaac-sim:5.0.0

4️⃣ Start Isaac Sim livestreaming

PUBLIC_IP=$(curl -s ifconfig.me) && \
./runheadless.sh --/app/livestream/publicEndpointAddress=$PUBLIC_IP \
                 --/app/livestream/port=49100

5️⃣ Connect

• Use the Isaac Sim WebRTC Streaming Client
  (Download Isaac Sim — Isaac Sim Documentation)
• Point it to your instance’s public IP

👉 And that’s it — from zero to streaming Isaac Sim in minutes, without painful setup.

This workflow makes NVIDIA Brev a great entry point for anyone who wants to explore Omniverse workloads quickly, whether for prototyping, testing, or scaling experiments.

Setting up a local environment for Omniverse development can be a hassle — configuring GPU drivers, installing the right software versions, managing dependencies… you know the drill.

But what if you could skip all of that and launch a fully configured RTX-ready workstation in the cloud — in under 20 minutes?

Here’s how to do exactly that using AWS and NVIDIA’s Omniverse Workstation AMI. You’ll have a virtual machine ready for development and your first USD Composer Kit App running in no time.

🔧Step 1: Launch an Omniverse Workstation in AWS

1. Open the AWS EC2 Dashboard and click on “Launch instance”.
2. Under Amazon Machine Image (AMI), search for “nvidia” and switch to the “AWS Marketplace AMIs” tab.
3. Choose either:
   - NVIDIA Omniverse Development Workstation (Windows) — great for dev work, or
   - NVIDIA Omniverse Workstation (Windows) — includes the Omniverse license ($1/hour)
4. Click Subscribe (or Subscribe on instance launch).
5. Select an instance type:
   - Recommended: g6e.2xlarge (64 GB RAM, 8 vCPUs)
   - Lightweight: g6e.xlarge (32 GB RAM, 4 vCPUs).
6. Create a new key pair and download the .pem file.
7. Click Launch instance.

🔐Step 2: Enable Remote Desktop Access (RDP)

1. In the instance summary, go to the Security tab.
2. Click the security group link, go to Inbound rules, and click Edit.
3. Add a rule for:
   - Type: RDP
   - Port: 3389
   - Source: Anywhere — IPv4
4. Click Save rules.

🧩Step 3: Retrieve Your RDP Password

1. Select your running instance, click Connect, and go to the RDP Client tab.
2. Wait ~4–10 minutes for full initialization.
3. Click Get password.
4. Upload your .pem file and click Decrypt password.
5. Copy and save the password securely.

💻Step 4: Connect via RDP

1. Download the Remote Desktop (.rdp) file from the AWS Console.
2. Open it with your local RDP client and connect.
3. Enter the decrypted password.

Congrats — you now have remote access to a fully equipped Omniverse development workstation! 🎉

🧱Step 5: Create and Launch a USD Composer Kit App

Now that you’re in, let’s build a sample Omniverse Kit App using the official template.

1. Open a terminal and create a source folder, e.g., C:\_SOURCE
2. Clone the Omniverse Kit template:

git clone https://github.com/NVIDIA-Omniverse/kit-app-template.git

3. Navigate into the project

cd kit-app-template

4. Run the template wizard:

.\repo.bat template new

5. Accept the terms, choose:

- Application
- USD Composer
- Press Enter for the remaining prompts to use default names.
- Select No for additional streaming layers.

6. Build & Launch your new Kit app:

# build (once) 
.\repo.bat build

# launch 
.\repo.bat launch

Boom! Your own USD Composer Kit App is now running in a powerful RTX-accelerated virtual workstation in the cloud.

Whether you’re experimenting with Omniverse development or onboarding a new team member, this setup saves time and removes friction.

Troubleshooting

Insufficient vCPU Quotas

Your quota for vCPUs might not be sufficient

• go to Service Quotas -> Dashboard and search for “on-demand”
• apply for more vCPUs for “Running On-Demand G and VT instances” ( we need 8 per additional AMI ) .

No Access / Can’t connect with RDP

Make sure to download the RDP file after you’ve started the machine — the IP adress will change each time you start it, so don’t reuse previously downloaded RDP files.

Build & Run Docker Container

In order to use the API server we have to clone the repository with the docker image description from github. It is a fork of the original repository from Tencent with added API server.

Let’s build the docker image and start our container. Make sure to have Docker Desktop the Linux on Windows subsystem (WSL) installed.

Please note the LICENSE information from Tencent, which excludes the EU, UK and South Korea.

# checkout repository 
git clone https://github.com/perfectproducts/Hunyuan3D-2.1.git . 
# build docker image 
cd docker 
docker build -t hunyuan3d21 .
# run docker container with gpu support and exposed server port 8081
docker run --gpus all -p 8081:8081 hunyuan3d21
...
# (first start takes around 45 minutes to download dependencies) 
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8081 (Press CTRL+C to quit)

Make sure that the server is reachable by opening the documentation page of the server API and by executing the health operation:

http://localhost:8081/docs#/status/health_check_health_get

Once the server is running we are sure that every dependency has been downloaded so lets commit the state of the container to a new image which is ready to use on container startup.

# save our ready to use container to a new image 
docker commit [your container name] hunyuan3d21:readytorun

# here's how we run our new image 
docker run -it --rm --gpus all -p 8081:8081 hunyuan3d21:readytorun

Now we’re ready to build our own Omniverse KIT App !

Build & Run Omniverse KIT App & Extension

Let’s check out the source code and build the KIT app with our Hunyuan 3D Extension:

# clone repo 
git clone https://github.com/perfectproducts/synctwin_hunyuan3d_ov .

# build KIT app 
repo build 

# run KIT app 
repo launch

Now we can launch the KIT App with our Hunyuan3D KIT extension.

To try our extension we take an arbitrary image from a website — here we use the image of an industrial machine.

• After we’ve saved the picture we select it by clicking on the image button (1)
• Set the host and port of the docker container with the preferences button (2). The status line on the bottom of the widget should say (Online)
• Click on “Generate 3D” button and wait for the generation of your 3D model. It should be loaded after generation (takes 1–2 minutes).

Happy generating !

Troubleshooting

Increase WSL Memory Limit

If the server runs out of memory during generation you can assign more memory to the WSL.

Edit C:\Users\<YourUsername>\.wslconfig and increase the memory limit to more than 16GB to prevent crashes in the docker container due to out-of-memory. (for me 32 works)

[wsl2]
memory=32GB
processors=4
swap=8GB

Add Firewall Rule for Port 8081

Windows Defender Firewall blocks external access by default, even if Docker is exposing the port .

🔧 Fix:

Press Win + R, type wf.msc, hit Enter.
Go to Inbound Rules → New Rule.
Choose Port → TCP → 8081
Allow the connection → Apply to all profiles.
Give it a name like “Docker App Port 8081” → Finish.