Magic Leap and NVIDIA collaborate on digital twins for enterprise
April 12, 2023 – Magic Leap has introduced that it’s collaborating with NVIDIA to pioneer immersive, photorealistic, ray-traced digital twins in NVIDIA’s Omniverse platform, to be able to create new efficiencies and worth in engineering, structure, retail, and different industries.
Magic Leap started its collaboration with NVIDIA to discover a technique to allow off-board computing and streaming of 1:1 scale 3D digital fashions to its Magic Leap 2 augmented actuality (AR) headset. In keeping with Magic Leap, the collaboration permits enterprise customers to render and stream immersive, full-scale digital twins from the NVIDIA Omniverse platform to Magic Leap 2 with out compromised visible high quality and contains an extension integrating widespread enterprise modeling packages into the UI.
Technical developments that the businesses have labored on embrace:
- Ray tracing and Deep Studying Tremendous Sampling (DLSS) that permits content material to be rendered with extremely detailed, photorealistic high quality;
- Magic Leap Dynamic Dimming know-how, which eliminates 99.7% of ambient mild from the show to make AR usable in a higher number of shiny lighting situations;
- Correct environmental lighting and reflections throughout digital content material;
- Depth-based reprojection functionality that eliminates latency and anchors digital content material persistently inside a person’s atmosphere.
To assist make design and 3D modeling extra environment friendly, and to empower enterprises and professionals who create, design, and collaborate to render bodily correct, photorealistic digital twins on the Magic Leap 2 gadget, Magic Leap created an Omniverse extension that renders content material to the headset from the commonest enterprise modeling instruments, resembling Autodesk Maya, with out the necessity for conversions or optimizations. Content material is saved in Omniverse Nucleus as USD information. In Nucleus, customers can reside edit and collaborate on the content material.
By merely clicking “View on Magic Leap”, a picture is rendered on the server and streamed to Magic Leap 2, the place it’s immediately displayed for the person within the right place, orientation, and scale. Omniverse’s core XR extension will robotically render the high-quality stereo photos and submit them to the industry-standard OpenXR API.
The server-side OpenXR distant rendering runtime then leverages NVIDIA GPU acceleration to composite and encode the stereo photos. These photos are then transmitted over the community the place they’re obtained, decoded, and displayed on Magic Leap 2.
Magic Leap 2 can be always sending head poses, eye monitoring, controller state, and different enter knowledge again to the server the place it’s surfaced to Omniverse via OpenXR. On the again finish, different customers can proceed to reside edit the USD content material with adjustments immediately rendered and streamed to Magic Leap 2.
Magic Leap said that by leveraging numerous Magic Leap 2 options and NVIDIA RTX GPUs, the answer helps to mitigate widespread challenges, together with:
- Reprojection mitigates latency and locks content material in place – Magic Leap 2 can tolerate greater than 100 milliseconds of end-to-end latency whereas sustaining an immersive expertise for the person;
- NVIDIA RTX ray tracing and DLSS allow photorealistic high quality;
- Dynamic Dimming know-how makes content material opaque and provides behaviorally-realistic shadows;
- Correct environmental lighting and reflections guarantee immersion. The corporate added that it’s actively researching methods to make this performance much more practical via dynamic lighting estimation.
Magic Leap famous that it’s going to proceed its collaboration with NVIDIA, and can construct upon its developments in digital twin know-how to ship higher worth to enterprise customers. With extra immersive and accessible digital twins, groups can speed up design, deepen collaboration, and remodel design critiques and different experiences.
Magic Leap’s integration with Omniverse is at present in early entry, with an anticipated common availability launch within the first half of this yr. To search out out extra about working with Magic Leap to deploy digital twins throughout enterprise and enterprise, click on here.
Picture credit score: Magic Leap
Sam Sprigg
Sam is the Founder and Managing Editor of Auganix. With a background in analysis and report writing, he has been masking XR {industry} information for the previous 5 years.
