iXR OSC & NDI receives 360 videos via NDI and sends hand coordinates via OSC
Workflow
1. Create real-time interactive multimedia content using your laptop.
2. Connect your Vision Pro and laptop to the same Wi-Fi network.
3. Receive hand-tracking data from Vision Pro through OSC and add interaction if needed.
4. Render the content with an equirectangular panoramic image.
5. Send the content to Vision Pro through NDI.
6. If the background color of the panoramic image is rgba(0, 0, 0, 0), it will show your environment.
7. If the background shows the virtual environment, it will not show your environment.
Note
It is recommended to use a video resolution of 4096 x 2048.
Sending 360 videos (4096 x 2048) through NDI requires a fast wireless router. It has been tested with a DECO BE11000, which had a bit of latency.
With TouchDesigner
In this tutorial, TouchDesigner is used to create real-time interactive multimedia content. You can download the example project file on my GitHub.
HDRI file is not included in the project file. Please import your HDRI file.
Open the TouchDesigner project and set the IP Address and port.
Select your NDI source.
Press Start!
