This package provides a H264 recorder in Unity3D sending data over RTSP.
To repeat what is the main purpose of this package:
Vizario H264RTSP is a lightweight library to enable RTSP streaming FROM iOS or Android devices (respectively standalone apps or Editor Win64 and OSX) to a local PC or remote server. The library allows you to record the content as rendered by a Unity game camera and push the encoded H264 bitstream to a server instance or serve it directly from the device within a (WIFI) environment.
This is a list of findings over the last 2 years on using the package. The list will be extended over time with new information.
- Q: Does it work within VR setups?
- A: Yes, it was tested to work within Oculus Quest 2 and Oculus Quest Pro.
- Q: Can it do audio as well?
- A: No and it will not any time soon. The Unity audio setup is far too complex to implement this feature.
- Q: Is there a test package available before I buy it?
- A: No, but if you send me a nice email with some info about your use case, I can probably make something work.
- Q: There is something not working as I expected. What should I do?
- A: Send me an email! There are always cases which might work one or the other way, but it heavily depends on your setup.
- Q: How are the internals working and where are the bottlenecks, particularly concerning a VR case?
- A: If you are working in Unity3D, you are working in a 3D game engine. It is not a 2D player app, which accesses the “surface” of your device directly. Therefore, there are copy operations involved that essentially bring the decoded image content into a texture surface, which is then transferred to the GPU. To quote some of the discussions I had previously (thx to Kyle Johnson for testing this out):
I’ve gotten streaming to work on the Quest just fine. It wasn’t hard at all. I tried all 3 streaming modes, as well as both render texture and camerainput. Everything seems to work fine. On the camerainput modes, you do have to set the InputFormat to COLORSPACE.RGBA32, but otherwise, it’s fine (I had to do that on my android phone as well). Vulkan or GLES3 work fine.
720p30 streaming is doable. It definitely hits the frame rate a bit on any real scene (the spinning cube stays at 72fps, but I tried a few real scenes and it couldn’t hold. Performance metrics are really hard on the Quest 2, but I’d say you take about a 50% hit on performance (half of what you could previously render at 72fps) when streaming at 720p30. 1080p30 is too much, even for a basic scene to hold at 72fps. 480p is better.
Generally, performance seems on par with Unity’s WebRTC package, which also works on Quest with similar limitations. The latency of the latter is better, but the setup is quite painful by contrast because you have to deal with the signaling. Also, the ability of your package to run as a server and just watch on any device with VLC player is pretty cool.