How Beyond Sports brought live commentators into its virtual broadcasts, replacing a complex motion-capture setup with a more flexible VR solution.
Great commentators help viewers connect with a match. They do more than narrate, they provide context, explain complex situations, help newcomers understand the intricacies of the sport. Just as importantly, they bring humour and personality to the broadcast.
Even though our alternative broadcasts of sporting events, or alt-casts as we call them, have distinct storylines and amazing visuals, they don’t feel complete without commentators. The challenge we faced was that cutting to a real-world studio would break the immersion, so we needed to bring the commentators into the virtual world without losing the spontaneity and personality of a live broadcast.
That led us through a multitude of solutions from full-body motion capture to consumer VR hardware and, eventually, a networked system capable of supporting multiple commentators in the same virtual broadcast.
The Past
The first solution proposed by our clients was to work with a third-party studio to create real-time 3D representations of the commentators. They used a common full-body motion-capture solution (mo-cap): the commentator would put on a jumpsuit with tracked markers, allowing every limb to be tracked in real time.
The full-body motion capture worked, but it was expensive and cumbersome. Every broadcast required a specialist setup; a dedicated studio and commentators dressed in motion-capture suits. It gave us the result we wanted on screen, but the setup made it difficult to reproduce, scale or take into new production environments.
That’s when we got our hands on the relatively new Meta Quest Pro. It had hand tracking, and we could estimate the position of the feet in software, allowing the avatar to move naturally without tracking the entire body. Which, to be honest, wasn’t anything new or exciting. Many VR games and applications did this already. But Meta Quest Pro did what no other consumer-grade headset had done before: face and eye tracking. That was the difference-maker. It allowed the virtual commentator to capture more than body movement; it could reproduce expressions and eye direction, bringing more personality into the performance.
The Present
Creating a custom avatar with Meta’s software development kit – the tools used to build software for its hardware – was relatively straightforward. The harder task was keeping the experience running smoothly. VR creates depth by generating a different view for each eye. Our already demanding simulation therefore had to generate three views of the scene: one for the broadcast and two for the headset.
If you've ever had the unfortunate experience of using a poorly performing VR application, you know the problem: a low frame rate can cause nausea. Not to mention that sub-60 frames per second would not be acceptable for broadcast standards.
Our solution borrowed an idea the gaming industry has used for decades: connect multiple computers over a network and divide the work between them. One machine produced the broadcast, while another handled the VR headset. Rather than duplicating the entire simulation, we only needed to send the commentator's movements between them. This separation helped protect the quality of the broadcast while giving commentators a responsive experience in the headset. And if the performance became an issue on the VR computer, we could reduce the visual fidelity of the environment without changing what viewers saw. No one besides the talent using the headset would see the simplified environment.
This network solution worked great. But why stop there? Why not put “multi” in the multiplayer? Now that we had a networking solution, we could support multiple commentators simultaneously! That meant we no longer had to treat a virtual commentator as a single-person technical demonstration. We could recreate the conversation, chemistry and spontaneity of a real commentary team inside the same virtual environment.
Indeed, that’s what we did for our very first VR-powered broadcast.
The Future
When Meta discontinued Quest Pro, we faced a challenge familiar to anyone building around emerging technology: the hardware roadmap had changed, but our broadcast requirements had not. elying on discontinued hardware was a risk we could not afford as our broadcasts continued to evolve. Alternative headsets gave us a way forward.
Moving to a different headset wasn't simply a matter of swapping one for another. Meta uses their own proprietary software to handle avatars, so we needed to support both that software and the alternative headset’s approach. We used the transition to remove the dependency on a specific brand altogether. The result is a device-agnostic system that gives us greater control over future hardware choices and reduces the risk of one discontinued headset disrupting the broadcast workflow. If everything goes according to plan, commentators will be using different and up-to-date headsets instead of the obsolete Quest Pro in our next stream (and trust me; you won’t want to miss it — it is Star Wars!).
What This Changed
What began as a way to replace a cumbersome motion-capture setup has become a more flexible way to bring live talent into a virtual broadcast. We replaced a clunky mo-cap solution that only worked with a green screen with characters fully immersed in the 3D world. The headset hardware costs a fraction of a professional full-body motion-capture setup, and requires considerably less preparation from the on-air talent. Simply put: putting on a headset is far easier than getting into a full-body mo-cap suit. For our broadcast teams and partners, the benefit is a setup that is easier to deploy, less dependent on specialist hardware and flexible enough to support multiple commentators inside the same virtual production.
We've already come a long way and we have many more exciting features in mind. When you think about it, this is the worst version of the VR commentator you will ever see. As the hardware improves and our software develops with it, virtual commentators will become more expressive, more natural and easier to bring into increasingly ambitious alternative broadcasts. We are finding new ways to make live talent part of the experience itself. The future is bright, fun, and immersive for alternative sports broadcasts!
Written by Anton Semchenko
