In today’s highly competitive media landscape, the on-screen presentation of a media company’s programming can make all the difference between ratings success and failure.
That’s why many have embraced virtual sets as a cost-effective way of improving their on-air look by either replacing an existing physical studio set or expanding the studio space without physically building any new walls.
Virtual sets have also proved to be a valuable tool to help presenters tell complex stories in a way that is more understandable to the average viewer. This increases program loyalty and ratings by leveraging computer-generated graphical elements – both in front of and behind the presenter. It also allows the on-air reporter or host to walk through 3D space as the camera moves with them and intuitively interact with virtual charts or other types of infographics.
Raising the bar on creativity
The new generation of virtual sets can be configured in a variety of ways, depending on usage, need for flexibility and budget. A virtual set requires a graphics artist and a 3D computer graphics software package but can be customized for different shows and repurposed in a matter of seconds. In addition, less crew is required — in most cases there’s only one operator per program production — and broadcasters can start small, perhaps with a single camera, and then grow the virtual set and add more physical and virtual cameras as required.
With virtual sets, physical cameras can move in 3D space, while the image of a virtual camera is rendered in real time from the same perspective. Therefore the virtual scene can adapt at any time to the camera settings. This is what differentiates a virtual set from the traditional technique of chromakeying. And a virtual studio does not require any postproduction work because it is rendered in real time. In addition, virtual sets can be combined with physical sets, adding an extra layer of depth to the physical studio space.
Augmented reality
While the addition of a virtual background can place studio presenters in any location in the world, augmented reality (AR) graphics are rendered in front of the on-air talent and help liven up the set in ways physical set pieces cannot. When used in combination with each other, as many media companies are now doing, the on-air effect can be captivating. AR systems are used to place virtual 3D graphic elements in the physical world. These elements are arranged three dimensionally and attached to a live video signal, then output as a combination of live video and graphics. The technology takes the idea of virtual graphics from virtual sets but places them in the front of the presenter or in the foreground of a studio. A big benefit is that these graphical elements can then be used in tandem with a physical set, adding an extra layer of depth to the studio, or extending the studio space, without the need for a chromakey or green screen wall. The technology can even be used for outdoor studio environments, such as during live sporting events.
Camera tracking
Virtual sets rely on camera movement data that must be carefully tracked with a real-time rendering and compositing system that creates these highly realistic environments. Camera tracking is critical in order to convey an accurate on-air impression of a presenter walking around and interacting with a virtual world. The tracking data from the camera and the rendering of the graphics are composited and output as a final image. Keyed graphics can be included on top if necessary.
Currently, there are four types of camera tracking systems used with virtual sets: Mechanical, Optical, Image-based, and trackless – though this not technically a tracking solution.
Mechanical tracking
Mechanical Camera Tracking Systems consist of specialized hardware, typically a camera tracking head and a lens encoder that can be installed on cameras and on pedestals, in the studio or in the field. These mechanical tracking systems work in tandem with a compositing server for each camera (which is controlled through the video production switcher in the control room or OB van). Mechanical tracking systems offer the best accuracy and can cope with any camera movement as long as the camera is put on a tripod, a crane or rails.
Optical tracking
Another type of motion tracking system uses Optical Tracking technology to capture the camera data and the location of the talent within the virtual set. This technology comes from the motion capture capabilities originally developed for the film and gaming industry. A typical mo-cap system uses a series of specialized illumination units to capture reflective, moving markers with a set of cameras. The technology tends to be more expensive than mechanical tracking technology, but it also provides a lot of flexibility to add extra cameras and tracked objects to the set. In addition, it allows for greater freedom of movement for the on-air talent within the virtual set. Therefore, multiple cameras and objects are trackable simultaneously using the same system.
Image-based tracking
The third, and most recent entry into the virtual space, is Image-based tracking. With an image-based tracking system, there are no tracking heads on the camera, and no additional gear is required at the venue or on the camera. The technology allows for tracking a camera using the television camera image (pixels) alone to calculate the movement of the camera. A computer application does the tracking calculations based on the recorded or live TV footage. The camera can move around in the studio or around the sports field.
The usage of image-based tracking can be limited by a number of environmental factors. These include extreme light conditions, inclement weather or even all green environments like sports fields where no contrasting features can be detected.
Trackless solutions
Virtual sets can also be used without a tracking system. Vizrt’s real-time 3D rendering system, Viz Engine, includes 16 virtual cameras. Each of these can be applied to a physical camera in the studio and used as a texture in the virtual set design. Movement in the virtual set, such as zooms and pans can be done by creating a transition with these textures in the software.
The business case for virtual technology
The same virtual set and AR technology that is used in a confined space of a virtual studio can also be used outdoors with equal success. Virtual graphics are increasingly being used on the outdoor playing field, for live sports analysis as well as for commercial sponsorships. The players are keyed above the graphics so that the virtual graphics appear as part of the field. Virtual ads can also replace the physical ads in an outdoor stadium or indoor arena. The technology allows media companies to customize and display different ads for different viewer locations.
Minimal financial investment
Building physical sets is an expensive undertaking for a media company. By using a virtual set, a single studio can be used for multiple productions, changing very easily the design of the set from show to show. It allows shows to have their look periodically updated with minimal financial investment.
With their ability to transport viewers to places they have never been before, virtual studios bring flexibility and cost-effective yet high production value to any type of media company. Take, for example, what the BBC did during its 2015 election coverage in the UK. The broadcaster virtually re-created the government’s headquarters at 10 Downing Street, complete with individual seats displayed to help illustrate election results projections and how the balance of power might shift. It was a marvel for its artistry and realism as much as for the information it imparted, and the BBC’s high ratings reflected this.
Separating from the competition
Virtual sets and AR graphics have become a powerful tool for program differentiation, creating the illusion of being in a bigger studio than might actually exist while making the scene more attractive on air. The use of both technologies also results in increased program production, something very broadcaster wants to improve upon.
Versatility, cost effectiveness, ease of installation and minimal staff requirements are all key benefits of virtual set and augmented reality systems. That’s why the number of media companies using virtual sets and AR graphics continues to increase with time.
