View this article published in IEN magazine.

To the generation that remembers competing for the high score on Galaga at the local arcade, it may come as a surprise that competitive gamers now create a multi-billion-dollar industry. Today, more people pay to watch rather than game themselves. It’s the equivalent of paying to watch over your friend’s shoulder as they blast alien invaders, except online, with thousands of viewers looking over the gamer’s proverbial shoulders. We aren’t in the arcade anymore: welcome to eSports.

A leader in the gaming industry broadcasts up to 100 videos a month – from live streams to produced video – across multiple platforms. To provide quality internal and external programming, regardless of hearing ability, they needed the ability to add subtitles or captions to all that content.

The media support and services group was tasked with finding a captioning solution that could handle the quantity and unique demands of the gaming industry (jargon, complicated gamer tags, and more). Varying entities within the company – including development and support studios, platform engineering teams, marketing, even HR recruiting teams have a hand in this gaming group and need media support—and captioning.

Thinking Outside the Gamer Box

“We want to put content out there that will be relevant to people that are hearing impaired. We’re responsive to the community at large and our customer base.” said an Executive Producer at the company, explaining the decision to provide captioning and subtitles for their content.

The gaming industry throws some wrenches into what would otherwise be considered smooth captioning. Video game live streams feature multiple people speaking, often simultaneously. Gamertags often contain complicated symbols, non-English words, or misspellings, for example, g@m3rt@gz. Even the system itself breaks standard syntax sometimes.

With so much content coming in from various departments and gamers, the Executive Producer needed an automated and hands-off captioning solution. He found it at the National Association of Broadcasters (NAB) convention in 2017, at the Link Electronics booth: the ACE-2000 Automated Captioning Engine.

ACE up the Sleeve

Unlike other automated captioning engines that operate as part of a subscription, the Executive Producer and the company’s Chief Engineer could caption off the shelf with the ACE-2000 without having to pay out of pocket continually. Having a solution in-house is a significant return on investment at a rate of at least 50 videos a month.

The ACE-2000 is a trustworthy speech recognition system that doesn’t rely on a specific voice. It can create captions from any speech. It converts speech to text using a cutting-edge computational linguistics program. It sends that text out as captioning data to a caption encoder, either by RS 232 (Serial) or TCP/IP. The ACE accepts audio by de-embedding the first two audio channels from an SDI or HDMI video input or through balanced AES or analog audio. In addition to http, udp, mmsh, the ACE can accept audio using rstp, rtmps, and most commonly used codecs.

When the ACE-2000 arrived, the support team immediately put it through its paces. They ran multiple live video sources from gamers through the system to test it. They then took the text files created by the system and read along while listening to the videos without making corrections for any company-specific words. It was 92% accurate, right out of the box. The speaker turn function was also on point, allowing readers to see when someone new was speaking by displaying > >. Speaker ID also displayed that person’s name.

Next, they began improving system performance by adding unique proper spelling and naming. “Learning” is accomplished by analyzing text files to grow the dictionary. ACE uses a web crawler scheme to scan websites containing words unique to either the company or the gaming industry. The system searches through the sites and adds words to the dictionary, thus expanding its vocabulary. As long as the unit is connected to the internet, it will automatically update the dictionary, extending the machine’s capabilities even further.

The ACE-2000 was this gaming company’s first dive into closed captioning and subtitling, yet the team found the learning curve minimal. Most of the audio they work with is embedded, though non-embedded audio can be brought in as analog with a breakout cable.

Additionally, the support team at Link Electronics was ready to provide excellent customer service, offering 24/7 live support with an engineer rather than an answering service.

The support team at Link was very responsive—there was an excellent back and forth interaction with them getting the system set up and testing it, the Executive Producer said.

Live Gamers on Air

Live shows from gamers are captured live with automated closed captioning, then brought offline after the fact. Recently, they held a live event with more than 24-hours of nonstop audio and video using the ACE-2000 for accessibility channels so viewers could watch and read what was happening as it unfolded.

Live events are scheduled ahead of time using the web-based Graphical User Interface (GUI) systems, whereas offline events are processed on an as-needed basis. Since this group supports media services for many different entities within the company, it was important that employee-deposited files remain secure and are not shared or viewable by other groups. Link Electronics worked with them to create a two-folder system, wherein users drop their content in a “watch folder,” and their results show up in a separate “results folder.”

Online or live captioning is performed as the event is happening. Offline or post-production captioning is completed by processing media files through the ACE-2000 after the event.

G@M3 OVR

While the automated system is hands-off and is actually teaching itself new vocabulary, there are still errors processing gamer tags and acronyms. To error is human and non-human, apparently.

Despite the occasional error, the ACE-2000 has allowed this gaming company to provide accessible content to all users efficiently and cost-effectively. It’s a small act from one of the industry leaders in gaming, but an act that welcomes all users regardless of ability.

As the Executive Producer puts it: it’s just the right thing to do.