Creating closed captioning for live local news is error-prone and often lags far behind the speaker. To improve accuracy and speed, Sinclair is using IBM Watson Captioning across its local channels.
IBM Watson Media announced that Sinclair Broadcast Group is rolling out Watson Captioning across its local TV stations. The first channels go live today, with the rest following by the end of the month. The technology will handle the closed captioning tasks for the stations’ live news broadcasts, including breaking news, weather, and live sports segments.
Live news is challenging
Live news programming is particularly difficult for captioners to translate accurately. The many references to local names, locations, and events may be unfamiliar to a human captioner that isn’t located in the region. Local accents and idioms can be problematic. These challenges
frequently can result in errors in translating the spoken word and can yield closed captions that lag far behind the speaker.
According to David Kulczar, Senior Product Manager, Watson Video Analytics at IBM Watson Media, IBM worked with Sinclair to enhance Watson Captioning from its VOD roots to take on the live local news challenge. He says that much is known about the news broadcast before it airs and that this information can be used to train the AI in advance:
“It’s well-understood content that they <the broadcaster> generally know in advance. Often, we know what’s going to be said, and it’s delivered by a clear speaker with an accent that’s very easy to understand.”
Training the system
There are a variety of ways to train Watson Captioning. Local stations can feed the system news copy, and type in or batch-process specific words and phrases. Mr. Kulczar says the station can also leverage past broadcasts:
“If I had stories from the past ten years for my local station in Austin, Texas, I could feed all of that in beforehand. So, Watson Captioning would have local celebrities, local politicians, all that stuff ahead of time.”
If the system makes a mistake, a human operator can correct it, and the AI learns from the mistake. However, Mr. Kulczar doesn’t expect somebody to monitor the captions all the time. Many stations will carefully monitor the system to start. However, as errors get corrected, after a week or so he believes they will gain confidence in the system and allow it to operate autonomously. In trials, he says, stations are already seeing considerably more accurate results than they get with human captioners.
There are other advantages to the AI approach for closed captioning. Generally, the text is 3 seconds or less behind the spoken word, much faster than traditional closed captioning systems. As well, once a station owner has trained the system, it continues to build on its knowledge and becomes more accurate over time.
Dealing with adverse conditions
All the training that Watson Captioning receives prepares it for expected accents, words, and sentence structures. However, news anchor teams frequently go off script, and sometimes way off. In these cases, Mr. Kulczar says the system will translate what is said accurately, even if it is sentence fragments and disconnected phrases:
“If they are bantering, it’s going to be stuff that’s well known to the system. So, it is just going to translate it accurately.”
There are still situations where the AI can falter. For example, if there is a lot of background noise accuracy can fall. As well, if a speaker switches between English and another language the AI may falter. For example, the system delivered the captions for a mayoral town hall meeting given by New York’s Bill de Blasio, Mr. Kulczar says the AI coped with all his off-the-cuff remarks:
“The only thing we saw that was slightly out-of-bounds was de Blasio tends to break into Spanish. With Spanish, it <Watson Captioning> tried to translate Spanish as English because we didn’t tell it he was going to be speaking Spanish.”
However, IBM is enhancing the AI to identify a different language. It will then indicate in the closed captioning when the speaker is not speaking English. A version of the captioning product for Spanish-speaking TV stations is currently under development.
Why it matters
Closed captioning of local news, weather, and sports segments can be very challenging for captioners.
This can be annoying to those reading the closed captioning. It can also be dangerous in emergency situations.
Sinclair is rolling out IBM Watson Captioning to try to improve accuracy and speed of its closed captions.