Google has announced its first direct speech-to-speech translation system called “Translatotron” that can convert verbal communication from one language to another while maintaining the speaker’s voice and tempo. Translatotron is based on a sequence-to-sequence network which takes source spectrograms — a visual representation of frequencies — as input and generates spectrograms of the translated content in the target language, Ye Jia and Ron Weiss, software engineers at Google Artificial Intelligence (AI) wrote in a blog post.
The model makes use of two other separately trained components — a neural vocoder that converts output spectrograms to time-domain waveforms and a speaker encoder that can be used to maintain the character of the source speaker’s voice in the synthesised translated speech. For now, the results of Google’s demonstration of the translation system lag behind a conventional cascade system, but the search engine giant said it has “demonstrated the feasibility of the end-to-end direct speech-to-speech translation”.
“By incorporating a speaker encoder network, Translatotron is also able to retain the original speaker’s vocal characteristics in the translated speech, which makes the translated speech sound more natural and less jarring,” Jia and Weiss said.
The company said that Translatotron is the first end-to-end model that can directly translate speech from one language into another.
“We hope that this work can serve as a starting point for future research on end-to-end speech-to-speech translation systems,” the blog post noted.
Lately, Google has been working aggressively on its translation systems. Adding more languages to its real-time translation feature, earlier in 2018, the company introduced accents in Google Translate with a variety of languages in region-based pronunciations.
In February, Google extended the “Interpreter” mode on its Assistant to Home speakers and other third-party devices to help users have real-time conversations with compatible devices and translate them into 26 listed languages.
