A new plan by Amazon to use artificial intelligence (AI) to dub films could spell the end for voiceover actors.
The technology giant has patented a system that would see computers learn the voices of Hollywood stars such as Tom Cruise by studying their films.
Amazon’s computer systems could then automatically generate foreign language versions without any need for voiceover actors to dub the audio.
The company used the example of “The Last Samurai” as an example use of the technology in its patent filing.
By analysing how Cruise sounds in other films such as “Mission Impossible” and “Rain Man,” Amazon could recreate his lines from “The Last Samurai” in French or Polish while still sounding recognisable.
Amazon could use the technology to automatically convert thousands of foreign films and television shows into English to stream on its Amazon Prime Video service.
Major advances in creating realistic-sounding speech from famous people could also lead to renewed fears of “deepfakes” where technology is used to create videos that show politicians and other public figures saying things they never actually said in reality.
Poor quality dubs of films and television have often been criticised for ruining viewers’ enjoyment. Netflix scrapped the English dubs for the first two series of Spanish show “Money Heist” in 2019 and recorded new versions after some viewers had complained about poor acting.
“Previous attempts to automatically generate a localised dub for videos have had limited success because of the lack of diversity in available voices, accents, and other factors like age, gender, or speaking mode, which can result in, for example, relying on an adult voice for dubbing a child character,” Amazon wrote in its patent filing.
Last year, Amazon employees published an academic paper in which they described training AI on 47 hours of speech recording so that it could automatically dub films. They concluded that the system required more work on aspects such as “lip synchronisation.”
If successful, the technology could also be used to create recognisably famous voices for other Amazon products such as its audiobooks or even its Alexa virtual assistant.
However, the existence of a patent granted to Amazon doesn’t guarantee that the company is going to start releasing AI overdubs any time soon. Technology businesses often use patent filings for marketing or to prevent their rivals from developing certain projects.
Amazon has previously patented unlikely projects including a handle to allow people to windsurf using a drone and a miles-long whip that would fling satellites into space.