There''s a good chance you''ve ever seen a Star Wars episode on Disney Plus, whether it''s obvious or not. While Mark Hamills performed in both The Mandalorianand The Book of Boba Fett, as well as for an as-yet-unidentified character in Obi-Wan Kenobi, Lucasfilm has asked that Respeecher keep the name of the character a secret for the time being, however there is certainly no shortage of potential applicants.
Polygon spoke with Respeecher CEO Alex Serdiuk to understand a process that for many fans no doubt borders on sacrilege. Serdiuk emphasized the human element behind the platform itself from the start. So we can [studios] to scale voices, age voices, and even resurrect voices for certain projects.
So far from the mental image he is smuggled up by terms like artificial intelligence and voice cloning that of a sound engineer navigating lines of conversation through a computer algorithm that then removes audio files Respeechers work on Star Wars is surprisingly performance driven. While Darth Vader himself might be more machine now than man, if the Ukrainian company is supplying his voice (and remember, we said if) the essence of the characters voice is still very much flesh and blood.
There is no artificial intelligence available yet, and I doubt it would exist, allowing us to use it only on a turnkey basis to produce the performance we desire.  We need another human voice, if we want to, so that all the inflections, the accent, the speech style, and pace that AI is not good at achieving, according to Serdiuk.  It takes all the performance, all the acting from what we call a source voice, and then we complete the transformation.
Aside from this, a Respeechers pipeline allows actors like Hamill to record different takes just as they would on an actual set, which the company''s experts may later modify at their end based on notes from showrunners like Jon Favreau or Deborah Chow.
Typically, studio projects and films may have a tens of thousands of takes for each line, and that means that we would need to convert all of those [into the younger voice], send them back, and maybe send different versions because we used to train different models with different setups. Can you make [a line reading] sound a bit more like they ask? and we would work to make it sound a bit more like they ask.
So, is the ultimate intention to reimagine a traditional performance through nontraditional means in the case of The Mandalorianor The Book of Boba Fett, as though Hamills lines were somehow transmitted directly from the set of 1983s Star Wars: Return of the Jedi? The objective is to make it sound like it was recorded yesterday in the studio by the target voice themselves, according to Serdiuk.
However, there is still the possibility with voice cloning technologies like Respeechers that the synthesized performance it produces will be artificial, even if viewers aren''t certain why. Serdiuk also admits that the Hamills de-aged voice is significantly improved than in The Mandalorian, thanks to small yet significant improvements in how Respeechers'' AI model was developed to emulate the actors'' vocals. At the same time, the CEO is quick to note that while many fans complained that
The Mandalorian and The Book of Boba Fett were all so impressive considering the quality of Respeecher''s legacy assets. So we had something from tapes, some old ADR recording, something from a video game, Serdiuk says. This might be the major challenge in many projects that involve aging or resurrecting [performers voices], because a lack of data and the high quality of data create additional hurdles in making it sound good.
While the CEO of Respeecher believes that overcoming these data-related challenges has been worthwhile, as long as industry giants like Lucasfilm have begun. [We] started with the idea of putting a synthetic speech on the level where it would go through sound engineers and Hollywood studios and land in large productions. So, when they accept our sound, when they say something good about the sound we were able to produce, and that is a very complex and lengthy technical challenge, it really encourages us and assists us grow.
Serdiuk claims that the growing acceptance of voice cloning technology might result in not only making studios stand-in for deceased actors. For example, would Rogue One: A Star Wars Story in which Guy Henry imitated the voice of Grand Moff Tarkin as a grandma? Not necessarily, according to Serdiuk, who sees Respeechers voice cloning technology as one of several viable solutions at disposal.
While Serdiuk said that if Lucasfilm ever called upon Respeecher to recreate the voice of a dead actor, it would only do so with the approval of the actors estate. The fact that Lucasfilm is a repeat customer suggests that the company has made significant in-roads for voice cloning techniques.
Serdiuk argues that it will enable smaller film and television studios and video game developers to expand their budgets even further. He also talks about democratizing the technology so that smaller companies may cut expenses further. He also suggests that it is experimenting with a voice actor who has lost their voice to allow them to perform again.
Serdiuk has lost sight of what it takes for Respeecher to be part of a certain space opera set a long time ago in a galaxy far, far away. It''s something special. From a technical perspective, they have been disrupting the entertainment industry since the beginning, and the way they do their movies is fantastic. So, it''s a huge honor to have a relationship with those people and learn from them.