A deep dive into deep fakes

By Christopher Dunne | 28 Jun 2019

Deep fakes, which refer to AI technology that’s used to alter video or audio content, have received a lot of interest lately – you may have seen the Mark Zuckerberg deepfake video where he seems to be talking about controlling and stealing billions of people’s data – but while the downside to altering such content is stealing column inches, it’s worth considering the benefits of this technology too.

In this episode of the Radiolab podcast, they play audio from the Adobe Max 2016 ideas forum where they were premiering VoCo, which is described as “Photoshop, but for audio”. The host manages to manipulate audio from co-host Keegan-Michael Key to make him say something he didn’t. He even types small phrases into VoCo and Key magically says them.

Adobe’s thinking behind the software is for use in post-production. For instance, if an actor needs to say something else in a scene, then the studio could easily and cheaply get the actor to do so, without needing to hire them for a day’s call back.

It’s been over two years since its inception and Adobe still hasn’t released VoCo to the masses, with some stating that the reasons behind this are that it hasn’t yet worked out a way of watermarking or making it clear that the content isn’t real.

Taking this technology a step further, some very clever people from Stanford University have managed to not only copy audio, but copy a person’s face too. Using machine learning and multiple images, the computer can manipulate the frames in between what’s real and not.

In most of the examples of this technology in use, the audio is pretty close to perfect. Video isn’t quite there yet, but with time it’ll almost certainly improve. Adobe is yet to release VoCo, but until that happens, other companies like LyreBird and WaveNet have come to market with similar technology and propositions. And over on the mobile technology front, Samsung has created an app that can do a deep fake video by only feeding it one single image. We might not be too far from a future where anything and everything we see on TV could have been doctored!