Facebook Announces Results of 'Deepfake Detection Challenge': 65% Accuracy

[SaM]

An anonymous reader quotes Engadget:

In September of 2019, Facebook launched its Deepfake Detection Challenge – a public contest to develop autonomous algorithmic detection systems to combat the emerging threat of deepfake videos. After nearly a year, the social media platform announced the winners of the challenge, out of a pool of more than 2,000 global competitors…

Facebook spent around $10 million on the contest and hired more than 3,500 actors to generate thousands of videos – 38.5 days worth of data in total. It was the amatuer, phone-shot sort you’d usually see on social media rather than the perfectly-lit, studio-based vids created by influencers… The company then gave these datasets to researchers. The first was a a publicly available set, the second a “black box” set of more than 10,000 videos with additional technical tricks baked in, such as adjusted frame rates and video qualities, image overlays, unrelated images interspersed throughout the video’s frames. It even included some benign, non-deepfakes just for good measure.

On the public data sets, competitors averaged just over 82 percent accuracy, however for the black box set, the model of the winning entrant, Selim Seferbekov, averaged a skosh over 65 percent accuracy, despite the bevy of digital tricks and traps it had to contend with… While the company does intend to release these models under an open source license, enabling any enterprising software engineer free access to the code, Facebook already employs a deepfake detector of its own. This contest, Facebook CTO Mike Schroepfer explained, is designed to establish a sort of nominal detection capability within the industry… “A lesson I learned the hard way over the last couple years, is I want to be prepared in advance and not be caught flat footed, so my whole aim with this is to be better prepared in case [deepfakes do] become a big issue,” Schroepfer continued. “It is currently not a big issue but not having tools to automatically detect and enforce a particular form of content, really limits our ability to do this well at scale.”