Deepfakes Security Risks

By David Balaban, Computer Security Researcher

Deepfakes, videos where a person’s face is replaced with a nearly identical copy through the use of artificial neural networks, have instilled panic in experts since they first emerged in 2017. For instance, one of the major fears boiled down to the escalating risk of “porn vengeance” one’s ex-boyfriend can wreak. All it takes is some skills plus a computer powerful enough to frame up a naughty video featuring the guys’ ex-sweetheart. Natalie Portman and Scarlett Johansson, who has been “deepfaked” in quite a few adult videos, have even condemned the Internet altogether.

In order to combat the threat, Microsoft and Facebook have recently announced a contest to identify deepfakes more efficiently. The tech giants offer a prize pool of $10 million to researchers who can come up with the best deepfake detection algorithms. Not to mention the initiatives of DARPA (the U.S. Defense Advanced Research Projects Agency), which spent nearly $70 million on similar efforts over the past two years.

But here’s the thing: these projects aren’t really preventive anymore, because the first deepfake crime has already been perpetrated. According to the Wall Street Journal, the chief executive officer of a UK energy company was defrauded of €220,000 last March. He gave the green light to transfer the funds to a Hungary-based supplier because his boss, the head of a German parent firm, had repeatedly instructed him to do so. The truth is, a crafty scammer simply leveraged AI (artificial intelligence) technology to mimic the senior executive’s face and voice and demand that the payment be made within an hour.

The con artist used software that fully imitated a specific person’s voice, including the intonation and even German accent. The message appeared to come from the correct German address of the boss, and the CEO of the UK branch additionally received a valid-looking confirmation email with the contacts and credentials required for the money transfer. Perhaps the only giveaway was that the boss kept emphasizing how urgent the matter was, but that’s actually a typical thing in a business environment.

At the end of the day, the whole amount was gone. First, it was transferred from the Hungarian account over to Mexico, and then it was dispersed all around the world in multiple small portions. The crooks didn’t stop at that point, though. They used the same trick to request another urgent transfer so that the supply process could “get a boost.” That’s when the UK company CEO felt something wasn’t right and gave his real director a phone call.

What was happening next was a mess. The British manager started receiving calls from the impostor, then from his actual boss, and so on and so forth. Their voices sounded exactly the same. The names of the target company and its employees haven’t been disclosed due to an ongoing investigation. The thieves are still on the loose.

As a matter of fact, this story wasn’t necessarily the first instance of theft relying on deepfake AI technology. Symantec researchers claim to have spotted at least three cases where mimicking someone’s voice allowed impostors to dupe companies into wiring money to the wrong bank accounts. One of these victims lost millions of dollars this way. Moreover, based on the collected evidence, the scammers, in this case, and the ones who hoodwinked the CEO in the UK were different people. In other words, deepfake frauds are gradually becoming mainstream, and they aren’t a prerogative of some solo hacking genius.

Believe it or not, the average schoolkid will be able to carry out a similar attack in the near future. To make it work, the wannabe swindler will need to find a credulous target and gather enough video and audio samples to pull the impersonation trick.

The Google Duplex service is already capable of imitating one’s voice accurately to make phone calls on behalf of them. There are plenty of startups, most of them based in China, whose goal is to create deepfake services that will be available free of charge. The authors of different deepfake apps are even competing with their counterparts to see which products can generate a truer-to-life video of a person based on a minimum data set. Some of them even claim that soon enough, a single photo of you will suffice.

In July 2019, the Israel National Cyber Directorate (INCD) issued a security alert about a new attack vector targeting senior company executives, regular employees, and even government officials. Essentially, this is the first-ever real AI threat. According to the warning, there are applications intelligent enough to mimic your voice and accent immaculately after “listening” to you for only 20 minutes. If a video or audio of you speaking for roughly half an hour is available online, or in case someone spent a while nearby in a cafe with a voice recorder turned on, then the criminals may be able to reproduce your voice and say anything to anyone as if they were you.

While the threat landscape is continually growing, it is crucial to understand what new types of attacks appear and which security countermeasures work. As to deepfakes, there are no viable countermeasures for this type of cyber-attack so far. Vigilance is probably the only way to be a moving target. If someone calls you on the phone asking you to transfer a large amount of money, you are better off using another communication channel to double-check if the person is who he or she claims to be. You can do it via instant messengers, social media, Skype, or email. A face-to-face conversation is certainly the perfect way to dispel or confirm your doubts.

If you happen to be proficient in machine learning technologies and if you don’t mind getting a delicious slice of the $10 million pie, you can try your hand at the above-mentioned contest initiated by Facebook and Microsoft. Alternatively, you might want to launch a startup of your own that will offer a commercial deepfake detection service to the government and major companies. Such solutions will soon become indispensable.

 
Bio: David Balaban (@PrivacyPC) is a computer security researcher with over 15 years of experience in malware analysis and antivirus software evaluation. David runs the Privacy-PC.com project which presents expert opinions on contemporary information security matters, including social engineering, penetration testing, threat intelligence, online privacy, and white hat hacking. As part of his work at Privacy-PC, Mr. Balaban has interviewed such security celebrities as Dave Kennedy, Jay Jacobs and Robert David Steele to get firsthand perspectives on hot InfoSec issues. David has a strong malware troubleshooting background, with the recent focus on ransomware countermeasures.

Related:

• Applying Data Science to Cybersecurity Network Attacks & Events
• Top 7 Data Science Use Cases in Trust and Security
• The Rise of Generative Adversarial Network