Accountants fear generative AI opens frontier for cyber criminals
Four in five finance professionals are concerned hackers are using generative AI to create increasingly sophisticated and targeted attacks, according to research.
Novel social engineering attacks or email scams are becoming increasingly sophisticated by using linguistic techniques, increased text volume, correct punctuation and no links or attachments, specialists say.
This trend suggests that generative AI such as Chat GPT is providing an avenue for threat actors to craft targeted attacks at speed and scale, according to cyber defence specialist Darktrace.
A recent study by Darktrace reveals 78 per cent of finance professionals are concerned that hackers may use generative AI to create scam emails indistinguishable from genuine communication.
Advancements in generative AI mean that email scams can now be written in the exact language and tone of the voice of the person they’re trying to impersonate, such as the CEO for example. It may even reference a personal anecdote or joke, said the cyber defence specialist.
The Darktrace research indicated that 61 per cent of people look for poor use of spelling or grammar as a sign that an email is fraudulent but these types of malicious cyber campaigns contain no mistakes.
“The spelling and grammar are perfect, it has personal information and it’s utterly convincing but your CEO didn’t write it. It was crafted by generative AI, using basic information that a cyber-criminal pulled from social media profiles,” said Darktrace.
Given the rise of AI-powered attacks, Darktrace said companies can longer put the onus on humans to determine the veracity of communications.
“This is now a job for artificial intelligence,” said the cyber defence company.
“Self-learning AI in email, unlike all other email security tools, is not trained on what ‘bad’ looks like but instead learns you and the normal patterns of life for each unique organisation.
“By understanding what’s normal, it can determine what doesn’t belong in a particular individual’s inbox. Email security systems get this wrong too often, with 79 per cent of respondents saying that their company’s spam/security filters incorrectly stop important legitimate emails from getting to their inbox.”
Threat actors are also rapidly exploiting the news cycle to profit from employee fear, urgency, or excitement.
“The latest iteration of this is the collapse of Silicon Valley Bank and the resulting banking crisis, which has presented an opportunity for attackers to spoof highly sensitive communication, for example seeking to intercept legitimate communication instructing recipients to update bank details for payroll,” said Darktrace.
Darktrace reported that 73 per cent of employees working in financial services organisations have noticed an increase in the frequency of scam emails and texts in the last six months, based on its global survey of 6,711 employees across the UK, US, France, Germany, Australia, and the Netherlands.
In the past six months, 70 per cent of employees in the survey said they had noticed an increase in the frequency of scam emails and texts.
Around 3.4 billion phishing emails get delivered every day, according to estimates.
Human error still an issue
The Darktrace research indicated that nearly one in three global employees have fallen for fraudulent email or text in the past.
Finance and HR professionals were the most likely industry to send an email to the wrong person, compared to other industries.
Almost half or 48 per cent of HR and finance professionals stated that they had sent an important email to the wrong recipient by mistake.
In addition, one in four finance professionals reported having fallen for a phishing or other fraudulent email or text compared to 19 per cent of all Australians.
The vast majority of finance professionals said they were also concerned about the amount of personal information available about you online that could be used in phishing and other email scams.
In the survey overall, nearly two in five employees said they had sent an important email to the wrong recipient with a similar looking alias by mistake or due to autocomplete.
This number rises to over half or 51 per cent in the financial services industry and 41 per cent in the legal industry, adding another layer of security risk that isn’t malicious.
About the author
