“ChatGPT with no ethical boundaries”: WormGPT fuels AI-generated scams


Cybercriminals are increasingly using generative AI tools such as ChatGPT or WormGPT, a dedicated malware model, to send highly convincing fake emails to organizations, allowing them to bypass security measures.

A new wave of highly convincing fake emails is hitting unsuspecting employees. That’s according to British computer hacker Daniely Kelley, who has been researching WormGPT, an AI tool optimized for cybercrimes such as Business Email Compromise (BEC). Kelley is referring specifically to his observations on underground forums.

In addition, special prompts known as “jailbreaks” are exchanged to manipulate models such as ChatGPT to generate output that could include the disclosure of sensitive information or the execution of malicious code.

“This method introduces a stark implication: attackers, even those lacking fluency in a particular language, are now more capable than ever of fabricating persuasive emails for phishing or BEC attacks,” Kelley writes.


AI-generated emails are grammatically correct, which Kelley says makes them more likely to go undetected, and easy-to-use AI models have lowered the barrier to entry for attacks.

WormGPT is an AI model optimized for online fraud

WormGPT is an AI model specifically designed for criminal and malicious activity that is shared on popular online cybercrime forums. It is marketed as a “blackhat” alternative to official GPT models and advertised with privacy protection and “fast money”.

Similar to ChatGPT, WormGPT can create convincing and strategically sophisticated emails. This makes it a powerful tool for sophisticated phishing and BEC attacks. Kelley describes it as ChatGPT with “no ethical boundaries or restrictions”.

WormGPT is based on the open-source GPT-J model, which approaches the performance of GPT-3 and can perform textual tasks similar to ChatGPT, as well as write or format simple code. The WormGPT derivative is said to have been trained with additional malware datasets, although the author of the hacking tool does not disclose these.

Ein Screenshot mit weißem Text auf rotem Hintergrund.
A phishing email generated with WormGPT. | Image: Daniel Kelley, via SlashNext

Kelley tested WormGPT on a phishing email designed to trick a customer service representative into paying an urgent bogus bill. The sender of the email was the CEO of the targeted company (see screenshot).


Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top