ChatGPT is enabling script kiddies to write functional malware

Since its beta launch in November, AI chatbot ChatGPT has been used for a wide range of tasks, including writing poetry, technical papers, novels, and essays, planning parties, and learning about new topics. Now we can add malware development and the pursuit of other types of cybercrime to the list.

Researchers at security firm Check Point Research reported Friday that within a few weeks of ChatGPT going live, participants in cybercrime forums—some with little or no coding experience—were using it to write software and emails that could be used for espionage, ransomware, malicious spam, and other malicious tasks.

“It’s still too early to decide whether or not ChatGPT capabilities will become the new favorite tool for participants in the Dark Web,” company researchers wrote. “However, the cybercriminal community has already shown significant interest and are jumping into this latest trend to generate malicious code.”

Last month, one forum participant posted what they claimed was the first script they had written and credited the AI chatbot with providing a “nice [helping] hand to finish the script with a nice scope.”

The Python code combined various cryptographic functions, including code signing, encryption, and decryption. One part of the script generated a key using elliptic curve cryptography and the curve ed25519 for signing files. Another part used a hard-coded password to encrypt system files using the Blowfish and Twofish algorithms. A third used RSA keys and digital signatures, message signing, and the blake2 hash function to compare various files.

The result was a script that could be used to (1) decrypt a single file and append a message authentication code (MAC) to the end of the file and (2) encrypt a hardcoded path and decrypt a list of files that it receives as an argument. Not bad for someone with limited technical skill.

“All of the afore-mentioned code can of course be used in a benign fashion,” the researchers wrote. “However, this script can easily be modified to encrypt someone’s machine completely without any user interaction. For example, it can potentially turn the code into ransomware if the script and syntax problems are fixed.”

In another case, a forum participant with a more technical background posted two code samples, both written using ChatGPT. The first was a Python script for post-exploit information stealing. It searched for specific file types, such as PDFs, copied them to a temporary directory, compressed them, and sent them to an attacker-controlled server.

The individual posted a second piece of code written in Java. It surreptitiously downloaded the SSH and telnet client PuTTY and ran it using Powershell. “Overall, this individual seems to be a tech-oriented threat actor, and the purpose of his posts is to show less technically capable cybercriminals how to utilize ChatGPT for malicious purposes, with real examples they can immediately use.”

Yet another example of ChatGPT-produced crimeware was designed to create an automated online bazaar for buying or trading credentials for compromised accounts, payment card data, malware, and other illicit goods or services. The code used a third-party programming interface to retrieve current cryptocurrency prices, including monero, bitcoin, and etherium. This helped the user set prices when transacting purchases.

Friday’s post comes two months after Check Point researchers tried their hand at developing AI-produced malware with full infection flow. Without writing a single line of code, they generated a reasonably convincing phishing email:

The researchers used ChatGPT to develop a malicious macro that could be hidden in an Excel file attached to the email. Once again, they didn’t write a single line of code. At first, the outputted script was fairly primitive:

When the researchers instructed ChatGPT to iterate the code several more times, however, the quality of the code vastly improved:

The researchers then used a more advanced AI service called Codex to develop other types of malware, including a reverse shell and scripts for port scanning, sandbox detection, and compiling their Python code to a Windows executable.

“And just like that, the infection flow is complete,” the researchers wrote. “We created a phishing email, with an attached Excel document that contains malicious VBA code that downloads a reverse shell to the target machine. The hard work was done by the AIs, and all that’s left for us to do is to execute the attack.”

While ChatGPT terms bar its use for illegal or malicious purposes, the researchers had no trouble tweaking their requests to get around those restrictions. And, of course, ChatGPT can also be used by defenders to write code that searches for malicious URLs inside files or query VirusTotal for the number of detections for a specific cryptographic hash.

So welcome to the brave new world of AI. It’s too early to know precisely how it will shape the future of offensive hacking and defensive remediation, but it’s a fair bet that it will only intensify the arms race between defenders and threat actors.