A new cyber threat? A look at AI worms

It’s no secret that the world of technology is on a steady curve of development – just in the past few years we’ve seen chatbots, AI and virtual reality challenge the boundaries of how we use technology. And sadly, with the great development of technology comes the development of ways to exploit it.

The research begins

Fortunately, in this case, the new cyberthreat has been developed and explored by a team of researchers, who wanted to see AI’s full potential – both good and bad. The team from Cornell Tech wanted to show what kind of risks there are involved with autonomous AI systems; they thus developed a generative AI worm.

• The purpose of the AI worm is to spread malicious software from one system to another and potentially steal crucial information from the user.

This, as researcher Ben Nassi states, is a brand-new way of conducting cyber attacks that we, potential victims, and users of AI, aren’t prepared for.

Creating the new AI worm

Ben Nassi, alongside colleagues Stav Cohen and Ron Bitton, developed the AI worm they named Morris II (the name was a recognition of the computer worm, Morris, that roamed the internet back in 1988).

Nassi, Cohen and Bitton have specifically designed the AI worm to work within some automated functions in an email inbox. Here they programmed the AI worm to attack the email assistant to steal data and information from the user’s inbox and furthermore send out spam emails to other users.

Examples of the data the AI worm can steal are:

• Phone numbers
• Bank credentials
• Addresses
• Names

The research team tested the AI worm in controlled settings within a closed test environment so that no users or devices were unintentionally damaged or affected by the worm.

How AI worms work

If you’re familiar with any type of AI – whether it’s chatbots, virtual reality or augmented reality – you know that you often have to give the technology a prompt to act on. It can be "Write me a recipe for spaghetti carbonara" or "Make a painting of Leonardo Da Vinci and Michelangelo in a conversation". Then the machinery works its magic and draws up a text or picture of what you asked the software to do.

In the case of the AI worm, Nassi, Cohen and Bitton programmed it as an "adversarial self-replicating prompt" meaning that the technology can replicate itself with a prompt – from itself. So, the AI worm makes an output, a prompt, and answers to that with another prompt – and so on.

To test this, the research team made an email system where it could receive and send messages using generative AI technology. By testing this, they discovered two ways they could exploit the new technology;

• Using a text-based self-replicating prompt
• Inserting a self-replicating prompt but by doing this via an image or jpg. file

In the first instance, the team wrote an email where they included an adversarial text prompt. This prompt infects the database where the email assistant uses RAG (retrieval-augmented generation). Once the email is registered by the RAG, it can create a response to the prompt – and in this case, it will be to steal data from the email, since the team acting as hackers, has ordered the program to do so.

In the second instance, the "hackers" have infected an image with a prompt, that orders the technology to forward the message to other email accounts. What’s tricky about this type of attack is that the image can contain many types of malware, spam and messages – it can thus be forwarded to potential clients and harm any business and private user.

The future of AI worms

Ben Nassi, Stav Cohen and Ron Bitton created AI worms to spread awareness of the many, and fatal, vulnerabilities that lie within the technology of AI.

One of the main reasons the team decided to develop and publish the exploitation of AI is to spread awareness to the big developers, so they can reprogram their software. Currently, big developers such as Google and OpenAI have yet to react to this research on the vulnerabilities that lie in AI technology.

One of the areas users should pay particular attention to is the programs and features where you can allow the software to take actions on your behalf – e.g. sending emails or booking an appointment. We should furthermore pay attention when we connect AI software to other kinds of software since the AI software gains access to the information stored in the connected software or device.

If you e.g. connect an AI software to your email account, you should consider what information you have in your email account but also on the device on which you access your email. If hackers should reprogram the AI you’re using, you might put all your personal and confidential information on the line.

The Cornell Tech team sadly predicts that we’ll see AI worms in the near future in the cyber world. It’s only a matter of time before hackers discover a way to reprogram and hack AI technology. That is why it’s important for us to spread awareness of this problem before it becomes a crucial problem for us.

Mitigating AI worms

Experts point to users when it comes to mitigating AI worms. We can e.g. limit the amount of actions AI can take on our behalf and thus avoid it having access to our personal data.

Another thing that should be pointed out is, that often programs and software can detect anomalies in behavior. Looking at the self-replicating prompt feature, we know that it multiplies prompts at a fast pace. Often this can be detected by safety mechanisms within the software and thus be caught before it has done too much damage.