Custom large language model (LLM) hacking tools are available across the dark web, promising to help attackers generate successful lower level cyberattack campaigns.

Many of these LLMs appear to be copies of commercial models, according to researchers. They are trained on malware datasets making them malicious by nature.

The majority of these malicious LLMs are advertised as hacking tools to help attackers launch more successful malware and phishing attacks. However, some are being advertised as legitimate penetration testing aids.

Researchers are wary of this dual-use dilemma, stating that "[a]ny tool powerful enough to build a complex system can also be repurposed to break one", highlighting the danger of these models.

One such AI tool is KawaiiGPT, which is open-source, free, and available on GitHub the most popular software development site worldwide.

Another is WormGPT which states that it is an "AI without boundaries" as its primary selling point.

Sources: https://cybernews.com/ai-news/hackers-use-underground-ai-models-malware-phishing-attacks/

Commentary

Workplace participants may utilize "dual purpose" LLMs to introduce these LLMs into organizational systems not knowing they are dangerous. Or, they may use the tag "dual purpose" to conceal malicious intent. Either way, the potential harm exists.

Malicious LLMs are quite dangerous. They can generate scripts that infiltrate systems beyond their intended purpose, suggest dangerous configurations, or embed hidden backdoor access into organizational systems.

Consequently, organizations should create governance regarding the use of any unvetted AI tool, including "dual purpose" LLMs, including:

· Establish clear policies regarding AI use in the workplace, including which AI tools are approved, and which are not

· Establish procedures to verify all generated code before deployment

· Place limits on what tasks workplace participants may use AI tools to perform

· Require all vendors and contractors to disclose their use of AI tools, including the exact AI tools they use

· Update cybersecurity training to include information about malicious AI that is marketed as a tool for another purpose

· Educate workplace participants about the risks of using unvetted AI models

The final takeaway is that AI dangerous LLMs and other AI tools may allow workplace participants to cause damage, whether intentionally or unintentionally.