Guardrail-free chatbots can aid extremists, study finds

Wired reports researchers got a chatbot with its safety guardrails removed to give detailed weapon instructions, showing how easily models can be modified.

A new report shared with Wired shows how quickly open-weight AI models can have their safety guardrails stripped out, and how those modified models are being used by extremists.

What the report found

The Counter-Terrorism AI Benchmark report comes from Tech Against Terrorism, a UK nonprofit. It documents how, within a day of Alibaba releasing a small open-weight model designed to run on a laptop, modified versions advertised as "abliterated", meaning their refusal behavior had been removed, were circulating online.

Wired reports that one neo-Nazi figure quickly announced he was using such a modified model to build an extremist chatbot and began selling access to it. Researchers also showed that a model with its guardrails removed would provide detailed instructions for making an improvised explosive device after being told the user intended to cause mass harm.

Tech Against Terrorism's executive director, Adam Hadley, argued that public debate focuses heavily on long-term existential risk while paying too little attention to these modified models, which are causing problems now.

Why it matters for builders

The core technical point is that safety behaviors trained into a model are not a lock. With open weights, anyone can fine-tune or edit the model to remove refusals. That does not mean open models are bad, as they have large benefits for privacy, cost and research, but it does mean safety cannot rely on the model alone.

If you build products on open models, add your own layers: input and output moderation, rate limits, logging of abusive patterns, and clear terms of service. If you host models for others, be careful about which community fine-tunes you offer; a model card that advertises removed guardrails is a red flag.

For platform operators, including anyone running a model marketplace or plugin directory, the report is a reminder to review what gets listed and to act on reports of misuse quickly.

The broader question of whether the benefits of open models outweigh these risks is explored in another of our AI News articles.

Source: Wired. This article is PickAI’s own summary and analysis based on the original reporting.