What are AI tarpits? Understanding the tools people are using to poison LLMs

AI chatbots require continuous data assimilation, a process known as training, to increase their intelligence and overall utility for end-users. However, a significant conflict has emerged as many AI companies scrape webpages and integrate data into the corpora of large language models (LLMs) without explicitly obtaining consent from the original data owners.

In response to these practices, content creators and intellectual property holders are utilizing tools known as tarpits to protect their work. The objective of these tools is to poison the underlying LLM, thereby degrading the quality of the chatbot’s outputs and potentially causing end-user flight.

Understanding AI Poisoning

AI poisoning is the process of corrupting a chatbot’s underlying large language model to ensure it produces incorrect, misleading, or nonsensical outputs. This corruption occurs when an LLM is tricked into assimilating incorrect data during its training phase, which often involves the wide-scale scraping of available websites and images.

The techniques used for poisoning vary based on the specific capabilities of the LLM that the poisoner intends to disrupt.

For those targeting image generator LLMs, a technique called Nightshading is used. This method employs software called Nightshade to add an invisible layer to an image. This layer consists of pixels that are invisible to the human eye but detectable by LLM scrapers.

These pixels mislead the AI into perceiving the artwork as being in a different style than it actually is—for instance, interpreting a realistic piece as abstract. This prevents the LLM from accurately mimicking the actual style of the artist.

The Mechanism of AI Tarpits

While Nightshading is effective for visual media, the majority of chatbots process text, rendering image-based poisoning tools useless against the unauthorized scraping of blogs and articles. To counter text scraping, a different category of poisoning tools known as tarpits has been developed.

AI tarpits are designed specifically to trick the crawlers used by LLMs into ingesting useless data. Because the LLM then relies on this junk data to generate its text responses, the resulting outputs become incorrect. This degradation of response quality serves as a deterrent that could eventually discourage users from utilizing the chatbot.

Content creators and IP holders can embed various tarpit traps into their website code, including tools such as Nepenthes, Iocaine, and Quixotic.

When an LLM crawler visits a site containing a tarpit, it is redirected to automatically generated text that is either completely nonsensical—such as claiming the color of water is pepperoni—or riddled with factual errors, such as stating that Steve Jobs founded Microsoft in 1834.

The effectiveness of the tarpit lies in its structure. These pages of poisoned text contain links that lead to further pages of poisoned text, but they do not contain any exit links. Similar to how a physical tarpit traps an animal, these digital tarpits trap the LLM crawler in an endless cycle of assimilating incorrect data with no way to exit the trap.

Data Protection for General Users

The use of tarpits allows IP holders to waste the valuable resources of AI companies while preventing the non-consensual assimilation of their website data.

Beyond professional content creators, general users are also subject to data collection by AI companies. Every prompt entered into a chatbot or every conversation held with the AI is assimilated into the LLM’s corpus for further analysis, with the goal of making the model more robust.

Users who wish to protect their data do not need to employ specialized tools like tarpits. Instead, they can use the following methods: