GitLab AI: Safe Code Turned Malicious – Research
- AI-assisted developer tools,marketed as essential for software engineers,can be easily manipulated by malicious actors.
- Legit researchers demonstrated how Duo could be tricked into inserting malicious code into a script.This attack, triggered by prompt injections, can also expose private code and vulnerability details.
- Prompt injections, a common chatbot exploit, involve embedding malicious instructions into content the chatbot processes.
AI developer tools are vulnerable. This research exposes how GitLab’s Duo chatbot, designed to streamline growth, can be manipulated to insert malicious code. Researchers found that prompt injections, a common chatbot exploit, can trick AI assistants into unfriendly actions, exposing private code and vulnerability details. Malicious actors leverage merge requests, commits, and source code to mislead the AI, highlighting the risks when AI is deeply integrated into development workflows. Stricter input validation and monitoring interactions are essential. Stay informed with News Directory 3 for tech updates. Discover what’s next for AI security.
AI Developer Tools Vulnerable to Code Injection Attacks
AI-assisted developer tools,marketed as essential for software engineers,can be easily manipulated by malicious actors. GitLab’s Duo chatbot, promoted for its ability to streamline workflows, is among those susceptible to attacks.
Legit researchers demonstrated how Duo could be tricked into inserting malicious code into a script.This attack, triggered by prompt injections, can also expose private code and vulnerability details. The vulnerability arises when users instruct the chatbot to interact with external content, such as a merge request.
Prompt injections, a common chatbot exploit, involve embedding malicious instructions into content the chatbot processes. large language model-based assistants readily follow instructions from various sources, including those controlled by attackers. Attacks targeting Duo utilized merge requests, commits, bug descriptions, and source code to mislead the AI.
“This vulnerability highlights the double-edged nature of AI assistants like GitLab Duo: when deeply integrated into development workflows, they inherit not just context—but risk,” Legit researcher Omer Mayraz said.
Mayraz added that hidden instructions in project content can manipulate Duo’s behaviour, exfiltrate private source code, and leverage AI responses for harmful outcomes. The incident underscores the need for robust security measures in AI-powered development environments to prevent code injection and data breaches.
What’s next
Developers should exercise caution when using AI tools and carefully review any code generated or modified by these systems. implementing stricter input validation and monitoring AI interactions can help mitigate the risk of prompt injection attacks and protect sensitive data.