Millions of people now use AI tools like ChatGPT to draft emails, generate code, and manage daily tasks—but those same tools are vulnerable to attacks that can inject malicious code into users’ systems. The problem is that as AI assistants become embedded in workplaces and personal devices, their extensions and plugins create new entry points for cybercriminals. Prompt injection, data poisoning, and adversarial attacks exploit the trust users place in these platforms—studies show 60–70% of people accept AI-generated outputs without checking them. Meanwhile, 72% of web application vulnerabilities already stem from coding flaws, a pattern that now extends to AI. AI-SAFE tackles this by building a browser extension and screen reader that evaluates codebases in real time, flagging malicious code before it executes. If successful, the system could prevent attackers from exploiting AI plugins to compromise personal data, corporate networks, or critical infrastructure. The project focuses on a specific, practical vulnerability—API extensions—rather than trying to solve all AI security problems at once. For the millions of users and organisations now relying on tools like ChatGPT, Copilot, and DeepSeek, a working shield against code-based exploits would make everyday AI use significantly safer.
View original technical description
The emergence of tools like ChatGPT has significantly reshaped public perception of Artificial Intelligence. Since its launch in 2022, ChatGPT has gained approximately 300 million users. This growth has been further complimented by platforms such as Copilot and DeepSeek, with around 1.8 million and 12 million users, respectively. The rapid advancement of these tools, along with the expanding range of capabilities they offer, has opened unprecedented opportunities. As a result, many organisations have begun integrating them into their workplace applications and users have started resorting to these platforms for the simplest of tasks. This has thus transpired into use of such tools for even the smallest tasks such as drafting out an email. With the rise in interest in these tools, developers have expanded their use with the support of extensions that cover various domains, including personalised chatbots, healthcare assistants, development frameworks, and specialised applications like data analysis and code generation tools. ChatGPT alone is estimated to have around 1,000 plugins catering to diverse use cases. However, this rapid expansion and widespread integration has also introduced new cybersecurity vulnerabilities. These include prompt injection attacks, data poisoning, model inversion, adversarial attacks, and direct exploitation by cybercriminals. A primary motivation behind most such attack vectors is the creation of malicious code or files intended to exploit users. The success of these attacks is further complemented by the high level of confidence users have on these platforms with studies indicating that 60%-70% users trust outputs generated by Large Language Models (LLMs) without further verification. Despite safeguards against generating harmful code, vulnerabilities persist, particularly in API extensions, which serve as critical entry points for attackers. Malicious actors exploit these endpoints to inject harmful code, compromising users. This mirrors conventional security breaches, where 72% of web application vulnerabilities stemmed from flaws in coding practices. AI-SAFE addresses these challenges by leveraging advanced AI to assess the suitability of codebases and detect potential risks. Its real-time evaluation, facilitated through a browser extension and a screen reader, helps prevent the execution of malicious code, ensuring a more secure development environment.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know