The Evolving AI Threat Landscape: How Adversaries Weaponize Generative Models

Introduction

Since our February 2026 report, the Google Threat Intelligence Group (GTIG) has observed a significant shift from experimental AI-assisted cyber operations to industrialized adversarial use of generative models. Drawing on Mandiant incident response findings, Gemini insights, and GTIG's proactive research, this article examines the dual-edged nature of the current threat environment: AI as both a powerful engine for attacks and a prime target. Below, we explore key developments including automated exploit generation, AI-augmented malware, and supply chain attacks on AI infrastructure.

The Evolving AI Threat Landscape: How Adversaries Weaponize Generative Models — Source: www.mandiant.com

AI-Driven Vulnerability Discovery and Exploit Generation

For the first time, GTIG has identified a threat actor who developed a zero-day exploit using AI. The criminal group intended to use it in a mass exploitation event, but GTIG's proactive counter-discovery may have prevented deployment. Additionally, state-linked actors from the People's Republic of China (PRC) and the Democratic People's Republic of Korea (DPRK) have shown strong interest in leveraging AI for vulnerability discovery, signaling a new era of automated hacking.

AI enables rapid analysis of large codebases to find weaknesses, and can even generate proof-of-concept exploit code. This capability lowers the barrier for sophisticated attacks, making zero-day exploitation more accessible to a broader range of threat actors. Learn about AI-augmented development for defense evasion.

AI-Augmented Development for Defense Evasion

AI-driven coding tools have accelerated the creation of infrastructure suites and polymorphic malware by adversaries. These tools allow rapid iteration and obfuscation, enabling attacks that evade traditional detection methods. For example, suspected Russia-nexus threat actors have integrated AI-generated decoy logic into malware, making analysis harder for defenders.

Key techniques include:

Polymorphic code generation: AI creates multiple variants of malware to bypass signature-based detection.
Automated obfuscation networks: Adversaries build layered proxy and redirection systems using AI to hide command-and-control traffic.
Decoy logic injection: AI generates misleading execution paths that confuse sandbox analysis.

Autonomous Malware: The Rise of PROMPTSPY

AI-enabled malware, with PROMPTSPY as a prime example, marks a shift toward autonomous attack orchestration. This malware interprets system states and dynamically generates commands via AI models, manipulating victim environments without human intervention. GTIG analysis has revealed previously unreported capabilities, including the use of large language models (LLMs) to craft convincing phishing messages and adapt to network conditions in real time.

The autonomous nature of such malware allows threat actors to scale operations while reducing the need for manual oversight. PROMPTSPY can execute multi-step attacks that adjust based on defenses encountered, making it a formidable tool for persistent threats.

AI-Augmented Research and Information Operations

Adversaries are increasingly using AI as a high-speed research assistant throughout the attack lifecycle. From initial reconnaissance to lateral movement, AI helps analyze vast datasets, identify optimal attack paths, and automate reporting. In information operations (IO), AI fabricates digital consensus by generating synthetic media and deepfake content at scale. The pro-Russia campaign "Operation Overload" exemplifies this, flooding platforms with AI-generated comments and videos to manipulate public opinion.

Agentic workflows further enable autonomous attack frameworks, where AI agents operate independently to achieve strategic goals. This evolution from simple automation to autonomous decision-making poses significant challenges for detection and response. How adversaries obtain and abuse LLM access.

Obfuscated LLM Access: The Underground Economy of AI Tokens

Threat actors have developed sophisticated methods to obtain anonymized, premium-tier access to AI models. They use professionalized middleware services and automated registration pipelines to bypass usage limits and evade detection. This infrastructure enables large-scale misuse, including generating malicious content, crafting exploits, and automating reconnaissance—all while subsidizing operations through trial abuse and programmatic account cycling.

Key strategies include:

Automated credential creation: Bots generate thousands of accounts using leaked or synthetic identities.
API key reselling: Underground markets offer stolen or cracked API keys for major AI providers.
Proxy rotation and CAPTCHA solving: Services provide undetected access to models, often for a fee.

Supply Chain Attacks on AI Environments

Adversaries such as "TeamPCP" (also tracked as UNC6780) have begun targeting AI software dependencies and cloud environments as initial access vectors. By poisoning open-source libraries used in AI pipelines or compromising plugins for popular AI frameworks, attackers can inject backdoors that persist across multiple deployments. These supply chain attacks yield multiple benefits: data theft, model manipulation, and infrastructure hijacking.

Organizations must vet third-party AI components rigorously, monitor for unusual API usage, and implement runtime security controls. The convergence of AI development and cybersecurity demands new defensive practices, as traditional supply chain risks now extend to the models and data powering intelligent systems.

Conclusion

The GTIG AI Threat Tracker reveals a rapidly maturing adversarial ecosystem where AI is both a weapon and a target. From zero-day exploits crafted by criminal actors to autonomous malware like PROMPTSPY, the threat landscape is becoming more sophisticated and harder to defend against. Organizations must adapt by investing in AI-specific security measures, sharing threat intelligence, and staying informed about emergent attack patterns. The dual-use nature of AI demands proactive, collective defense to stay ahead of adversaries.

Tags: