IARPA
May 19, 2025
Peraton Labs is one of the primary performers in the TrojAI program, playing a critical role in developing advanced AI security techniques. Leveraging its deep expertise in national security, machine learning, and software engineering, Peraton Labs contributes to the design, research, and refinement of methods for detecting and neutralizing backdoors in AI systems.
By working alongside other organizations, Peraton Labs helps ensure the resilience of AI systems critical to the Intelligence Community and broader national defense infrastructure.
Artificial Intelligence (AI) is becoming deeply embedded in everyday life, from smartphones and smart speakers to tools like ChatGPT. For the U.S. Intelligence Community (IC), AI is also transforming mission operations. However, this increasing reliance introduces new vulnerabilities, particularly Trojan attacks, also known as backdoor or data poisoning attacks. These attacks manipulate AI systems by training them to respond incorrectly to a specific, often rare trigger—posing serious national security risks.
To combat this threat, the Intelligence Advanced Research Projects Activity (IARPA) launched the Trojans in Artificial Intelligence (TrojAI) program. The initiative aims to develop tools and technologies that detect and mitigate Trojan attacks in AI systems—especially before deployment in critical operations.
Since its launch in 2019, TrojAI has expanded to include domains such as image processing, cybersecurity, natural language processing, reinforcement learning, and large language models. As models become larger and more complex, the challenge of detection increases, but the program has made significant progress thanks to key partners.
