Irregular, an Israeli AI security testing startup, was at the center of recent AI incidents involving OpenAI, Anthropic, and Meta. Misconfigurations in its testbed allowed models to access the public internet, highlighting the critical role of third-party security testing in AI development.
TL;DR
- Irregular's testbed misconfigurations led to AI models from OpenAI, Anthropic, and Meta accessing the public internet.
- The incidents underscore the growing importance of specialized companies in AI security testing.
- Irregular is valued at $450 million and has raised $80 million in funding from Sequoia and Redpoint Ventures.
What happened
Irregular, a Tel Aviv-based AI startup, provides a cybersecurity testbed for AI models. Recent incidents at OpenAI, Anthropic, and Meta revealed that misconfigurations in Irregular's testbed allowed their AI models to access the public internet, which they should not have been able to do.
OpenAI acknowledged the issue in a blog post on August 4, stating that the misconfiguration allowed models to access the internet. Anthropic had previously mentioned that its Claude model may have accessed the internet, and Meta also disclosed a similar incident this week.
Irregular's testbed is used by major tech companies to evaluate the security of their AI models. The incidents highlight the critical role that third-party security testing plays in the development of AI models.
Why it matters
The incidents underscore the growing importance of specialized companies in AI security testing. As AI models become more powerful, the need for robust security testing becomes increasingly important to prevent malicious actors from exploiting vulnerabilities.
For developers and startups, the incidents serve as a reminder of the importance of thorough security testing. As AI models become more integrated into critical systems and infrastructure, the consequences of security breaches can be severe.
For investors, the incidents highlight the potential of companies like Irregular that specialize in AI security testing. As the AI industry continues to grow, the demand for specialized security testing services is likely to increase, presenting a significant investment opportunity.
Key facts
- Irregular is valued at $450 million and has raised $80 million in funding from Sequoia and Redpoint Ventures.
- The company provides a cybersecurity testbed for AI models, allowing developers to evaluate the security of their models.
- Recent incidents at OpenAI, Anthropic, and Meta revealed misconfigurations in Irregular's testbed that allowed models to access the public internet.
- OpenAI acknowledged the issue in a blog post on August 4, stating that the misconfiguration allowed models to access the internet.
- Anthropic had previously mentioned that its Claude model may have accessed the internet, and Meta also disclosed a similar incident this week.
- Irregular is among the few organizations with the technical expertise needed to conduct advanced security testing for foundation model developers.
- The incidents highlight the critical role that third-party security testing plays in the development of AI models.
Context
The rapid development of AI has created a need for a limited number of companies that specialize in particular areas of the market. These players include experts in data training and annotation, running evaluations to assess a model's capabilities, and conducting operational security tests designed to identify weak spots that bad actors could exploit.
Irregular is one of the few organizations with the technical expertise needed to conduct advanced security testing for foundation model developers, alongside groups such as the non-profit METR and Apollo Research.
As AI models become more powerful, their ability to act in malicious ways is becoming a significant threat to governments and corporations, particularly given the risk of them hacking into critical systems and infrastructure.
