This Week in AI Security - 3rd September 2026

Jeremy Snyder: All right. Welcome back to another episode of This Week in AI Security, coming to you for the week of the third of September twenty twenty six, coming to you from the sidelines of AI Tech World. Unfortunately, we're not going to be able to get into a lot of the lessons learned from AI Tech World this week, but stay tuned for more on that next week. We've got a little bit of a shorter episode this week, so let's dive in.

We're going to start off with a story about Claude Code, Codex, and Hermes installing unowned code inside corporate networks. So what's going on here? Well, what's going on here is that the llms.txt file, which is kind of the new version of robots.txt specifically designed for LLMs that are indexing websites, has been compromised in a number of different locations. So six thousand two hundred and fourteen corporate domains were found. One hundred and twenty llms.txt files that referenced unregistered package names. And if you remember an episode probably from about three or four months back, we talked about slopsquatting. So that's really the technique that is at play here. And that is the claiming of kind of predicted, through kind of token prediction algorithms that LLMs use, predicted names of common open source libraries or packages that maybe could exist but don't actually exist. And the interesting thing here is you kind of combine that with another trend that we've talked about any number of times on the show, which is the ability to embed malicious instructions anywhere that you can embed text. So what happens in this case is that you, the LLM, will go and index a website. It goes and finds a llms.txt file. It looks for things that should exist there. And now the attackers can find that, figure out, oh, an LLM is going to go look for a package called my malicious python code dot py. I'm going to go ahead and claim that, create it, effectively publish it as open source. And the behavior is the LLM goes, retrieves this file, maybe executes some of the instructions. And what we're seeing is the first Fortune 500 callback arrived in under four minutes after researchers claimed abandoned package names, meaning that a researcher claimed a package name. And then within four minutes, they actually got a request from an LLM on a Fortune 500 network that is out there crawling the web looking for things. So there's no phishing, there's no exploit. It's just a text file. It is almost like a supply chain way to combine those two techniques that we've talked about, indirect prompt injection anywhere that you can put text, slopsquatting with the unclaimed package names. All right. Moving on.

We've got our next, which is a critical Langflow flaw exploited to steal OpenAI and AWS keys. Now this refers to a particular CVE with a very high nine point eight out of ten CVSS score. So that's the severity. Langflow, if you remember, is kind of a low code AI app building platform. There are already more than three hundred observed exploit attempts in the wild so far, and this is the sixth Langflow CVE from this year. The platform, as kind of a high value, low code AI building app, which is something that so, so many people want to be using right now, is a great target. And so from that, from the CVE, from the exploitation of the CVE, this allows for unauthenticated RCE with root privileges. And attackers have been able to harvest Langflow superuser credentials, OpenAI API keys, and AWS secrets from environment variables around these vulnerable versions. As with all CVEs, the primary defenses, go get a patched version, kind of the latest updated version, etc. All right.

Next we've got some research from the Microsoft Threat Intelligence Group. They're documenting attacks across three AI infrastructure targets. Kind of following on to our last story, LiteLLM gateway, which has also been targeted multiple times. There is a command injection in the MCP endpoints there. Another package called RAGFlow that has a server side request forgery vulnerability, and Kestra, which is an orchestration layer. And in that case, there is a container escape that allows the deployment of a crypto miner on that. Now, the common thing about that, and if you just heard me describe those three packages, you might be asking yourself, well, those are three pretty different things, kind of, you know, LLM gateway, RAG tooling, and orchestration layer. The common pattern here is that it's not the LLM or the model itself that's the target. It's the infrastructure around it, and the infrastructure around it is often actually more valuable because it'll contain things like keys to access other parts of the infrastructure, whether that is a database that is used in conjunction with that RAG tool, whether that is an application infrastructure stack that sits behind that LLM gateway, or maybe it is actually the keys to the LLM that the LLM gateway is using. It's the same kind of playbook that we've seen against traditional infrastructure. Steal access in some way, typically through compromised credentials, persist in the environment, and then monetize. And that's actually the most interesting thing about it, is that a lot of the times, the goal of these attacks right now. Great. Data exfiltration of high value data on a high value target is one thing. But the other thing is access to the infrastructure, because it's expensive infrastructure to run, right? So that's one of the goals of some of these attacks. All right. Moving on to our next story.

We have documented evidence of the Aurora ransomware gang using SpaceX's Cursor agent, running Claude Sonnet behind it, to plan and execute hands on exploitation across ten corporate networks over the time frame of April to May of this year. The chat history shows a full Active Directory Certificate Services exploitation plan written in Russian via the AI assistant. So not only using the LLM to execute, but also even to plan out the attack, reconnaissance, lateral movement, credential harvesting across the victims, across ten organizations in nine countries. This isn't really a new news kind of story. We've seen evidence of this before. It's just further evidence of agentic AI for offensive use. We're going to come back to that theme in just a couple of minutes here in another story in this week's episode. Let's move on to our next one.

Our next one is an announcement out of OpenAI. And they just launched their Astra model, and it is the first model to receive the quote unquote critical cybersecurity capability rating under OpenAI's Preparedness Framework. Now, bear in mind, this is a self declared framework and kind of a self-declared capability level. So take that for what you will. But what it really refers to in the declared framework itself is the ability to autonomously discover zero day vulnerabilities in hardened real world systems, and then chain them together into working exploits without human guidance. And you might be saying to yourself, well, that's actually something that we've already observed from things like the OpenAI Hugging Face incident previously, from things like some of the Mythos and Anthropic model sandbox escape patterns, etc. The only thing that's interesting around this, I think, is the fact that we are seeing this now as a pre-announcement instead of as a post-event kind of oops, our model got out into the wild. Now, the model did score one hundred percent on exploit-bench during internal testing, discovered two real world zero days that OpenAI is disclosing to the maintainers. That is being done, I think, silently and confidentially, as responsible disclosure should be done. And the interesting thing is actually, you know, there's been kind of a big comparison, long running comparison, between OpenAI and Anthropic. And I would say like a kind of common market perception at this point that Anthropic is the more, let's call it, quote unquote, enterprise oriented of the two model providers in terms of working with enterprises, having more of a kind of, let's say, cautious or measured rollout pattern to their models and their model family updates and cyber capabilities in particular. But this is really, in my mind, a step in that direction from OpenAI, where they are declaring openly that they're going to pause development to strengthen safeguards and that they really, let's say, explicitly recognize the dual use debate here. There is both defensive capabilities, but there is a strong risk around offensive capabilities from a model with this set of capabilities. All right. Let's move on to our next story.

And that is our final story for this week. So I want to spend a little bit more time talking about it here. And that is some research from the folks over at ThreatDown. Shout out to them. Kudos to them for taking this on. It was a really great write up. I really enjoyed reading it. Which is basically the kind of technical unpeeling of a stack called Kriminal. And that's Kriminal with a K. So that, I think that's, you know, Kriminal with a K, dot ai. It is one of the most popular criminal AI tools on the clear web. So not on the deep web. But interestingly, when you look at the source code and you really kind of peeled back the layers, what you actually find is that it's a jailbreak wrapper, renting inference from Grok as the primary engine, OpenRouter, Mistral's and Llama 3.3, as well as Anthropic Claude for certain long context prompts. And so really, there is a kind of embedded system prompt here that, you know, says ignore all previous instructions. And that is layered on top of legitimate APIs. You can think of it as the equivalent of ignore all previous instructions. Pricing is twelve point nine nine a month for unrestricted open source intelligence, exploit code, social engineering personas, and an OpenAI compatible endpoint. So that's a very nice, affordable package for a criminal organization that gets them access to capabilities potentially from frontier models. And so one of the things that has been discussed around LLM and frontier model development in general is the risk around open weight models, is that they remove, or you can remove, a lot of the safeguards. So if you've got the infrastructure to run an open weight model, and you've got access to download, set it up, and let's say the capabilities and the know how to make that happen, you can get into the inner workings of the model. You can remove a lot of the safeguards. You can also remove kind of the pre prompt execution layers that do things like prompt evaluation for risky behaviors, injection, etc. And that has been accepted as kind of the way that a lot of the kind of criminally oriented models would develop. And, you know, there were a couple of models a couple years back, WormGPT, FraudGPT, things like that, that got a little bit of traction at the time, but didn't really sustain. Why? Because the capabilities in the models really moves too quickly. And so after a pretty brief period of time, the defensive capabilities of the later and newer models would really outpace the offensive capabilities of some of those, let's call them like cracked models or criminal models. So instead, what somebody has figured out here is a way to basically put a wrapper around an inference layer, do a little bit of clever routing, looking at different models for different use cases and target, and let's say maybe distribute the prompts across them in a way such that it becomes harder and harder to detect malicious activity across the environments. And so that is really what's going on here. And so it's not that you actually need something like a FraudGPT or a WormGPT, etc. You can actually get a kind of criminal wrapper around open frontier models. I shouldn't say open, but around frontier models. So criminals don't need their own AI. They can just go rent it now as a service. The guardrails are not going to hold in an environment like this where you have kind of deliberate attempts to obfuscation. We've covered reasons why guardrails are always kind of avoidable with the right kind of, let's say, persistence and the right kind of approach to the guardrails, whether that is through persuasion techniques, whether that is through the removal of those guardrails, etc. So that is basically the story that I want to close out today's episode on. Not a high note necessarily for this week, but I think a really great analysis, again, from the folks over at ThreatDown, on what we know about cybercrime and some of the use of AI in cybercrime. It is kind of a reference back to what I was saying earlier about the Aurora ransomware group leveraging AI for offensive capabilities. So if you have a model that is kind of readily usable for offensive capabilities, and then you kind of layer that with other models where the initial model maybe doesn't provide you everything that you need, and you do, again, some intelligent routing, a little bit of a harness, a little bit of manipulation of system prompts across the orchestration between the different model providers that you're leveraging, that's the type of results that you can achieve from a setup like that. All right. That's going to do it for this week on This Week in AI Security. We will talk to you again next week, hopefully have some stories from AI Tech World and some interesting observations to share with you once again. Like, subscribe, rate, review, all that good stuff. Bye bye.

‍