This Week in AI Security - 16th July 2026

All right. Welcome back to another episode of This Week in AI Security, coming to you for the week of the sixteenth of July 2026. And again, we're a little bit lucky this week that we've got fewer stories to get through, which means we get a little bit of extra time to dive in on some of them in more detail, which is great because we've got a couple of really interesting ones, as well as a couple that we'll probably get through a little bit more quickly, because they are just rehashes on things that we have discussed many times on the show already.

So we're going to start with our first story, which is top AI agents built to catch malicious code can be tricked into running it. And you can file this under the bucket of AI coding practices, IDE environments, all of these areas where we have seen examples of indirect prompt injection, where we have a set of malicious instructions embedded in something like a Readme file embedded into something like a code repository, commit comment, or something like that, in which a LLM that is being designed to automate a process will pick up that malicious text and then execute on it. And in this case, we're talking about agents that are built to scan malicious code.

So what is this? If you think about it, you know, there's a lot of hype around, again, Claude Mythos and some of the other models with their new cyber capabilities and their ability to scan code repositories, interpret the software code, and then figure out vulnerabilities on that. Right. And so examples of this are anything from all the vulnerabilities that are being disclosed right now around Windows, around different pieces of office software, around third party web applications, around Apache, around common open source libraries and frameworks that have been in use for a long, long time. So think of it like this. You take a piece of software, Mythos or other, and you point it at a code repository and you say, hey, security, scan that code repository for me, help me identify all of the vulnerabilities and weaknesses in my coding practices that I didn't know about. It starts that scan. Somewhere in that code repository, you have an embedded set of malicious instructions.

Now, those could be intentionally embedded by an inside threat actor. You know, insider risk. This could be a supply chain takeover where somebody, you know, with malicious intent gets access to one or more files in your repository and inserts a set of malicious commands in there. They could be a joke, a test, who knows what. But effectively it is, you know, your code base itself can be the trigger of a prompt injection attack, not on your code base itself, but on a third party engine that is scanning your code, looking for those security vulnerabilities. So interesting story over there. It's being dubbed friendly fire as an attack, which I think is kind of interesting.

All right. Moving on. We've got our next story. And this is one of the ones that I do want to spend a little bit more time talking about. And this is something called hallux squatting. And if you recognize the portmanteau there, that's hallucination and squatting. I want to take you back to a story and a trend that was observed pretty early on in the code agent kind of birth of code agents, if you will. Right. And so coding agents are those agents that you use to help you write software faster. And if you're familiar with modern software development techniques, and I'm going to take it as a given that pretty much any listener of this show is, you know, that you don't write every function that you need. There are certain things where it would just make more sense. Hey, I'm not going to add the ability to download and parse PDF files. I'm going to go use an open source third party library that helps me read PDF files.

And what was observed early on is that LLMs will actually hallucinate that such a package exists. And they'll create a package called pdfreader.py, for instance, using my example and for instance, the Python coding language. And what these researchers from the Tel Aviv University Technion have figured out is that this hallucination pattern hasn't really gone away. But the interesting aspect of it is that the the the ability to predict what names are going to come out is actually pretty high. So you can get about an eighty five percent accuracy on predicting the name that the LLM is going to predict for. Again, let's say that PDF reader function. Originally this term was called slop squatting. When the LLMs would manufacture a, you know, a class of software or some open source packages, etc. the interesting thing is that the hallucinated names are consistent across different foundation models, and it kind of does speak to, let's say, at least a common algorithmic approach into token prediction and token generation.

And so the agentic botnets that that kind of, you know, will look for code being written. They can now weaponize this at scale by predicting these names, pre-registering all of those packages and libraries and then inserting in them malware. And that could be as simple as a set of malicious prompts or instructions like we just talked about in the previous story. Or that could be as complex as actually having active malware or obfuscated malware embedded in some of those packages. So this has gone through a couple of different names. Now, as I mentioned, we talked about slop squatting. Palo Alto had come up with something called phantom squatting, and they had originally observed two hundred and fifty thousand domains sitting unregistered. And then we've now got this kind of next generation of it, which is this hallux squatting, kind of predicting what names are going to be created inside the code and in the manufactured or the identically created code that people are going to predicting, going to be predicting and prompting for, and then using that as a attack surface to go embed malicious content.

All right. Moving on to our next story. And this one was a really interesting for me because it ties into, I'd say like a larger meta problem that we've been talking about as an industry for a long, long time. And that is the speed of patching. So let me rewind for a second and kind of play out this story as it's unfolded from my perspective. So the first is, of course, we've seen the CVE in Langflow. It's CVE 2026-55255, and it's the first AI agentic orchestration platform. Well, actually maybe not the first that had a CVE, but it is the first one where the CVE has now made it into CISA's KEV. And so the KEV, if you're not familiar, is the known exploited vulnerability catalog. So this is, you know, not only is the CVE theoretically there. So we know that there is a weakness inside the software, but we know that it is actively being exploited in the wild. We have documented evidence of this. We've got log files, we've got breach events, whatever the case may be.

Now, what's interesting is that when a CVE like this lands in the CISA KEV, it typically kicks off a timer for federal and state agencies and I think also branches of the military, as well as federal civilian departments to act on that. And that timer, depending on the severity, I think, can be anything from three days to like ninety or one hundred and twenty days. Although there has been recent discussion, which we covered on the episode about the, uh, about CISA, kind of suggesting that those timelines need to be reevaluated given the rapid availability of exploit software.

So now the way that I think about this is, okay, we've gone from knowing that like, hey, it's not always the LLM that's the target. It's very often some of these open source packages around the LLM. We've got the ability to discover the CVEs in the code. We've got the ability to now generate the exploits around that. We've now observed those exploits being used in the wild. And now we kind of come full circle to the point where this has risen to the level of state agencies, or rather federal agencies now being required to patch this. So that's one thing. It's really a kind of a new milestone, if you will, in the life cycle of a vulnerability around a brand new piece of technology.

And I don't know for certain how long it was for things like, let's say, cloud or initial SaaS applications to start having these vulnerabilities that rose to that level. But this is a really, really fast time frame. If I remember right, the first report that we had about this vulnerability was probably about three months back. And so from three months of the identification of the CVE to the validated use in the wild of the exploit to the requirement to patch is very, very rapid compared to previous things. So that's one very interesting aspect of it to me.

And the other is I just have to call out here again that, you know, what is this CVE actually about? It's an IDOR flaw. It's about letting an authenticated user execute any other user's workflows. So this is what would be known modernly I think as BFLA, a broken function level authorization from an API perspective. So a couple of themes that we've talked about any number of times on the show. Number one is that it's not always the LLM. It's very often the kind of the plumbing, the infrastructure, the supporting software around it. And number two is that the API is the attack surface in so many cases here, because that is really how AI gets executed.

All right, moving on to our next story. We have another report of the use of agentic AI to really speed up cyber attacks. And in this case, it was an AWS environment compromised for concurrent work streams that were apparently running credential harvesting, backdoor creation data exfiltration from RDS, and impact actions, which zeroed out queues to help obfuscate the attack. I'm not going to go into this one in as much detail, but it is kind of a follow up to last week's story that we had about the first kind of AI ransomware that was executed really, quote unquote, by agents. And again, I think the details on all of these are a little bit fuzzy. So this is what the indications look like, but not necessarily take it as full fact at this point. So it's nothing really crazy in terms of new techniques or anything like that. No new zero days, novel malware, anything like that. It's just tried and true cloud techniques looking for credential harvesting aspects, looking for keys, exposing code, looking for places that keys might have been left open, etc. so nothing too crazy there.

All right. And I want to finish up this week's episode with a little bit of deeper dive onto an article from the CEO of Microsoft, a blog post from the CEO of Microsoft, Satya Nadella. And it's framed here on TechCrunch as a quote unquote, shocking warning to companies using AI. But I want to stop for a second and kind of like cool down the hype and actually talk about what is really being discussed in this article. So Nadella starts with this kind of bold claim that enterprises are actually paying twice for their use of enterprise AI models. And so they're paying once with money, which is really directly a result of token usage. And again, by handing over proprietary knowledge that makes the model useful. The way to think about this is that models learn from prompts, tool usage, and especially the corrections that we make. So I issue a prompt. I don't get my response or my desired response. So then I kind of go correct my response, rather my prompt and I try again. And this is something that we try and try again. And this is being kind of categorized as something called distillation.

And so one of the things that has been observed is that more and more enterprises are looking at this and saying like, hey, I can actually get my own model from one of these open source open weight models and start to use this distillation process on my own and try to make it smarter. And the model makers are actually arguing that that is out of bands from the terms of service of a lot of their usage. And Nadella's argument is that it is hypocritical for model makers to freely train their models on public web data, but then restrict their enterprise customers from doing this distillation action to the models themselves or onto models that they're trying to train themselves. And so, you know, Nadella is proposing that enterprises should retain ownership of their data, both in terms of the data that they use, but also at the layer of the prompts and the feedback and all of the reinforcement that users of these systems are providing in order to make it better.

He called this something called exhaust, like the exhaust coming out of your car. And it's kind of this byproduct of the use of the system, just like your exhaust is a byproduct of your use of the car. I don't know if I love that name necessarily, but I do understand where the argument comes from. It is, again, this byproduct of the use of the system that you're using. So great. I'm using AI. I want the best results out of it. I'm going to start prompting it. I'm going to start correcting my prompts. I'm going to start iterating, getting better at it, etc. what I think is really interesting about this, though, is really the recognition that all of that prompt data and all of the interaction history becomes a massive data set in itself that is beneficial for so many things.

And this is actually one of the things that we're super excited about at FireTail. And just to plug for a little second here, we've got a feature that we call prompt intelligence that takes the prompt history of your users within the organization, and it actually distills it down to help you analyze primarily topically and semantically, how your users are using AI and for what topics they're doing it. And I think that's one of the key things that we've realized from this data set. The distillation aspect of it is quite interesting as well, and it does represent this tension that in my mind, is actually quite parallel to the tension that exists between like open platforms and proprietary platforms.

And effectively, we're in this era where we've got our first set of open source models, and really only one or two of the leading open source models come from the West. And there's a lot of fear of the open source models coming from, most notably China. And I understand that fear, because the complexity of these systems is such that it's hard to imagine anybody really understanding every aspect of an LLM and then being able to say, well, I can use this open source model, and I'll just disable that and I'll do my own distillation using this data. In any event, it's a very interesting academic exercise on thinking about what's the value of the data that you're putting into the LLM. Who should own that value? Because that's ultimately what we're after. Here is the value itself and the mechanism for collecting that and the mechanism for analyzing that data.

All right. On that note, as always, we've got all of the articles linked. If you have any questions on anything you've heard today, please feel free to reach out. If you've got stories you want to highlight for us, please feel free to reach out. Otherwise, we'll talk to you next week, like subscribe, rate, review, all that good stuff. Have a great week. Bye bye.