This Week in AI Security - 30th July

All right. Welcome back to another episode of This Week in AI Security, coming to you for the week of the thirtieth of July, 2026, the week in the run up to Black Hat. I'm really excited. I know we're going to see a ton of stuff next week in terms of new disclosures and whatnot. So I expect next week's episode to be a doozy, even if apologies in advance, it may get to you a day or two late. But let's dive into this week. We've got just a few stories to get through, which again, leaves us some time to focus on some of the bigger stories and dive a little bit deeper.

So we're going to get through a couple of the quick first ones pretty quickly. First, we've got in the AWS API MCP server has a policy bypass with startup failure. This is basically the classic kind of fail open versus fail closed design. And you could think of it as a little bit of a design flaw from a security perspective. And the basic idea is this if on startup, the security policy data doesn't load correctly to give the API MCP server the right kind of like apply, um, access and deny lists to the API MCP server. Then it just fills open and allows everything. And in a lot of kind of firewall or firewall or WAF kind of scenarios, this is the default behavior because the logic is basically the following. If I don't have the appropriate rule set, I don't want to block potentially legitimate traffic because that legitimate traffic could be, you know, customers, partners, new revenue, whatever the argument may be. And so that is in fact the case here. This was fixed in version 1.3.47. Your IAM permissions were still the boundary.

And so there's kind of two things I want to highlight around this. Again. Number one, it's not the LLM necessarily. It's all the infrastructure around what you're building. It's not the MCP server itself, it's the infrastructure around what you're building. So that, you know, that theme we've had, I don't know, probably fifty times in fifty episodes at this point, but the other one is the IAM boundary that we talked about here. So your IAM permissions were still enforced even if the API MCP server did fail. Open and allow traffic to come in if requested, an IAM boundary. That IAM boundary was still in place and would deny access to, you know, whether it's services, data, or what have you. Um, but what that actually implies is that you really got to think a little bit about what that IAM boundary is. And, you know, the default recommendations around IAM, especially for new services like this, number one, go with least privileges, really only provision access that you need. And we know all the challenges associated with that. So I'm not going to get into that right now. And number two, don't reuse IAM roles. IAM roles are free. You can create as many of them as you need. So please follow that guidance.

All right, moving on to our next story. We talked, I think last week and probably the week before about slop squatting, which is, again, this behavior by threat actors to kind of create and advance hallucinated packages for various functionality. I think I gave the example last time of, you know, write PDF as a Python library and Python or a coding agent would hallucinate that a write PDF dot py function is a common open source library around this. So just some additional research around that, because some of the previous research said that, you know, I think they had tested, uh, I can't remember exactly what number the original research had tested on how many packages would be hallucinated consistently across different LLMs. So here's some reinforced data around that. So fifty three different flops coding targets across Claude Sonnet 4.6, GPT-5.4 Mini, Gemini 2.5 Pro and DeepSeek 3.2. And they all have hallucinated the same fake package names about eighty five percent of the time.

But here's the twist that I actually really liked about this research. Why? I actually included it in this week's roundup instead of saying, oh, we've already had this story. The. When you applied it not to package names for use inside software code repositories, but for skills inside AI agents. The prediction went to one hundred percent consistency, and we've had any number of supply chain attacks around different skills and packages. So that aspect of it, I think is really interesting. And I do think we are going to see an uptick in the usage of local agents installed for personal productivity purposes. And as we move in that direction. You know, workers like myself included, are going to need our agents to have skills. I use an agent, for instance, to put together the stories in this week's roundup. And then it works with me on the final cut. But for instance, one of the things that my agent doesn't do, because it doesn't do it very well right now, is build the slides that we're using as the background, but it doesn't do that right now. Well, I do expect it will do that well, and I'll be pushing that part of the task off to my agent in the future, but it's then going to need to skill a skill like take screenshot a skill like build PowerPoint slides, or whatever the case may be. And it's those skills that are one hundred percent consistently predicted by this slop squatting technique. So that's going to be something to keep an eye on in the future.

All right, moving on to our next story. And we've got a maybe bug, maybe a security flaw, maybe just design flaw in the Claude Cowork Linux VM. And so what was found is that the Claude Cowork can break out of the Linux VM environment to the host Mac. So this is specifically for Linux virtual machines on Mac operating systems, and that the Claude Cowork can then find SSH keys and cloud credentials. 500,000 macOS users are reportedly affected. I guess that's the number of people who have this Claude Cowork version installed. And what it relies on is the fact that the virtual machine shares folders via a root daemon. So between your macOS and the virtual machine, there is a coworker that uses the Apple virtualization framework and both kind of your your host operating system and your virtualized operating system use that same folder. And so it's really just using that folder and the sharing permissions on that folder to access the credentials and then potentially break out in read write files anywhere on the host operating system.

This was reported to Anthropic reportedly closed as informative, but no fix necessarily provided. And I kind of understand it because, you know, Anthropic doesn't control the host operating system. That's Apple. And this seems to be something in the design of the Apple Cowork D as opposed to something that Anthropic could have control over. But one of the interesting things around it, this is the second IDE sandbox escape that we've covered in the last several weeks. So it is something to bear in mind if you're going down the path of creating virtualized environments for IDE purposes, specifically for IDE purposes.

All right, moving on to our next story. What we have here is a CSRF. So that is a CSRF in ChatGPT workspace agents disclosed by the team over at Zenity. Kudos to the Zenity folks around them. And effectively the structure of it is the following. One phishing link can silently build and authorize and deploy an attacker controlled agent inside your org with the access of the employee whose environment this is running in. So what this really refers back to is the fact that, as we've talked about, all of this kind of, uh, agent driven behavior that reads text from various locations and performs actions on it is vulnerable to, you know, quote unquote, indirect prompt injection, basically plant malicious instructions in various places. And so CSRF is kind of a cross-site request forgery. It's a way to structure requests. And we talked about, I think last week or the week before, I can't remember specifically, we talked about prompt injection through URL request parameters. Very, very parallel. Very similar thing. Anywhere you can embed text instructions is potentially vulnerable. And in this case it is the ChatGPT workspace agent who again, our agents are going to read text and potentially execute on that text.

On that same line, the report from Sophos, their AI Security 2026 report, reports that agents are one of the highest value attack surfaces for attackers. Interestingly, from the Sophos perspective, and this is something that I did want to call out here, it is the identities tied to these AI agents. That is actually what makes them particularly high value. And it kind of ties into a couple of themes that we've talked about this week, which is that the identity is the thing that very often governs the access to different tools, data sources, etc.. Kind of going back to that AWS API MCP flaw that we led today's show with. So these AI agents, if you find ways to get into them. If you find ways to insert malicious instructions that they might act upon, and you can get the AI agent identities, things like OAuth token service credentials, those permissions, those are actually the high value targets because those will allow an attacker access to valuable data. So that is really one of the things that I would hammer home around them is think about, you know, the provisioning of the credentials and the permissions that you're assigning to agents.

But the other aspect of it that I would really hammer home is, do you have visibility into all of the agents that are running inside your environment, whether those are agents that you're building on cloud platforms using something like ChatGPT Workspace agent, or do you have agents on, or do you have things like open claw and locally deployed agents that your users are using for personal productivity purposes? Big question. You know, kind of the cardinal rule of security is without visibility, you can't really do anything about it. So that's a theme that I would hammer home. In addition to that identity theme and the value of the tokens, etc..

All right. Moving on to our last story of this week. This is the one that I wanted to spend the most time on. So we talked last week about the what is now being termed the open face breach, which is the OpenAI new model that was in some cyber capability testing environment and broke out of that environment and went on to, quote unquote, attack Hugging Face and all of the things that happened in that kill chain around the exploit of the proxy, zero day vulnerability, the exploit of the API attack surface that allowed it access into the Hugging Face environment. I think that aspect is still not one hundred percent clear, but okay, let's put that aside for a second. The point is, the agent kind of, you know, went outside the environment or the the model went outside the environment using some agentic functions, etc..

What was interesting from that story was, you know, and we posted a follow up article around this on our own blog around what allowed these two organizations to respond so quickly was the fact that both sides had telemetry and observability data around the actions, and they were able to look through the logs pretty quickly and then kind of parse out what exactly happened. And so this thing went contained actually relatively quickly. If you think about the speed that it actually occurred at. Right. So that is a real win from the defender's perspective. And then of course, there's other wins around, you know, organizational cooperation and the mindset around being open and transparent around how this all played out. And I actually say kudos to both organizations, and especially to the fact that they decided to collaborate as opposed to being like, uh, you know, kind of circling the wagons and staying quiet or being defensive about their own role in that. Obviously, more on the OpenAI side than on the Hugging Face side as the kind of target of the of the incident.

But one of the other things that came out of that, okay, great. You've got the observability data, but there's so much of it. It happened so fast. You want to respond very fast. Well, you need AI in order to respond to that. And if you remember from the reports and from the kind of post-event event postmortem, it came out that they couldn't use most of the major commercial models Hugging Face, rather couldn't use most of the available commercial models to analyze the log files. Why? Because in the log files, you saw a lot of malicious prompt activity, and the commercial models kept just, you know, blocking, blocking, blocking the analysis because they would say, hey, malicious prompt airdrop and they wouldn't actually dive any deeper. And so Hugging Face turned to a, I think now the officially recognized term is open weight model. And so the availability of that open weight model was actually a killer, you know, kind of key component in Hugging Face's, um, response to the overall event. So really critical piece of infrastructure and really critical tool in, in the whole process.

And one of the challenges around this is this kind of geopolitical standoff where most of the providers coming out of the US are not open weight. Most of them are private commercial models, right? The two biggest and most obvious being OpenAI and Anthropic. But you know, other companies, Google, certainly with the DeepSeek and their Gemma family of models and Amazon with their family of models, I think Nova is the name for the Amazon proprietary ones, and Microsoft's got some of their own models in addition to their key strategic alliance and partnership with OpenAI. All of those are commercial, closed source, closed weight, you know, proprietary models, right? And so the question then becomes between the US and this administration's stated goal of leading the world in AI and applying things like export bans on Anthropic models earlier this year. There's an argument amongst the industry like what's going to happen here? What's the role of, I'll just call it like open versus closed models.

And interestingly, a whole set of big tech leaders, um, Nvidia, Palantir, etc. came out and said, look, we're actually in support of open weight models because they don't apply some of those same restrictions. Now, it may sound very self-serving and easy for them to say from their positions where they have access to all the GPUs and the infrastructure to be able to effectively run these open weight models. But one of the things that I think is actually really interesting is that these models are actually coming down in terms of the requirements to operate them. And so it's actually becoming more and more realistic that you could see maybe not the entire model, but a subset or like a smaller version, a lighter weight version of an open weight model that is specifically tuned for one particular use case. In this case, you know, log forensics would obviously be that use case and relying strictly on these closed models, it doesn't necessarily make your organization or the internet as a whole any safer. It just means that you're limiting access to various capabilities.

The counterargument is, obviously that these open weight models don't have the same level, necessarily, of ethical guardrails applied to them. And we've done some of our own testing here at FireTail around, you know, generating malware, generating phishing emails, spoof identities, etc. and true and true to fact on the open weight models. It's much, much, much more easy to accomplish those tasks. And so that's the counterargument. Well, you don't want open weight models that allow people to do bad things in the world. But if you kind of take that argument and you zoom out with it for a little bit, you would also cut off internet access to pretty much anybody who has a malicious intent, and you just can't know who those people are.

So I appreciate the kind of the broader perspective on this. And I do think from a personal perspective, not necessarily speaking on behalf of FireTail, I do see the value of having open weight models in the ecosystem. They allow you to experiment. They allow you to kind of push certain capabilities forward for various purposes, whatever they may be. And yes, potentially including malicious in some cases. But I think there is more upside than downside. It just does reinforce that, like from a defender's perspective, if you know that these things are out there in the world, you have to consider your own defenses, what the threat model around your own organization is. And if you are as somebody probably listening to this podcast, I would, I would guess you probably are in this boat, somebody who's trying to enable AI adoption and enable the deployment of large scale AI, including, you know, agents agentic workflows and what have you. The importance of the core basics of, you know, inventory visibility, observability, and policy enforcement. These core cybersecurity fundamentals actually becomes doubly important in that context. And I think that'll be my final thought for today's episode. Thanks so much for staying with us. We will talk to you next week on another episode of This Week in AI Security. Bye bye.
‍