This Week in AI Security - 5th November 2026

All right. Welcome back to another episode of This Week in AI Security, coming to you for the week of the fifth of November twenty twenty six. We've got a lot of stories to get through. And I'm a little short on time today, so it's going to be a quicker than usual episode as we run through a couple of interesting themes from the week past. So without further ado, let's go ahead and kick things off and get started.

So first of all, the story of this week that I really want to spend the most time focused on is actually leading off today. So that is a defense contractor giving their personal AI agent access to a secure Pentagon system. This is sources who disclosed to Wired. Kudos to Wired, great technology publication. If you're not following them, I do highly encourage that. They got these tips from a number of places that an Accenture employee connected a secure DoD, Department of Defense, system holding controlled unclassified information to their personal Claude Code account so that an agent could write and deploy code. They gave the agent their own login credentials, which gave it admin access, and let it bypass the controls in force at normal login. So the agent also needed outbound internet to do work. And so doing that, you can potentially open up an access chain from a secured system to the public internet. So credentials, passwords, and a map of the NIPRNet appear to have been sent to a third party site. Wired won't name the site, but says that NIST has already flagged it as a high risk site because it carries a flaw, that lets attackers inject instructions into a coding agent to steal credentials. This ran for two months before detection. Under normal circumstances, a contractor has to report incidents like this within seventy two hours. Accenture apparently did not initially report the incident at all. The Accenture allegations, quote, found to be without merit, according to Accenture. But they won't say who investigated, and they don't dispute that an incident occurred.

Claude, you may remember from the Mythos time frame and from some of the, let's say, political ramblings going on at the time, Claude is officially on the Pentagon's banned list. So when you take into consideration all the things that went into what is definitely a security incident, whether these details are exactly as reported by the Wired story. But bear in mind that Wired does follow journalistic standards, so two sources confirmed, double checked, etc. Now, while they won't name the sources and won't name the site, that's typical. I don't find anything off about that. Accenture's denial, you know, who knows? Who knows exactly where the truth lies. Maybe their own investigations didn't find the same set of facts. Maybe they didn't go deep enough. Who knows? But let's just say you take this at face value. Think about all the things that kind of went wrong in this chain.

Number one, an unapproved AI service. Start with that on the basis, right? So this is a shadow AI system operating in a network where that vendor is explicitly banned. But there were no technical controls in place that prevented staff on that network from using it. So that's actually two items in there. Number one, unapproved AI service. Number two, no control to block it. Number three, the personal account as opposed to a business account. And let's just pause for a second and highlight here what some of the key differences are. The main key difference is not actually the technology. The main key difference is the contractual obligations between the user and the vendor. In this scenario, when you sign up with an Anthropic paid account, you already get a certain level of access that is slightly above, let's say, the free tier. But you definitely also get access to a contractual level that covers things like security and data handling around your prompts. Typically, you'll also have a training opt out that is just like a default structure of a paid plan. But when you upgrade from that to a business plan or to an enterprise plan, you get additional certainty around data sovereignty, data residency, and other things like that. That was not the case here. This is a personal Claude account, and that is explicitly called out in the article. So that's maybe number three or maybe number four, depending on how you want to look at it.

Number five, default credentials shared with the agent. That's not best practice, right? Best practice is a unique identity for that agent, for the purpose of that agent, scoped ideally to least privilege. I know that's not often the case in a lot of these, what I tend to call personal AI agents. But that's just a really crazy set of things. And then the last thing that I'll say on here is the disclosure time period. Seventy two hours is the contractual obligation. And the reports here are that there was no reporting, and it ran for two months before it was found and identified by a third party source, not by the contractor responsible for this. So that's a whole lot of things, but all of it stems back to, like, really the first two things that I mentioned, lack of visibility into what's going on there, and then lack of policy enforcement about what is and is not allowed. And you could extend that policy enforcement to, let's say, a situation like, you're allowed to use this tool, but only the licensed, company approved version of it, where the company has an audit trail, where the company has visibility into what actions were taken, etc. So that is probably the big story of the week. And unlike weeks past, I wanted to lead off with that today. All right, moving on to the next stories of the week.

I want to revisit the LiteLLM story, which has been reported on a couple of times here on This Week in AI Security, going back several months now to the original compromise. And what's new in the story this week that I wanted to highlight is that there's some more information about two aspects of it. Number one was how the initial breach happened. There was some early speculation that it happened around exploiting a vulnerability. But what it seems to be the case now is that on the twenty seventh of May, an admin API key was committed to a public repo, and it was picked up from there within three hours before it got deleted and then reuploaded. But nobody followed through on revoking the key, which is really the key thing. And now the attacker sat dormant with it for a period of months before using it to read the agent platform AI API and then push a new version out there. That was the malicious version. The next thing about it was that when they used that API key to get access to the version, the token that they got back had overly permissive access and allowed them to publish, and to reject the push request for overriding the malicious version. And basically every downstream control worked, but none of them mattered, in that it was caught by an outsider within a twenty minute period. Kudos to the team over at Upwind. But it had been live for some time, and it was, you know, forked and pulled a number of times. So I just wanted to kind of highlight what we know now about how this whole thing kicked off. And yeah, that's basically it on that side.

Aside now, along a similar theme, which is around the fact that, you know, LiteLLM is not the LLM itself. It's kind of the gateway to using LLMs, right? So we have a vulnerability identified by the folks over at Symantec in a relay service used by dozens of organizations, including industrial automation, semiconductors, pharmaceuticals, shipping. So a lot of heavy industry using this, mostly mainland China, plus some multinational subsidiaries in South Korea, Taiwan, and Malaysia. But what it does is it installs the real CLIs, then rewrites their config. And the override of that config sets the sandbox mode to, quote, danger-full-access, end quote, which enables network access for the write sandbox, and it swaps the real API credentials for relay issued tokens, and disables telemetry and error reporting. So if you think about what could happen here, basically it could intercept your key, replace your key with a key of their choice, which will route to an attacker controlled environment, turn off your audit trail. You won't have any visibility into what's going on there. This was actually behaving as designed, but reconfigured. And so my theme here to call out is that, remember, it's not necessarily the AI, the AI agent, or the LLM behind it. It's a lot of the plumbing and the infrastructure, including the software infrastructure that you might be using to run that environment. All right, moving on to our next story.

And this is also on the topic of configuration rewrites. So we have a CVE, a relatively high severity here. And what it comes down to is a Copilot in Visual Studio or SQL Server Management Studio that bypasses a path validation check on the mcp.json file. And the mcp.json file with a trailing dot, a trailing period at the end of it, resolves to the same file name inside a shared Copilot DLL called Microsoft.VisualStudio.Copilot.Core.dll. And what that means is that if you have two versions of that file, or if you get a malicious version of that file with a malicious set of instructions, that will actually open it up to remote code execution on there. The DLL doesn't do a path resolution to check whether it actually, you know, has this trailing dot or not, and invalidate the copy that does have the trailing dot. It will pick up that file and, you know, read it and execute on it. So just a CVE to watch out for. Patch if you haven't.

And I wanted to follow up, and kudos to the team over at Check Point. We talked last week about the Qwen model and the phone home that appeared in the Cohere team's testing of the Qwen model. So Cohere reported last week, in case you missed last week's episode, that Qwen 3.8 Next attempted to curl a payload to an Alibaba Cloud bucket in Beijing, that was interpreted at the time as kind of a phone home, meaning, like, an open weight model that shouldn't be connected to some back end infrastructure, basically sending some data from a local instance back up to that environment. Now, the Check Point team, to their credit, they reran the same probe string sixty four times to try to check what was going on here. They got fifty two distinct host strings, seven using AWS host names, six using Alibaba, twenty that are local or internal looking, eleven public looking domains that don't exist, and one that's not even a valid top level domain. So it doesn't appear that there was, let's say, a single hard coded URL like this Alibaba bucket that was observed in the first instance. But maybe this is something called model jaggedness, which is an explanation of why the model might have tried to reach out to a number of different locations. It has a set of locations in its training data set that it thinks it should be sending payloads to, and it might cycle through them differently depending on different contexts that go into that. And if you know anything about, like, let's say, LLM calculations and inference, you know that depending on the context window, depending on previous conversations, it might nudge slightly one way or the other as far as which URL it might go out to. Now, none of this is to say, first, that it's not surprising that an open weight model is phoning, quote unquote, home, or phoning anywhere. That's already a surprise on its own. But secondly, this also doesn't disprove the backdoor theory. And it could very well be the case that there is a set of backdoor locations that it tries to reach out to, because maybe it doesn't have access to Alibaba, maybe there are network filters in place, etc., etc. So it's just something to be aware of at this point. And it again kind of goes back to a comment I made last week, which is that, let's say you're pulling an open weight model thinking, well, I'm just going to own and contain this model for my own purposes. Are you giving that model internet access, or access to tools that can go access the internet? Do you know for a fact that, let's say you've maybe air gapped the model, so it couldn't, even if it wanted to reach out and phone home and submit a payload somewhere, do that? That's, I think, the thing that, more than anything, took people by surprise around the behavior that was observed. All right, moving on to our next story.

This is a little bit of an update, and also, again, kudos to the Check Point folks who had some stuff or some data around this. We have talked any number of times around the fact that if you have an IDE or a development environment, anything that it can access and read in text format is an attack surface. And we've talked about the fact that this is, you know, comments and pull request comments and Jira tickets, names of Jira tickets, text files attached to Jira tickets, whatever the case may be. Anything that your IDE or your agent powered IDE can read, it may execute on. And the interesting thing that the researchers tested here was, is this consistent behavior? And they really tested two aspects of it. Number one is different models. Number two is different placement of the files. Will an unsafe instruction be followed if it's in a Readme or a Todo file or a config file or a code file or a data file, etc.? So they checked against forty long horizon tests, fifty six attack scenarios, nineteen different LLMs, and they put the mean attack success rate at nineteen point six percent. So about one in five times. And that's, I think, the key thing to know about, is that this is a relatively, I would call twenty percent of the time a relatively high probabilistic rate of success for any of these attacks. Definitely something to be aware of. The paper is available on arXiv.org, etc. All right, moving on to kind of our last set of stories for today.

First, this is that OpenAI, again, observed one of its research models going in an unexpected direction, in this case exploiting a misconfiguration in a DNS resolver to gain limited access to the live internet during a training run. OpenAI contained it, then paused training and evaluation of its most capable models, hardened infrastructure and monitoring, and only resumed real life inference training on the first of October. So they were paused between the twentieth of September and the first of October. And again, kudos to them for being public about this. This comes from their own blog, and I really appreciate when they do that, and Anthropic as well, who's also been very open and public about observed kind of misbehaviors that they've seen around this. So one of the updates that also came out of this is that OpenAI has now announced that to train browsing without the real internet, they've built an offline world, which is an indexed corpus that all of their models now train on. Containment is also achieved using defense in depth, restricted containers inside micro VMs with their own kernel and hypervisor, plus network and cloud perimeters designed to hold agents in even if they escape the initial sandbox, which is in kind of that container or VM environment. So that is something to really pay attention to. But one of the other things is a little bit of an update, which is that, as the red teaming continued, the number of vulnerabilities and misconfigurations the agents found declined significantly, which is some quantitative evidence that hardening does work. So when you apply these controls, the model was actually much less able to go kind of, quote unquote, off the rails, if you want to call it that way.

All right. And then we have some observations around a new threat actor group called Vapor Panda. I don't know if it's a new threat actor group. I wish, these naming conventions around the threat actor groups, I just wish everybody would settle on one. But this is the Vapor Panda group, as so dubbed by the CrowdStrike Overwatch group. They have identified a campaign using RTX, along with the Chinese developed DeepSeek LLM and a second commercial LLM, to conduct intrusions against multiple high value targets in parallel. This is across the Middle East. This is across Eastern Asia and also across Greece and Latin America. Also probing U.S. semiconductor, defense, nuclear, and pharmaceutical, telecommunications companies in the U.S., Belgium, and the Philippines. Nine confirmed compromises, password spraying against twelve more. I think the key thing to observe here is just, you know, the continued use of LLMs for this, for doing everything from locating new tokens, from accidental code commits, on a much faster basis, extracting them, putting them in, and then just also, of course, the speed and the scale of the attack. So in one of the instances, actually going back earlier, they found four hundred and fifty thousand requests in two thousand five hundred scripts across a single organization's environment in a short period of time.

All right. And the story that I want to close out today on is also on the theme of some developments in Asia, which is that Microsoft has been mapping North Korean threat actors across four adoption stages of AI, and they are all in the, quote unquote, operationalized phase, meaning that AI has been embedded in their workflow routines. This has been observed now in campaigns through three different fronts in a single week. One of these campaigns generated dozens of tailored lures in minutes. One built a new capability to trojanize a private source code repository. Some are also using the RTX framework, which is the same framework that we discussed earlier in this week's episode. And then it looks like Chinese language threat actors have taken that and developed it into a proprietary framework for launching their own attacks. So this has been observed by a number of different organizations who I think are collaborating on some of the threat intelligence around them. I really always, always, always have a ton of respect for organizations who are willing to actually put the greater good first, as opposed to using it for their own advantage. So it looks like included in the disclosures here are Sophos, Check Point, and Microsoft, just to name three of them. So a lot of respect to those teams for collaborating on this.

And that is where we will leave today's episode. I think the story for me that really stood out from this week is, again, that first story about the contractor hooking up an unapproved AI engine to a sensitive network, using their own credentials on a personal account. All of that happening in maybe one of the most blatant shadow AI violations that I've seen so far this year. And that is the note that I will leave you on. And if you want to eliminate shadow AI from your own organization and get visibility as soon as something like that does present itself in your network, please do hit us up over here at FireTail. We'd love to talk to you. Until then, we will see you next week on This Week in AI Security. Bye bye.