This Week in AI Security - 29th October 2026

All right. Welcome back to another episode of This Week in AI Security, coming to you for the week of the twenty ninth of October twenty twenty six. We've got a couple of themes to get into today, as well as I want to take a couple minutes to talk about the State of AI Security report that we prepared over here at FireTail. A little bit about that, so we'll save some time for that towards the end of today's episode, and also mention a little bit of a follow up from a story that we covered last week that we will now have posted a little bit more information on. And so with all of that said, let's get into the episode.

All right. So first, we've got a great story called, I love the title, Markdown Is the New Executable. Now, if you find yourself immediately saying, yeah, well, we've talked about markdown files before, that's only partially true. So we've talked about almost every other aspect of text available in different environments, particularly in the context of development environments. So you might remember that we've talked about, um, commit names, commit comments, anything around code repositories, environment files, anything like that. So what we've got here is a malicious agents.md, or markdown file, found in a customer's downloads folder. Why is this interesting? Well, it's interesting because what we're seeing right now is a dramatic rise in the number of desktop AI applications. You can think of things like ChatGPT desktop, Claude desktop, Grok Bot, Amazon Quick, and on and on down the list. And what we also see increasingly is users hooking these tools up, giving them disk access, connecting them to third party systems like Slack, Teams, whatever the case may be, in order to facilitate their own day to day work, increase their productivity, etc. Now, a downloads folder is kind of one of these default folders that you might want to connect things to and tell your local agent or your desktop tool, that file that I just downloaded, summarize it for me. And so in this case, what you get is you get a malicious agents.md file, which is a pretty basic text file. It's going to pass your virus scanner because there's nothing, you know, no malware inside it. It's just a set of bad instructions. It does instruct you to rewrite the machine wide configuration and install a malicious NPM package in ninety plus percent of the cases. Kudos, by the way, over to the team at Upforever Security for finding this. The config file that it writes to is ~/.codex/config.toml, and that leaves the folder in every future Codex session in any project. It still loads the attacker's instructions, and so you can leave this alone for days and that instruction is still there. The Claude Code variant had a planted CLAUDE.md file, which will reroute the Anthropic base URL to the researcher's domain. So there is a little bit of a debate about whether this was a research exercise or whether this was actually an attack in the wild, but it is a real risk. And so even if this was a research exercise, this is something that can be found out in the wild. Remember, anywhere you can plant instructions, and then you have an LLM or an agent picking those instructions up, those instructions can introduce an attack.

All right, moving on. We've got a story from the team at Cohere. And bear in mind that Cohere is an LLM model provider on their own. They provide the Command R and R Plus models that you might or might not be familiar with, but they were running some open weight models in their security team to test out some of the new open weight models, most notably Alibaba's Qwen 3.8 Flash. Next, they ran it through Microsoft's CTI-REALM benchmark to try to understand how well it performed, and they asked it to write some detection rules. But instead of actually writing the detection rule, it gathered information about the host and attempted to do a curl -X PUT, which is basically an outbound API request to place a payload into an Alibaba Cloud bucket in Beijing. And Alibaba Cloud, think of that as kind of an AWS or Microsoft Azure competitor run by Alibaba, which is a large Chinese technology firm. So the interesting thing here is that, you know, you think you're running what is an open weight model that is disconnected from any company's back end infrastructure, but it is only in a very narrow set of cases where it will actually reach back out. So it is less than one in one hundred thousand requests that will actually trigger this type of behavior. And it's kind of, you know, interesting to figure out, first of all, what types of requests will trigger this behavior, but also put this in the context of somebody who's running, let's say, like a high sensitivity workload for which they think, well, I better actually not use the cloud. I better use an open weight model that I can download and run in my environment. Are you going to either A, run such an exhaustive set of tests that will cover, you know, the one in one hundred thousand case that will actually trigger this type of phone home behavior? Or B, are you going to scan all the source code to understand that this capability exists embedded within that code? And I think the answer for the vast majority of users is they're either not going to have that capacity or that capability. And so this is one of these things that is actually a really interesting kind of risk to uncover from these open weight models.

All right. Moving on to our next story. We've got another disclosure from Anthropic where they disclosed that a researcher pointed Claude Mythos 5 at a statistics project. Claude learned from an archived copy of a state agency's website that its dashboard had access tokens to any visitor. It then requested an access token, picked it up, and queried the database without paying in order to get the statistics data that it was looking for. So it's just another example of an agent kind of, you know, going, quote unquote, off the rails, or quote unquote escaping the sandbox, or however you want to think of it. The way that I think of it is taking an unanticipated, unintended action. I should come up with a catchier catchphrase or a label for that, but I don't have a better one yet. Um, and Anthropic calls this, quote, reward hacking, when it cannot complete a task as given, it works around a restriction instead of stopping, end quote. Uh, they have briefed the organizations in question and cut live internet access from internal evaluations in the meantime. So this is just more of kind of what we have seen over the last several weeks, really kind of going back to the OpenAI Hugging Face incident. We've just had a continuous stream of disclosures of this type of unintended action taken by AI agents trying to achieve goals and perform tasks. And again, I've mentioned it a couple of times now, we're going to see more of this over time until we learn as a technology industry to put better guardrails and controls on the requests that we do. And we've got some work that we're doing at FireTail to help customers get closer to putting that right set of constraints on their agents. Stay tuned for more on that side.

All right, moving on to our next story. And this is kind of a broad thematic story, so you can attribute it to any number of organizations. I like the write up from The Register and how they kind of framed the whole thing very nicely. But this is, quote unquote, Black October. It really stems from the huge number of vulnerability disclosures that have happened during the month of October, and how that's basically created just a massive patching exercise and requirement across the technology industry. The most recent being this PHP2Shell vulnerability, as it's being called. The team over at Wiz calls it, quote, among the most severe vulnerabilities discovered in PHP and arguably in software history, end quote. And of course, we've talked about this before. PHP is behind things like WordPress, also behind a lot of kind of open source web applications typically used for things like internet management, portals of various devices, including most likely your home router or something like that. Um, exploitation has been observed in a number of locations. Any number of organizations have commented on the volume of exploitation, not only of PHP2Shell, but of other CVEs that have been disclosed as a result of all the vulnerability discovery. So there's no particular new news here in this one. It really is just kind of a culmination of something that we've been talking about for several quarters now. Hey, AI is really good at finding vulnerabilities. AI in the hands of the wrong people, who will then use those vulnerabilities to develop exploits and then attack organizations, is a bad thing. We've got to get better at patching, which is kind of, you know, as one cybersecurity friend of mine put it, this is twenty years of accumulated technology debt all coming due in the same month, which I thought was a really nice and succinct way of phrasing that.

All right, moving on to our next story. So researchers found an open source pen testing platform called Cyberstrike AI, configured with forty four specialized agents that was handling reconnaissance, exploitation, C2, command and control, persistence, and exfiltration. They targeted at least nineteen organizations, sixteen of which were in Japan. The platform was left discoverable and open, which was how these researchers found it. It generated Japanese language phishing forms and fake login pages, working targets in parallel while throttling per target traffic to avoid detection. So across those sixteen Japanese organizations, it wasn't kind of like brute forcing them or DoSing them or anything like that. It was a steady stream of attempted, uh, exploit requests. And thankfully, what it looks like is that, you know, despite the scale of this and despite the complexity of it, uh, very few organizations reportedly were compromised. The researchers discovered, quote, no session, no web shell, no exfiltration, end quote. So thankfully, that is the case in this particular instance. But we're just going to see an increased number of these. And we've reported on very similar stories for several of the last weeks on This Week in AI Security. So again, nothing too, too new. But every time one of these comes out with a little bit of scale around it. And also, I think importantly interesting to observe is a very particular, narrow geographical targeting here, sixteen out of nineteen organizations being in Japan, using the Japanese language, etc. That's a pretty interesting development. And we've seen similar targeting of Taiwan and any number of other geographies over the last several weeks.

All right. Moving on to our next story, which is a Warp breach exposes AI agent logs and user telemetry. This is from a company called Warp, who makes a platform called, unsurprisingly, Warp. It is an AI terminal company, and attackers exploited a vendor in its infrastructure and reached agent decision logs and telemetry. Warp has not yet disclosed what was inside each of the logs, but it is telling developers, please do assume the worst. Decision logs and telemetry for one percent of eight hundred thousand plus users, or roughly eight thousand developers, were exposed. The CEO, Zach Lloyd, has confirmed the incident. CrowdStrike has been brought in for incident response. AI agent logs are an interesting class of sensitive data. And part of the reason that they are is because they're actually verbose in natural language. And so if you've ever looked at, you know, kind of the response that you get from an LLM, that's very often present in the incident logs. So the more sensitive your question, the more sensitive the response might be. So if I think about different things that I might ordinarily think of as just telemetry, I'm going to think about server performance, query times, volumes of different queries. Like, I issued a query that was sixty four kilobytes, I got back a two megabyte payload, something like that. But in the case of these agent logs, you're really getting into a plain text version of, how do I segment these customers? And you're going to get back a response that has all of the customer IDs, presumably. And so that is where this user telemetry data can become really, really interesting and relevant. And similarly, if you're not using best practices around management of developer credentials, and you say, build me an application that puts an object in an S3 bucket, here's my AWS keys, you might get code back that has those keys directly embedded in it, and that might again end up in these agent telemetry logs. So, uh, you know, interesting development to think about telemetry data as being more sensitive data than we might have previously considered it in the past. And that's why I wanted to highlight this story. Uh, Warp hasn't named the vendor or the third party that was actually in question here. So if you're a user of that platform, please just, you know, follow their advice, best practices around, let's say, rotation of credentials and maybe changing passwords and all that good stuff.

All right. So two last things to wrap up today's episode. I did want to take a second to talk about the State of AI Security 2026 that we prepared and released a little while ago, back in September. And we recently did a webinar with our partners over at Cloud Security Partners, most specifically their Chief Information Security Officer, Rinaldi Rampen, earlier this week. If you are interested, you can find that webinar recording on our website or on YouTube, and you can also find it just from, you know, contacting FireTail. You should be able to navigate it pretty, pretty quickly. Um, there's a couple of things that I do want to highlight from the State of AI Security report that we talked about just a little bit, which is that we really covered the three big themes of development over the past year. And number one is the rise of desktop agents and the security risks that they bring. We kind of selected OpenClaw being the most emblematic story on that side. Number two is the rise in vulnerability, and vulnerability discovery and exploits. And on that one, we chose the launch of the Claude Mythos model family and all of the controversy that surrounded that, including export bans and things like that. And number three is really around the agent escapes. And of course, we've talked about stories here, but we used the OpenAI Hugging Face incident as really the starting point for how that became a big theme on the AI security side over the course of this past year. So the report is available. It's all open. There's no gate or anything like that, no form to fill out. You're welcome to access that data. If you have any questions around it, please do reach out to us. We'd be more than happy to talk to you about it.

And last but not least, I wanted to give a quick shout out over to a blog post that we just put live today, which is discussing the liabilities around letting agents access your infrastructure. And it's a callback to last week's story from JPMorgan Chase, where they came out with a public statement stating that they are not going to let Muse and other similar personal AI agents access their banking services. Um, now how exactly they're going to identify that, etc., I leave that to them on the implementation details, but I definitely trust that an organization with their level of sophistication has the capability to do that. On the flip side, we've put some time and effort into kind of analyzing what that means. So how do you think about that for your own organization? And if you look at several of the stories that we've covered over the last year and kind of the developments, both in terms of the technology, but importantly in terms of regulation and legal precedent, I think the thing that we took away from this, that is crystal clear now, is that your organization is liable for all the actions taken by AI agents that you or any of your employees undertake. And if you don't have the telemetry data to be able to say what did your agent do or not do, and your agent does a sandbox escape, you know, infiltrates a third party system, etc., you need to be able to kind of provably show what actions it did or didn't take, or you might be in a spot of trouble later on if any incident does arise. So we put some thought into that. If that's an area of interest, please do check out our blog for more on that. And without further ado, I will say thank you so much for taking the time to listen to this week's episode. As always, if you have any questions for myself or the FireTail team, please do reach out to us. And if you or somebody that you know is trying to get an AI inventory of who's doing what, where, who's deployed what agents, who's using what desktop tools to do what, definitely hit us up over here at FireTail. We'd love to talk to you. Thanks so much. Bye bye.

‍