This week’s episode covers ten stories in four themes. The main one is the rise of autonomous, agent-driven attacks.
.png)
This week’s episode covers ten stories in four themes. The main one is the rise of autonomous, agent-driven attacks. Jeremy covers Operation Snowball, a jailbroken agent that spent two weeks inside a European telco and 13 other organizations. He also covers Railway’s 26-hour fight with an attacker who used an open source offensive framework, and GitGuardian’s census of 45 agentic attack toolkits that are already exposed on the open internet. Next come the breaches at Fireworks AI and NEAR AI, plus the LLMjacking economy behind them. The story Jeremy spends the most time on is how a harmless “yes, don’t ask again” click in Claude Code put more than 31,000 secrets on public GitHub. The episode closes with backdoored coding agent plugins, OpenAI’s new Control Room, and the 35,000 unauthenticated apps built on Grok, some of them by employees of SpaceX and X.
Key Stories Covered:
settings.local.json files, and 5,343 of them were still valid.Episode Links
https://malbearlabs.com/posts/operation-snowball-inside-an-autonomous-ai-intrusion-campaign
https://blog.railway.com/p/incident-report-october-security-incident
https://blog.gitguardian.com/ai-pentesting-tools-beyond-artex/
https://near.ai/blog/security-incident-report-sglang-vulnerability
https://github.com/microsoft/vscode/issues/347426
https://www.cert.europa.eu/blog/beyond-the-model-backdoors-in-ai-agent-harnesses
All right. Welcome back to another episode of This Week in AI Security, coming to you for the week of the twelfth of November twenty twenty six. We’ve got a lot to get through this week. So without further ado, I’m going to go ahead and jump in because we’ve got about ten stories across four different themes to go through.
So first we’re going to start with Operation Snowball, which is an investigation inside an autonomous AI intrusion campaign. And this starts from a discovery of an open directory that exposed seventy plus session transcripts from a jailbroken DeepSeek Flash agent that ran autonomously for two weeks against a European telco and thirteen other hosted organizations. The operator used a seventy eight character, sorry, a seventy eight k. System prompt based on the little code source jailbreak template with zero recorded refusals across all seventy sessions. And that kind of again, speaks back to the jailbroken DeepSeek Flash instance that is, you know, had its ethical and cybersecurity guardrails removed.
It looks like it exposed thousands of subscriber billing records and reportedly reached a TrakCare healthcare database via default credentials, potentially exposing millions of patient records. Looks to be a Chinese language operator. No attribution to any government, nation, state or threat actor group currently at this time, but this is the most recent in a series of quote unquote, agentic powered attacks that we’ve seen. And we’re starting to see the formal development of toolkits that are based on LLMs or agent backed platforms to launch cybersecurity attacks.
All right, moving on to our next story. And we have an incident report from Railway that came out on October twenty first. It traced a twenty six hour running battle with the first exploit detected on the twentieth of October. The attacker fully evicted by the twenty first of October, a previously unknown vulnerability that partially escaped build file system isolation, yielding a build credential. That Railway’s internal Temporal cluster wrongly accepted as a trusted client. This allowed the attacker to read other builds shared histories, including a snapshot of Railway’s monorepo and Railway’s words. It gave a privileged view into how Railway works, enabling them to run agent fan outs, find any exploit and move laterally with significant speed, end quote.
Railway realised that they had been compromised only when the code hit probe endpoints that had been dead for months or years. And just as I mentioned in the previous story, this is being tracked back to kind of an open source framework for attacking purposes, a framework that is currently being dubbed RTX by Wiz, and a number of others who have observed this out into the wild. There’s an erroneous OIDC binding that yielded short lived cloud credentials, then SSH, then a host holding the variable decryption key, which decrypted environment variables for at most about two percent of the workspaces. But interestingly, it did allow for all of that access and lateral movement within the environment. So a pretty interesting to observe this.
And on the same topic, GitGuardian catalogued forty five open source agentic offensive frameworks. RTX being just one of them. They created fingerprints for four of them. Forty one of them rather on Shodan and the RTX family alone matches one thousand four hundred and twenty eight internet reachable services. As of the seventh of November. Eleven of the forty one serve a web console, which does suggest that if there’s a web console, there may be kind of a business model or an affiliate model created around them. The creator of RTX pulled the repo down on the eighth of October, seventy four days after the first commit, but one hundred and two forks and RTX named repos appeared in the takedown week alone.
HexStrike, which has over one hundred and fifty offensive tools and an MCP, and Agent Zero have all shipped with no authentication by default, meaning that these tools are available in pretty much anybody who has the ability to pull basic things down and execute them doesn’t have to go through really any hoops in terms of setting that up and running them for their own purposes. The clusters sit in a variety of different hosting providers, and mass deployment from one Docker image looks like it could be deployed to any number of other environments. So definitely something to be aware of and keep track of. And really, there’s no, let’s call it unknown attack techniques in case in enclosed within these agentic attacking frameworks, they’re just really good and really fast. Right? And that’s the thing to keep in mind here.
All right, moving on to our next story. And this is from Fireworks, which is an NVIDIA backed company. They disclosed a second breach on November ninth. Attackers pivoted through a compromised third party cloud provider on October twenty fourth to reach developer machines and exfiltrate customer data. The exposed exposed data includes billing info, resource metadata, and, for some customers, incomplete model weights and data sets, which is actually a really interesting wrinkle in here. The. The importance of model weights is something that you should not neglect. That’s going to be one of the most important things going forward in LLM development.
The first breach was in early October, which exposed API keys. Both have been linked to LLMjacking operations. They are currently working with Palo Alto Unit forty two on incident response, and they have also connected to Google, Mandiant, and the finding that is driving some of the incident response is that LLMjacking is driving record costs up towards one million per victim in cloud AI charges. And that’s one of the key things here is, you know, the the main goal is actually resource consumption. So definitely something to keep an eye out for, for your own cloud environments and for monitoring the importance of monitoring your AI consumption as you go.
All right, moving on to our next story. And this is a breach that happened to a company called NEAR AI. On November eighth, an attacker exploited a remote code execution flaw in the SGLang package, which is an open source inference engine, and that allowed them to access internal systems of NEAR AI, which allowed them to download account details for around one hundred users, including chat histories, settings, logs, etc. from eighteen legacy AI agents that had been built by those customers. Two cloud customers had their full API keys exposed in those logs. Those keys have since been revoked, but we don’t know as of right now, at the time of recording, what impact that might have had on those customer organizations.
NEAR AI took the response steps of taking the private chat offline, revoking all sessions and agent tokens, deployed a fix to this, and then notified all of their customers. So great response from them. Have to give them credit for that. That is a situation that I think we’re going to see all too common going forward, that we see these types of events. So the importance of an incident response plan becomes doubly important.
Moving on to a story, I want to spend a little bit more time talking about today. And this is the Claude Code leak of Thirty one over thirty one thousand secrets and tokens that have been exposed on GitHub. And the interesting thing here is the way that this happened. So how did this happen? So Claude Code, which is Anthropic’s AI coding agent or what. I would actually tell you, it is kind of a desktop based assistant that allows you to automate coding tasks, saves. And what it does is it saves permanently approved commands, including passwords and API keys that are contained in them into plain text and project files, specifically, the project file that is contained within your Claude settings dot local dot JSON file. And thousands of these are public on GitHub.
So GitGuardian scanned GitHub on as of the ninth of November for this particular file. So this kind of that Claude path. In these settings dot local dot JSON, they found thirty one thousand and eighty eight secrets across twenty three thousand two hundred and seventy six repositories. They then tested those keys and found that five thousand three hundred and forty three of them were valid, And another firm had actually found a similar number across them. Anthropic themselves were not breached, but particular customer accounts might have been. Those companies have been notified in order to rotate credentials around that.
And the interesting thing about this is that this came from something that seems like a very innocent user behavior. So what will happen is that you’re working on a coding task within Claude Code. And Claude Code will ask you, do you want to save this? And you’ll say yes. And at certain steps there comes a yes and don’t ask again option. And when that happens, it triggers a setting within Claude Code that will automatically commit all of these. And so all of these secrets and tokens then get written into that file. And in many developer configurations, that file is actually uploaded into their GitHub repos as an important file. That’s part of their kind of coding environment, something that they wouldn’t want you to, to, to lose or forget about.
Now Anthropic’s guard adds the file to your global git excludes. And that is one way that it protects it from being uploaded. So it wouldn’t actually get uploaded in that case. But what that does is it protects one laptop and it doesn’t travel across all of the environment or all of the organization’s end points. And it may not even also get added to the gitignore file, which which would then prevent it from being committed. So the median file had sat on the default branch for two hundred and seventeen days. And the research done by GitGuardian.
This has basically kind of digressed to where everybody is discovering these at this point. And it’s just one of those things that you’re going to have to work with your team to make sure that they’ve got their own Claude Code configurations correct, because this is, again, not a malicious user behavior. It’s a perfectly reasonable human reaction to say, this is going to slow me down in the future. I already know that things are working. Let me remove this friction. So don’t I don’t need to approve this every single time.
All right, moving on to our next story. We’ve got a couple of different vulnerabilities in the VS Code. This has been updated in the latest Patch Tuesday. High severity vulnerabilities that let a malicious MCP server achieve root root code execution through crafted elicitation notification, and another, which is a crafted install link that omits security relevant details. Both are fixed as of VS Code one dot one four five dot three. This is the third MCP related VS Code flaw in the last six weeks. We’ve covered some of the other ones. I don’t need to go into them in detail now, but there has also been an MCP TypeScript SDK with a CVE around it as well. Back in October.
So it’s just kind of more evidence of one of the themes that we’ve talked about here, which is that as we are moving as an industry, as a technology industry, super, super fast with AI development, the tools that we’re using to build things often have vulnerabilities and security flaws in them. And that’s just more evidence of that. All right. Moving on to our next.
And continuing on the theme of it’s not always the AI, but it’s very often the tools that we’re using to build around the AI. The CERT-EU AI team has patched local copies of two legitimate, widely installed extensions, one of which had about one point six million downloads a month. And these are for Claude Code. Two dot one dot three zero four and later, and also for Pi one dot three dot zero. Both of them hid a payload inside event handlers that the harnesses already run. In Pi’s case, it was a before agent start, and in Claude Code’s case it was a user prompt submit hook.
The trigger was an ordinary code review phrase LGTM, ship it, which is basically like, let’s go to market. Ship this and on trigger. Both of these were backdoors that would exfiltrate host and virtual machine details that would share user account information, network config environment variables, and credential files with no visible tool call and no model involvement at all.
And so the critical finding here is that the agent level file read restrictions did not apply to files that the plugin read directly, and the tool call inspection never fired because there was no tool call. This is actually embedded in the code in these two plugins. So there’s no, you know, let’s reach out to an LLM to decide what to do here. There’s a set of malicious instructions previously embedded. The agent permissions might constrain the model, but they won’t constrain this code that runs right inside that.
And the trigger could arrive via a pasted log file that the agent reads. Or it could arrive via that command being hidden, hidden in really any kind of, you know, downloadable text read environment. So could you could even potentially be browsing documentation that has that phrase embedded. And if you’re using an AI assisted or an AI assistant for coding, it might actually read that. And then if you’ve got that plugin installed, it fires off of that phrase. So remember, it’s not always the agent’s, not always the LLMs. It’s very often the tools that we’re using to build stuff.
All right. Moving on to our next story. So this is a announcement of a product launch from OpenAI. It is called Control Room came out on November ninth. It is centralized system that tracks all research inference workloads, including agentic sampling, and can halt model output within thirty five seconds max. The thirty five second delay comes from a caching layer that OpenAI is currently working to eliminate, so that that thirty five second kind of buffer will go from thirty five seconds down. It will separate the production model serving stack, which already has operational shutdown controls.
And OpenAI, during the same week announced a hackathon partnership for vulnerability hunting and expanded Daybreak read access after the September training escape incident and the Cathedral security sprint. This is where OpenAI wants to, quote, prove we can control it, end quote, and give confidence to a lot of organizations that are looking for ways to deploy secure agents on top of OpenAI and other model providers. So interesting initiative here that will have to pay attention to.
All right. And moving on to our last story for this week. And this is the Grok security blind spot, which exposes users but also interestingly exposes more than thirty five thousand Grok built apps. And so what have we found here? So a team over at a small company called Clara, kudos to them, discovered that over thirty five thousand five hundred active web applications generated by Elon Musk’s Grok AI were deployed to the public internet, completely devoid of authentication. The exposed apps leaked massive amounts of sensitive data, ranging from hospital patient briefings to plain text, employee passwords, login credentials embedded directly into the code of AI build, timesheet apps, and more.
So this is one of these real shadow AI nightmares. When you have non-technical employees who can bypass IT, departments get to a system where they can build things, and that system doesn’t have sufficient security controls to allow non-technical employees to build secure, governed AI agents. Those employees can instantly generate something that is functional, but doesn’t necessarily have any kind of security controls within. It won’t have gone through a secure software development lifecycle process, and a single chat prompt could configure IAM policies. So there’s a lot of risk to this type of deployment.
Um, and interesting about this, you know, this doesn’t just impact random users of the platform. It actually comes largely out of the Grok ecosystem. At least twenty four of the exposed apps were built directly by employees of SpaceX and X. These are unauthenticated apps from arguably in this case, people who should definitely know better. They leaked an internal corporate secrets, a map of nearly six hundred enterprise sales accounts, staff rosters, internal playbooks, confidential advertising proposal for a US federal agency.
So even though the company is building frontier, AI models are, you know, in the know should know how these systems work better and should be doing better. We see that the rush to push things out very, very quickly often override security requirements. And you get this kind of effect where anyone with the link privacy is not, is not is what’s being followed, as opposed to being the right way to approach this privacy or security aspect of it.
All right. That’s it. A little bit longer than usual episode for this week. We had too many stories. And we’re going to spend a little bit of time on the FireTail blog diving into one story in particular from this week that really resonated with me, that I want to share a little bit more of my thoughts on, and hopefully you will find interesting. If you want to understand how your own agents are working, I would highly recommend you join us for next week’s webinar where we talk about how we build secure agents here at FireTail. Happy to share some of our own lessons learned with you. All right. Thanks so much. Bye bye.