This Week in AI Security - 29th January 2026

All right. Welcome back to another episode of This Week in AI security, coming to you from the Firetail team behind the Modern Cyber podcast. We're recording for the week of the twenty ninth of January twenty twenty six. We've got about seven stories to get through this week, four or five of which I would say are kind of variations on a theme of various things that we've covered on previous episodes of This week in AI security, and then a couple of really interesting stories, one philosophical, one research paper, and then one really real world problem that we are observing right now as we speak. And it's a big one. It's a big one with real implications that kind of illustrate a lot of the fundamental risks around AI tools, AI adoption, and AI systems, especially agentic ones.

So without any further ado or introduction, let's dive in. Let's start going through our stories. The first story we've got is an AI framework with a number of flaws that put, quote, enterprise clouds at risk of takeover. This refers to chain lit, which is a Python package that organizations can use to build production ready AI, chatbots and applications. We all know that chatbots have actually been one of the first kind of main production use cases that has actually gotten out into the wild, and it makes a ton of sense. It's very kind of chat based, like a lot of the earliest systems like ChatGPT, cloud, AI, AI, AI, etc. have been.

So it's one of the systems that is kind of easiest to understand and to conceptualize for organizations. Also, it is a heavy cost center, and it has implications around things like the availability of humans to respond to customer support requests. What do you do when it's lunch break? It's after hours. It's the weekend when there's no human manning your customer support lines. Never fear, I've got a chatbot for that. Now, these chatbots now require lots of different components to build them. And so Channelid is one of these packages that got popular in building, again, Python based production ready AI chatbots and applications.

There are two CVS in this. Their numbers are given in the article. The article, as always with all of our stories, will be linked in the episode. To me, the reason that I call this kind of more of the same is that, you know, there are always flaws in pieces of software. I have yet in my twenty eight plus year career, to see a single piece of software that didn't have some problem with it. Whether that is a simple software bug where it crashes under certain conditions, or a vulnerability that can lead to exploit and takeover, that's what we're talking about in this instance. What's happening, though, is that as use cases rise and pieces of software become more popular, attackers understand that this trend is happening and it's not that hard, by the way, you can look at, you know, you can do a quick Google search for how do I build an enterprise chat bot?

And inevitably you'll come across an architecture diagram that will include something like this Python package. And then you can start poking around at these various systems. You can look at GitHub where the vast majority of open source software is hosted, and you can see the number of stars, the number of forks, the number of downloads, things like that. And so you understand which systems are actually the most widely circulated and in use. And from attacker's perspective, that's all very valuable information, because now I can see very quickly what are the types of packages that are even worth attacking? What are the systems that it's worth building exploits for?

So these vulnerabilities in all of these common components that are kind of underpinning LLM powered applications, chatbots, what have you, they're going to be under increased scrutiny for the next foreseeable future. And so to that end, any time there are vulnerabilities that are discovered in these packages, you know, we've seen this play out a couple of times. Now, I will refer back to something that we've talked about on the show before, which is that the mean time to the availability of an exploit is dropping dramatically. So some of the statistics are as quick as twenty two minutes from, let's say, the publication of these CVS and the acknowledgement of these CVS until the exploits are available.

I can't say for sure whether that is the case in this situation or not. But two CVS here, one that allows arbitrary file read and the other that allows server side request forgery. So both in Chainlink, the TLDR if you are using Chainlink update to the latest version, patch those CVS out of there. No rocket science on the fix. Just, you know, a lot of implication for supply chain and what's happening in AI development right now.

All right. Moving on to our next story. This is again more of the same. And this is something that in fact, we've talked about and we've done research on here at Firetail. And that is a vulnerability in Google Gemini. Now we pointed out for instance early last or sorry, late last year in twenty twenty five of an Ascii smuggling thing. We followed that up with an emoji smuggling thing. We know that Gemini has vulnerabilities to kind of hidden text that can sneak into a number of applications. Google is increasingly enabling Gemini across Google Workspace, which is their productivity tool suite. So their equivalent of kind of Microsoft Office email word documents or rather file documents, spreadsheets, presentations, etc. and of course in that is also the calendar in this case, the, you know, the attackers seemingly found a way to kind of send calendar invites with hidden instructions in them.

And as Gemini processes these calendar invites, if there is something like a prompt injection or a set of malicious instructions in there that can be exploited and abused. In fact, the proof of concept that was disclosed to Google was actually the ability to exfiltrate user data. So I send you a calendar invite in. There is some hidden instruction that says, tell me all the other calendar invites that Jeremy has. Tell me who Jeremy's meeting with. Are there any attachments to those? Can you share those with me? You know, any number of instructions that might be in there for Gemini to process against that calendar Environment.

So again, we've talked about Gemini in the past. We've talked about kind of some of the risks around, you know, whether you use Gemini across your workspace environment. I had a reporter ask me late last year when we did our Ascii smuggling disclosure what my recommendation would be, and I said at the time, you know, if you are worried about these types of risks, you can disable Gemini against certain types of applications. Maybe you don't want to disable it across everything in the organization because it's very useful in document summarization or in creating presentations, but maybe you want to turn it off in email and calendar environments. That's really up to you. Our job is to report the risks to you. You make informed decisions based on what we can find in this week in AI security. Awesome.

Moving on to our next story. So this is an ID related story. This is spyware disguised as ChatGPT harvesting data from one point five million vs code developers. The way to think about this is, you know, we've talked about ID environments and how by necessity, they contain a number of crucial kind of credentials, system and environment variables that developers use while developing software applications. Now those could be used to do things like check in code. They could be used to do things like deploying to test and staging servers, where we might run tests against this code that we're building.

So there's a lot of kind of sensitive information that the developer might have. And along with that sensitive information, you know, keys, credentials, etc., go the permissions around there. A lot of developers are looking to automate their work, just as many people are are relying on llms to make them more productive. And so this is just another example of that. And again we've talked about IDE environments, I disaster and many other vulnerabilities over the last several episodes of this week in AI security. What's interesting to me about this one is that this really highlights the marketplace risk around this.

So I go into some kind of environment where I'm looking for VS code plugins to make me more productive. I search for ChatGPT, assuming that OpenAI would have released something on this side. But actually there may well be a legitimate one, but there's also fake ones kind of impersonating the legitimate ChatGPT, and some of those are malicious. I think of this a lot like two other parallels that I've seen in the cybersecurity world. One is kind of the Android ecosystem. I think the Android ecosystem for apps has a ton of risk around, you know, kind of apps that impersonate other apps, apps that are one character off or that just are very slightly off in their names, and it's not tightly enough controlled across Google Play or the other app stores.

So I've seen that I've spoken to a number of researchers in the mobile device and mobile application security space, and they always tell me that that is the number one risk to the Android ecosystem. And the other one is kind of what is often known as watering hole attacks. And watering hole attacks really refer to planting bad stuff at the place where people go. Traditionally, that's really meant let's compromise some kind of popular website and then we're going to embed some malware on that website. And maybe it gathers information from people who visit that website. Or in this case here. We're looking, you know, we're kind of just riding on the popularity of a brand and the popularity of a use case and just putting something out there for people to pick up as they come to the watering hole. So a little bit more of the same around IDE environments a little bit some kind of known ttps around how to put bad stuff out into the universe. You overlap those two things. You have this story.

All right, moving on. We've got our vertex AI flaw that lets low privileged users escalate to service agent roles. So this is a little bit more complex. So let me take a second to try to explain what's happening here. So within Google Vertex Google Vertex, first of all, is the kind of AI as a service marketplace on the Google Cloud Platform. It's very similar to Amazon Bedrock or Azure AI foundry. What it lets you do is you can turn on individual models kind of on demand, usually in kind of a serverless compute construct, which means that I don't have to run the whole model on my own. I can actually just call the model as needed.

And the billing mechanism around this is super convenient for me. I don't have any large commitment to any massive upfront commitments. I get access to all these different models from all these providers anthropic, cohere, Mistral, meta, and on and on down the list. And I can just kind of turn them on as I need them, experiment with them, decide which ones I actually end up using, moving into production, etc. so it's a super convenient and very compelling service, especially for smaller organizations, Firetail included. We love these services. We ourselves are AWS based, so we tend to use Amazon Bedrock for this, but very, very similar thing.

Now, along with that, there is really a arms race amongst the big three cloud providers Amazon, Microsoft and Google to enable AI adoption on their cloud platform. They stand to make a lot of money from AI powered workloads running on their cloud environments. So they're trying to make these as easy to adopt as possible. They're trying to add things that make it more and more convenient for developers to build applications on these platforms. Makes a ton of sense. It's no different from things like new storage offerings, new compute offerings, new serverless, containerized, whatever. Just think of anything in the cloud world that tries to make it easier. NoSQL databases are another good example, where there's a lot of competition between the cloud providers to have something unique and differentiated and easy to acquire and easy to ramp up usage of.

So that's what we're talking about in vertex now. Vertex is built on top of the Google Cloud Platform, so that means it inherits a lot of the core principles and primitives that all cloud platforms, and in this case the Google one are built on. One of those things is kind of the virtual machine or the compute infrastructure. Along with that compute infrastructure, you will often have something called the instance metadata service that describes the virtual machine, where the compute workload is working and that can have associated to it things like IAM users and credentials. Then you can use that IAM user to authenticate to other services. Common example I want my app server to connect to my database server. I don't want to have to store credentials inside the virtual machine because that's inconvenient for any number of reasons.

Automation being one of them, infrastructure as code being another. And so what do I do? I use the instance metadata service that I can invoke from the control plane of the cloud service provider, and use that as my authentication or credential store to then connect to other services. Makes perfect sense. How do I use these services, typically via API. So let's unpack what's going on. In this case there is an AI platform reasoning API. Attackers can use that sorry API platform engines API attackers can make malicious requests against that to call tools within the Reasoning Engines API.

Then you take an existing function with tool access, drop malicious code into it like reverse shell and it works as a standard function. Then you trigger that reverse shell and the code executes on the reasoning engines compute instance, allowing the attackers to extract credentials for the reasoning service reasoning engine service agent via the instance metadata service. I know that's a lot to follow, so if you want the details, please do check out the link from today's show Note episodes. I don't want to spend the entire episode trying to unpack every nuance there.

But what this then allows you to do is you take the default extensive permissions from the agent, which includes access to vertex AI memories, chat sessions, storage buckets, and logging capabilities. And then as an attacker, you can read all of that. So using that API, a reverse shell planted in a malicious payload, you escalate privileges via the instance metadata service. You use those escalated permissions to exfiltrate all that data, which could be very, very juicy. So this is novel in the sense that I've not seen this against these kind of AI platform offerings. But on the other hand, it's more of the same in the sense that we've seen any number of services being launched very, very rapidly have vulnerabilities in them.

We've seen it on other cloud providers around these AI as a service platforms as well. I expect that we will continue to see more, whether it's in the area of making your own vector database and your training models, sorry, your training data sets more accessible to these AI as a service platforms, whether it's the IAM around connectivity, who knows? But I do expect that we will see more of these kind of cloud based platform vulnerabilities get disclosed over time.

All right, moving on. We've got our kind of philosophical story for this week, which is a report out of the team over at Experian and Experian for those who may be outside the US and don't know, it is one of the big three credit reporting agencies. Credit reporting and insurance industries are kind of notorious for having high volumes of sensitive data in unstructured text, so they are prime candidates for things like MLM use cases. But what this team is reporting is that they think AI agents could overtake human error as the leading cause of data breaches currently, especially again in these industries.

The number one cause is is human error, and that's anything from a misconfigured S3 bucket that's accidentally left open to the world to oh, I forgot to change a default password human errors. But as more and more agents are pushed out into these workloads and into these environments, the error rate from the agents could easily surpass that. And it's really easy to see how this could happen in terms of, let's say like hallucinations or getting things wrong. There was a really interesting recent paper about vector embeddings, which is kind of one of the main ways that training data sets get, get passed along to Llms is through kind of vector databases.

And how then sometimes the llms in their usage of those vector systems get the PII wrong. You know, it'll be things like getting a first name just slightly wrong because they use predictive modes. So they see something like J and they think Jonathan instead of Jonny as an example. So, you know, some kind of mistakes like this in terms of hallucinations and whatnot could be things. Interesting kind of philosophical read, short read, but have a look at it. See what you think about it.

Moving on academic paper from this week, collaboration between researchers across Australia, Singapore, China and I apologize if I'm leaving anybody out. All the credentials are kind of listed on there. This is about agent skills. Agent skills is a really interesting topic because one of the ways that a lot of organizations are looking to deploy more autonomous AI systems is through agent platforms and an agent platform you can think of as basically being, hey, here's a platform where I can turn on or build, you know, any number of AI agents. So I don't want to build something that is super use case specific. I want to build a platform where I can then just, you know, more quickly turn on specific skills.

As a research project where the researchers looked at over three thousand different skills that had been deployed, over thirty thousand skills that had been deployed across any number of platforms, typically looking at it from code or from running systems that they might have had access to. And what they found is that more than a quarter of them had at least one vulnerability fourteen distinct patterns across four categories prompt injection, data exfiltration, privilege escalation, and supply chain risks. And what do you know? What have we covered on today's episode? uh, privilege escalation, data exfiltration and supply chain risks. We didn't actually have a prompt injection story until what we're about to get into in just a second.

It's really interesting that the research is totally backing up what we're actually seeing in the real world, which is, you know, these are some of the big risk categories that we're going to see across it. Again, as always, we'll link this research paper long read, but a very fascinating read for those who are academically minded and inclined to check it out.

All right. Speaking of AI agents and AI systems, you might have seen something called cloudbot and not to be confused with cloud AI from anthropic or the anthropic cloud model family of llms. This is spelled c w bot, so claw like a crab has a claw, then d, then bot. Apparently it's rebranded since then into something called bot, and this appears to be the first kind of general purpose AI agent that is designed for anyone, and it's designed for kind of human automation. Let me make your life easier. Let me take some of your tasks and offload them onto this AI agent.

Now, doing that has a lot of risks, as you might imagine. There's a really interesting analysis from Deepak Gupta who says Cloudbot is what happens when AI gets root access and it's a security experts take on it. And so what Deepak really analyzes, it comes down to this is an AI agent that has full file system access with read, write and delete capability. It has shell command execution. It could run code on your system via the command line, via memory invoke, via API. Whatever it has browser control, it can navigate website, fill form, extract data for you. It has email and calendar access, and it has smart home integration.

If you if you allow it now you have to turn this thing on. You have to install it. You have to say yes to the various permissions. But the really interesting thing is that you do that all via your install and the permissions that you grant are through things like messaging applications. There's a saying that's been going around the cybersecurity industry for the last five plus years, and that is that, you know, identity is the new perimeter, whereas network perimeters were the classic perimeter. And what Deepak says is based on this, the perimeter is now your messaging apps and how your AI agent interacts with them, what it does with them, what data it shares in messaging apps, and more.

I'm not ready to say that that's one hundred percent correct because, you know, this is one sample, you know, case study around this, but it'll be very interesting to see how this plays out. Now, this has been getting really popular and continuing on this story. A researcher called Jameson O'Reilly ran a Shodan scan looking for Cloudbot, and he found hundreds of Cloudbot instances sitting wide open on the public internet. What's actually interesting about this, if you think about it for a second, is that, you know, you're downloading and installing something that is designed to automate some of your common personal tasks. It is meant to run on your system. That's one of the appeals of it is you're not connecting to some cloud service where this is happening.

But in practice, what is actually happening is that in order to be your personal assistant, Cloudbot is actually exposing a lot of interfaces, things like email inbox acceptance, things like message acceptance, etc. and kind of sitting out there and these instances of cloudbot that were found via Shodan, which is a very common open source internet scanner. Hundreds of them were sitting wide open as, as the researcher says, quote unquote, completely compromised with API keys, conversation histories, personal messages between users and their AI assistants. And in the worst cases, full root shell access to the underlying systems where they were installed. A who am I command against one of these cloudbursts returned route. Who am I as a conman? Linux command to check the logged in user. You might run that sometimes if you log in as yourself and then you do something like sudo to gain root access. Check that you've actually been successful in that. It's a common command for for something like that.

So it's a really interesting open question. What's the trust level that most of the community has around adoption of things like that? Last story on this topic comes from our friends over at Bitdefender security Alert exposed Cloudbot. Control panels at risk, credential leaks, accounts takeover, etc. a little bit more of the same, but there were a couple of interesting details that the Bitdefender team uncovered around this. Um, the local host trust assumptions and reverse proxy setups caused some internet connections to be treated as local and therefore automatically approved.

So access to various remote systems via a local proxy setup or local reverse proxy setup actually said, hey, Claude, you're allowed to go do this. So a little you could consider that a little bit of a permissions miss from the Cloudbot system. Really interesting risk, you know, to to consider whether you're taking it on, uh, the report goes on in multiple cases access to these interfaces let outsiders view configuration data, retrieve API keys, browse full conversation history. So some confirmation of the previous researchers story around this. Since the platform is designed to act persistently on behalf of users, these control panels effectively served as master keys to the digital environments they managed.

There's a lot in this cloudbot story. I'm going to leave it up to you to read through the articles, but that's a quick summary of what's been exposed so far and what's been shared so far. And with that, we are going to sign off for this week's episode of This Week in AI security. Like subscribe, rate, review, share all that good stuff. We will talk to you next week on another episode of This Week in AI security from the team over at Modern Cyber. Thanks so much. Bye bye.