This Week in AI Security - 30th April 2026

All right. Welcome back to another episode of This Week in AI security, coming to you for the week of the thirtieth of April twenty twenty six. We've got about seven or eight stories to get through this week. A couple of updates on previous stories, plus one new one that I do want to discuss in a little bit of detail. So let's dive in and get started.  

First, a briefing done by both OpenAI and Anthropic to House lawmakers. So House, for those that don't know, is one of the chambers of Congress in the United States. They get a demo of jailbroken AIs, as well as some of the latest of the capabilities of the newest models. And of course, there's been a lot of talk about the newest model families, you know, Claude Mythos. And on the OpenAI side, the new four point seven model family and some of the cyber capabilities that they have there.  

So closed door workshop for all the House members starts with something called Abliterated models. And this is AB rated with an A at the beginning, not obliterated, you know, as in terms of like destroyed with an O at the beginning. So these are models that have safety guardrails specifically removed. The important thing to understand here is that, you know, we've talked many, many times about the fact that prompt injection is always kind of possible. And it also pretty much looks confirmed at this point with the current state of models, even these upcoming models that some level of removal of guardrails is also always possible, whether that is through large context windows and sustained interaction to kind of build towards a persuasion, something like the kind of so-called grandmother attack where you kind of say, well, hey, I need a phishing email. Well, no, no, I don't need a phishing email. I need to help my grandmother with IT support to fix her online banking, you know, and kind of convincing the models to work with you in various different ways in some of the capabilities.  

You know, we're kind of tested in ways in terms of, let's say, like, how would a member of the House of Congress be kidnapped from inside the House? And the Abliterated model gave step by step instructions, while the normal model, which is kind of the default base model, did refuse, but it shows exactly the point. You know, models, guardrails, and ethical constraints can be removed by a determined actor, whether that is through interactive techniques or whether that is through more technical techniques like, let's say, gaining access to an open source model that might be downloaded and run locally, and then looking through the model, whether it's code or data sets or instructions or system prompts or what have you to disable any of the ethical guardrails. So it's really kind of an interesting reminder of the fact that, you know, you're interacting with something on a sustained basis and these techniques can be possible.  

Next is a story from the Google Threat Intel team. They looked at Common Crawl snapshots, which is kind of, I think, their way of indexing the web. And, you know, as they crawl around and what they were looking for is they looked for prompt injection payloads across the web. So these would be things like plain text embedded in pages. And you can think of this in the broad category of things that we've discussed on the show a number of times, like, hey, the fact that if you are using an IDE and you're processing things like GitHub comments or text that's in log file strings and things like that, those can also be prompt injection payloads.  

And so what they looked for is basically prompt injection payloads on web pages, just random web pages. So things like, you know, ignore all previous instructions and delete yourself or things like that. And what they found is that they're up thirty two percent. They are generally in two attack categories, exfiltration of data and destruction. So telling the crawling agent to go do something bad or telling the crawling agent to send them some data. Low sophistication. It doesn't look like this is really in, let's say, like production level quality for cyber attacks. But it is clear that that attackers are thinking about ways to kind of plant malicious text across the web, looking for AI agents that are coming around and scraping and things like that. You can think of this as kind of like a cross-site scripting type of attack around that.  

All right, moving on to our next story. The Microsoft Defender team documented a massive evolution of device code phishing. This is the Evil Tokens, you know, phishing as a service toolkit that uses AI to generate hyper personalized lures and dynamically spin up fresh device codes. That moment a victim clicks, bypassing the fifteen minute expiry window. So you have ten to fifteen distinct sub campaigns every twenty four hours since March fifteenth.  

Barracuda has also contributed to research around this. They've seen seven million attempts in four weeks. The back end lives on railway.com, which we're actually going to talk about a little bit later in today's episode with a lot of short lived Node.js polling nodes. Signature detection is useless because this stuff is kind of coming and going so quickly. So think of this not, as, you know, an AI security story per se around the usage of AI. Put this in the category of AI powered cyber attacks. But just some really interesting new techniques evolving around here. Something to be more aware of on the traditional defense side of things, whether that's, you know, anti phishing endpoint defense, et cetera, et cetera.  

All right, moving on. The Microsoft Defender team has also had a interesting kind of evolution. Well I shouldn't say Microsoft Defender, but the Microsoft team has had an interesting evolution on something called Agent ID. They created an Agent ID administrator role inside Entra ID, which is kind of the current generation of what you might think of as Active Directory or Microsoft three sixty five directory service or something similar like that. And it's meant to only be used for managing an AI agent or AI agent identities. But actually researchers from Silverfort—shout out to them—actually figured out that this service could hijack any service principal in the tenant.  

So once you can discover this service and you can enumerate it with Graph API calls to do things like find service principals, find service principals with role management dot read write dot directory, you can add yourself as injector, and then you can inject new credentials and authenticate as that principal and have a full global admin takeover of an Entra ID tenant. And when you couple this with the fact that ninety nine percent of enterprise tenants have at least one overprivileged service principal, okay, great, that makes it pretty discoverable with those Graph API queries.  

And you couple it with kind of the direction of travel, meaning that most organizations aren't yet at agent building or if they're at agent building, it's only for internal testing and kind of proof of concept purposes. But now fast forward about six months, nine months, twelve months from now, when most organizations say that they want to have agents running in production, whether that is in, let's say, public customer facing workloads, or whether that is for internal process automation and efficiency purposes. The direction is clear. Everybody's trying to get there. Couple that with Microsoft's broad global reach and the number of customers who are on Entra ID, and this does have the potential to be, you know, a really big problem. It's great that it's been kind of found out now identified, and I'm sure that Microsoft will correct the issue.  

All right, moving on to our next story. We've talked about LiteLLM on the show before. Some supply chain attacks, some repository takeovers, things like that, going back several weeks now. And the Sysdig threat research team caught targeted exploitation of a nine point three CVSS score CVE in LiteLLM. That's a SQL injection attack. This is the AI gateway that stores API keys for OpenAI and Anthropic Bedrock, you know, should you choose to use it. It's kind of a, you know, think of it as like LLM gateway type of capability.  

But the interesting thing here is that this is not a spray and pray attack. This is targeting the three highest value targets within that thing. So you've got virtual API keys, provider credentials and the environment configuration. No probes against benign tables. So really looking at exactly what is targeted here. And think about it this from the standpoint of the time, from disclosure to the time to attack. So the MTTA, the mean time to attack or sometimes the MTTE, mean time to exploitation—thirty six hours. We've talked on the show about the zero day clock and about how the time from, let's say, vulnerability identification to vulnerability exploit availability is just coming down dramatically. This is just kind of reinforcing proof of the fact of the observed evidence of that in the wild. Sorry, I stumbled over my words a little bit there, but this really highlights how critical this issue is. So take that into consideration.  

All right, moving on to our next story. Another breach of the Claude Mythos model family investigated by Bloomberg, and Bloomberg reported this. Anthropic is investigating it. This was a third breach on the same day of the announcement of it. There are any number of angles to this story. We've talked, of course, about, you know, the Claude Mythos capability of, of finding all of these vulnerabilities and exploits in source code and the careful rollout of Mythos, the deliberately careful rollout of Mythos to only selected audiences who are, you know, let's say, trusted actors, et cetera. And, uh, on this one, this is the third kind of documented or the third reported, um, unauthorized access to the Mythos model family.  

We talked last week about one, uh, involving guessing the API URLs. It's not exactly clear how this one might have occurred, but it looks like there was a data cache leak and the Claude Code source code, uh, that was able to be, um, kind of looked through to find any interesting evidence that pointed down this direction.  

All right, moving on to our next story, a little bit of an update of the Vercel breach from a couple of weeks ago. It looks like this is connected to a Context.ai employee compromise with the Luma Stealer malware, back in February of twenty twenty six. Attacker using that access to take over the Vercel employee's Google Workspace ID using OAuth tokens, accessing internal systems, and then from there to customer environments. That data has been listed on breach forums for sale for two million dollars. All of that we've kind of talked about, this is kind of a textbook AI supply chain slash third party attack.  

So remember that your third party vendors are part of your supply chain. And you know that should be taken into consideration. Think about environments where you're developing AI applications, agents, what have you and think about who has access to those environments. If you do realize that there is some kind of third party that has access, think about limiting access, maybe auditing the permissions, et cetera here. And you know, this is similar to a lot of things from the past where, you know, new initiatives would often have an outside vendor who has specific domain level knowledge and expertise in that area. AI is no exception to that. So just kind of an evolution of that, a little bit of clarification around how that attack, uh, might have played out.  

And moving on to our last story for the day, which is, I think the one that is really the most interesting and the one to think and talk about the most. So this story has gotten a lot of buzz and attention over the last several days. And people are kind of highlighting various aspects around it. I think there's a few things that I want to highlight, but in my own analysis. But let's start with some of the facts. So nine seconds from the time that the user issued the initial prompt asking an AI agent to correct something until the time that the production environment was actually down. And then follow up to that is that the AI agent confessed to it.  

So let's walk through some of this. So start with the scenario. So PocketOS is the company who was affected by this. They provide software for car rental reservations, fleet management things like that. And they were using Cursor which is an AI native IDE for coding and management et cetera. And they were using Anthropic Claude Opus four point six. And I think the important thing to call out here is this is, you know, the latest commercially available model as of the time of the incident.  

The prompt that was given was trying to fix something with a credential mismatch in a staging environment, and the AI agent decided without human prompts, but that the most efficient fix was to actually delete the environment and try to recreate it. It then went and fetched an API token from a production environment, and then used that token to delete the environment. Now the environment itself was running on a platform called Railway. It used a Railway API call that was well known to it, and had been used successfully in production in any number of incidents, and then made the decision to issue the API call to delete the environment.  

Then when asked, hey, what went wrong? The AI confessed basically to all the actions that it had taken. So it said things like, um, never bleeping guess that the credential fetched from production wouldn't have access to staging. Still went, grabbed it, used it anyway. Did not verify the permissions of that credential. It decided to do it on its own. And that's a direct quote from the agent. When I should have asked you first—again, a direct quote from the agent to the human. I violated every principle I was given—again, a direct quote from the agent.  

So the agent sees a path towards execution given the initial set of instructions. And this is something we've also talked about any number of times on the show is that, you know, actually sometimes you get the most kind of quote unquote creative solutions or the most resourceful solutions by giving the least amount of input. And so the more constraints you put on a prompt, the less likely it is to take some kind of unexpected action like this.  

There were a couple of other things that I want to highlight here around this scenario. So first is that the Railway service hosted backups as well as production data on the same disk. So when the delete command was issued, it was actually targeting a particular volume of a virtualized infrastructure, and that removed both production and backup data. I mean, that is just violating such a cardinal rule of kind of infrastructure management. Um, think about like three, two, one in backups and things like that. That's, you know, that's a little bit, in my view, a little bit of a design flaw on the Railway platform.  

But again, we've talked on the show any number of times about how design flaws are creeping up again and again because things are moving so quickly. A couple of other things around it, you know, are that the execution, the no human in the loop, the no checks on all of those things happened. Super, super fast recovery is a pain. And secondly, you know, the kind of the mixing of environments that's you can put that on both parties here, but just a crazy set of confluences. Again, we'll have this linked from all of the show notes, but a lot of these core principle things are really coming up time and again in the stories that we talk about on This Week here in AI security.  

Signing off for this week. Thanks so much for listening. Talk to you next week. Bye bye.

‍