In this episode, Jeremy explores how the automated "Vulnpocalypse" is officially manifesting in enterprise networks. As Microsoft logs a historic record-shattering Patch Tuesday to keep pace with AI-accelerated discovery, a new breed of open-weight logic malware is emerging to change the threat landscape forever.
.png)
In this episode, Jeremy explores how the automated "Vulnpocalypse" is officially manifesting in enterprise networks. As Microsoft logs a historic record-shattering Patch Tuesday to keep pace with AI-accelerated discovery, a new breed of open-weight logic malware is emerging to change the threat landscape forever.
Key Episode Highlights:
Episode Links
All right. Welcome back to another episode of This Week in AI security, coming to you for the week of the eleventh of June twenty twenty six, and actually may be coming to you slightly late in your feed as we speak today. We had an issue with the audio on the original recording. So you're getting a rerecording and let's get diving into it. All right.
So I want to start today's episode with a topic that we've covered a lot over the last several weeks. And that is kind of the, you know, Vulnpocalypse, as I have seen it dubbed in a new kind of portmanteau around vulnerability and apocalypse, obviously, but this is, you know, really stemming from some of the capabilities that are either proven or believed to exist in newer models like Anthropic Claude Mythos and the new ChatGPT Cyber Edition. And specifically, we're talking about, you know, the ability to discover vulnerabilities at scale. Right?
And so the interesting data point here, that kind of is evidence of a trend line that I think we're going to see is that the most recent Microsoft Patch Tuesday hit a record two hundred and six CVEs here, right? So let's break that down here to understand for a second what this is. So first of all, Microsoft Patch Tuesday, for those who may not have worked in the Windows ecosystem or may not have worked like I did as a system administrator in the Windows ecosystem, managing a corporate environment where you're pushing out patches to a number of desktop users. This is a thing that comes once a month, comes on a Tuesday, hence the name Patch Tuesday. Typically, it will include anywhere from sixty to one hundred, one hundred and twenty-ish fixes. And that's fixes total. So that's not just the cybersecurity fixes. That also includes things like bug fixes for functionality in different pieces of products, whether within the Windows ecosystem or common Microsoft applications like, let's say, office applications, etc., would very often be rolled into a Patch Tuesday.
So to now see that we are both at kind of double the scale of a typical number of releases in a Patch Tuesday, and that we're specifically talking about the number of cybersecurity capabilities is, I think, a really interesting indicator of the direction that patching has to go at scale and at the speed that it needs to progress in corporate networks, where we know that vulnerabilities are going to be discovered much more quickly going forward. Right? So one thing I will also say from my perspective is kudos to Microsoft for jumping on this and realizing the kind of the urgency and the seriousness of the vulnerabilities, obviously, given the massive footprint of their operating system. I imagine that we're going to see the same from Apple and from actually Linux distributions that have a lot of corporate footprint around them. So we'll have to see a step up there.
Now, what will be interesting on the Linux side is that Linux, for the most part, is still kind of open source. So I actually view this as an opportunity for, let's say like Amazon with their Amazon Linux distribution or Microsoft with the Azure Linux distribution, etc. to differentiate corporate Linux distributions, or Linux distributions that have corporate backing from mainstream open source distributions that you might see, like something like a Debian Linux or something that is still much more community support maintained, etc..
All right, moving on to our next story, just very, very briefly, the LiteLLM, CVE flaw that we talked about a couple of weeks ago. The only thing I want to point out here is that it has now transitioned from being just a CVE to now being present in the CISA's KEV catalog. And if you're not familiar, that is the Known Exploited Vulnerabilities, KEV. And obviously, you know, this just means that it is proven to have been used for exploits in the wild. A little bit further kind of verification of this, you know, hypothesized path that, okay, great, we can detect vulnerabilities by scanning source code. We can build exploits super, super fast. How quickly can we actually get those out into the wild? And if you think about kind of the time frame from when this CVE was detected, I believe that was two episodes back. Might have been three. So two to three week time frame on this. That is not crazy, crazy fast in the scheme of what we're, you know, what is expected around kind of quote unquote again apocalypse. But it is just evidence that that is the direction that things are moving. All right.
Moving on to our next story. And this is AI in cybersecurity or in fraud. So this is a report from Google. Their June twenty Beam twenty six fraud alert exposes the growing role of AI and sophisticated online scams. Now, this is primarily around deepfakes, voice cloning, and synthetic identities driving massive losses. One stat that really jumped out to me is a one thousand two hundred percent growth in this type of fraud. And just to kind of peel back, because I know that number can be a little bit shocking, that first of all, that starts off of a low base. So obviously when you're talking about a low number of numbers, when you go from two to four, you know, that is a doubling or one hundred percent increase, right? Even though net net, it's not that big an effect. So we're starting from a low base.
Second, this is specifically around just these deepfakes, voice cloning and identity impersonation using AI. So we're not talking about, you know, a one thousand two hundred percent growth in overall online fraud. We're specifically talking about this category. Now another interesting thing is that Google is rolling out Android protection against AI deepfakes in the Android app, or sorry, in the Android operating system. Rather, it's going to roll out first to Pixel devices naturally being, you know, coming straight from Google. But I do imagine this will roll out into other partner Android distributions like Samsung, LG, etc. some of the other major Android, uh, distributors of, of, uh, those phones and mobile devices.
All right, moving on to our next story. Speaking of kind of fraud and AI abuse, this is actually a story from last week that we missed in the end. I first, I apologize for missing it. I always try to keep us up to date with the most relevant stories in this week in AI security. I hope that's what you tune in for. But I will say a couple of things. We've been able to learn a little bit by sitting out for a week and then kind of following up here. So last week, the story was basically just, hey, threat actors are using Meta's AI chat support to hijack Instagram accounts.
Well, what we now know is that it is actually up to the level of twenty thousand two hundred and twenty five accounts confirmed, exploited by that. And what's interesting from my perspective is, you know, one of the headlines here, including one of the ones that we have referenced from the show or linked from the show notes today is it calls it exploiting a flaw. And I would argue that it's not necessarily a flaw in the AI assisted account recovery system. It's actually kind of known attack vectors around this, right? So it's everything from kind of, let's say latent prompt injection to direct prompt injection to context, window exhaustion that leads to prompt injection or that leads to kind of ethical guardrails bypass.
And so this is not necessarily what I would call, quote unquote, a flaw. This is actually a natural expected outcome. One could argue, of course, that there are safeguards that you should put in place around things like, let's say, you know, context, window exhaustion. What that really means is I just chat with something long enough that all of its stored guardrails kind of fade out of the context window, are no longer taken into account. Well, great. Well, so when you're building an AI support engine like this, what that means is you need to refresh the context window pretty regularly and make sure that those guardrails get reinterpreted, re-ingested and, you know, and kind of re-implemented on a regular basis. So TechCrunch has confirmation that Meta is alerting users whose Instagram accounts were targeted. A lot to do here. But it is one of those things where, you know, if you're thinking about deploying customer or public facing, uh, AI or LLM powered chatbots, you do need to think about some of these design aspects. And this is something where, you know, a red teaming exercise with a creative, you know, white hat, uh, who is going to think about how would I attack an AI system, uh, is a really beneficial exercise that I would recommend to anybody who's going down that path.
Routing on to our next story, we've got researchers at the University of Toronto, the Vector Institute, and the University of Cambridge who have built and tested a proof of concept, AI driven worm that doesn't operate on a fixed list of exploits. So let's unpack this for a second here. Most malware and worms has one or more sets of, let's say, targets or goals that they're trying to do, right? Get onto your system, steal passwords. Great. You know, that's, that's very normal, right?
But this is something that runs on open-weight models that are hosted on the compromised malware hardware. So what that means, the malware drops on your system. It brings a very lightweight model with you. It does not require the ability to discover new zero days. It actually then uses some reasoning in the environment that it's operating in to try to find ways to do distribute itself across that. Now there are a couple of stories that we're going to link on this. This is probably one of the most interesting stories from this week. It looks for vulnerabilities on the host systems that it is on, and it does that more dynamically through a little bit more of a, let's say, like Q and A type of approach, as opposed to just like, can I execute the malware task that I have been given? So it might, you know, use a little bit of reasoning and logic. Well, I can't get vulnerabilities on the local system. What should I do? Let me go distribute myself across the network, etc..
And this is something that some people are, are, you know, think could be, quote unquote, the stuff of cybersecurity nightmares. I tend to think that it is a great proof of concept from the standpoint that it is actually going to force the EDR vendors to confront this new kind of threat vector that might show up on endpoint device systems. And so I think that there will be, you know, maybe some short term pain, but maybe some medium to long term benefit that we learn around analyzing this amount, this type of malware and kind of heading it off quickly before it spreads far.
All right, last couple of stories. You know, we like to end on kind of some of the macro level, uh, industry, uh, stories as well as geopolitical and philosophical. We'll start with Anthropic rolling out a public version of Claude Mythos without cybersecurity capabilities. I can already tell you, you know, as somebody who's works in this space, my mind immediately jumps to, well, how do you know that you've removed the cybersecurity capabilities? Right? It's one of those things where you put a model out and you don't think it has a set of capabilities, but you realize that things like your ethical guardrails and guidelines may not always apply. Again, see the previous example around the Meta chatbots, context, window exhaustion, any number of things. So I'll be curious to see whether this proves to be true or whether we see, you know, next week that some threat actors have found ways to exploit this family, the Mythos family that is available to find different problems with it.
And then last but not least is, you know, there is this kind of global arms race around AI and AI capabilities. There have been anything to the level of tariffs, trade embargoes, export restrictions around GPUs, right, to try to keep the US in a leadership position. Report from Politico says that the US has at most, quote, "six to twelve months," end quote, before Beijing can compete with the new wave of hyper advanced AI models. So lest you think that, you know, there are only there's only one or two centers in the world where frontier models are being developed. No, rest assured that there are models being developed at many, many places around the world, including places that we don't talk about regularly on this show, whether those be less connected environments like Iran, or whether those just be places that are choosing to fly under the radar a little bit more from their academic research. So an interesting story. Take the time to read it. If that's something that is up your alley and that will do it for this week's episode of This Week in AI security. Apologies again for getting it out late to you. Like subscribe, all that good stuff. Talk to you next week. Bye bye.