This Week in AI Security - 5th March 2026

All right, welcome back to another episode of This Week in AI security, coming to you for the week of the fifth of March, twenty twenty six from the sidelines of the just concluded, unprompted security conference put on by our friends over at Gnostic and a number of other sponsors, I want to save the majority of today's episode for some of my observations on the last couple of days and what I saw at the conference. So we've got just a couple of stories to get through before we get into that. So without any further ado, let's get going.

We're going to start with a story out of Bloomberg about a hacker using Anthropic's Claude to kind of help in the theft of Mexican data, and what it appears to be is a hacker using a multi-layer approach, weaponizing cloud code for vulnerability identification while then pivoting to OpenAI's GPT four point one for lateral movement, and then a number of other techniques like SMB enumeration across network to find different network stores data access points on the network using some persona injection and persuasion techniques in a role play exercise to try to get cooperation and frame the entire exercise as a fictional bug bounty program and get cooperation out of the model.

And that's something that we've discussed here on the program before, is kind of always theoretically true, and it appears to be even more true for languages other than English, where the source data for training purposes may not be as rich. Pretty big exposure, about twenty distinct vulnerabilities that were leveraged across these agencies. Theft of about one hundred and fifty gigabytes of data looks like it's about one hundred and eighty million taxpayer records or more voter registration, civil registry files. So a pretty big takeaway of of Data Hall on this. I don't know that I would go so far as to call this the first AI orchestrated data catastrophe, but on a nation state level, this may be the first instance where it does look like an entire federal government may have had very critical data troves rated in something that is kind of AI assisted and orchestrated.

All right, moving on. We've got a number of CVS to go through very quickly across a couple of different products. First, something MC confluence or MCP confluence depending on how you pronounce it. Unauthenticated server side request forgery on leading to remote code execution in the most widely used Atlassian MCP. And it's it's a pretty big infrastructure bug for the MC protocol era and kind of the overall construct. And it depends on a couple of things that are kind of common MCP structures. So things like MCP innately trusting zero dot zero dot zero dot zero. So kind of like very open network access, not necessarily requiring authentication for a lot of the requests coming in. It's a lot of these kind of security basics that need to be kind of encapsulated. And there's a joke that I heard and I heard it at unprompted, but it's been circulated that the S and MCP stands for security. And, you know, the obvious punchline being that there is no S and MCP. So something to keep an eye on as far as MCP adoption goes. And as far as, let's say, the increasing popularity of MCP, whether we continue to see some of these issues.

Moving on to our next one, this is ServiceNow, the AI platform allowing remote code execution. Uh, no authentication required under certain conditions. It was sandbox confined. It has now been patched. But, you know, confirmed out there in the wild on something that did make it into production. So still, you know, some of these core understandings about basic principles around authentication, authorization, some of that popping up again and again as these platforms move very, very quickly.

Moving on to, let's say, arguably the biggest story of the week outside of unprompted, and that is around the Google Gemini AI panel and things related to the AI keys API keys rather for Google Gemini. So there is a flaw in the AI panel that allowed malicious browser extensions with basic permissions to escalate privilege, including doing things like accessing the camera microphone and taking screenshots without user consent. Now, on top of that, we have a really interesting story around Google API keys versus Gemini API keys. And I would say kind of a conflation of the two. That has led to some really interesting observations and discoveries from security researchers.

If you look at Google API keys, they've been around for a long time, and you can think of them as being used for anything from embedding Google ads within a web page to Google Maps API, where where, for instance, we had Firetail. We embed map Google Maps for our office locations into our contact page and things like that. And it's been accepted that these API keys are for lookup purposes only. And so it's been safe historically to embed them in things like this. But so they've been deemed to be low risk historically. But what's been interesting is that because of that, they don't necessarily flag in a lot of secret scanning tools. So when things like GitHub repo scanners that look for secrets and look for hard coded credentials, these keys that start with this ALS or ALS, a prefix, have often been ignored because they've been viewed to be safe.

Now, these keys that are being used for Gemini often have more than just lookup permissions, and so they're not necessarily safe. So it's a kind of a fundamental shift in the way to think about these from being, you know, these kind of safe, not really secrets that you have to think about in the same way that you think about other things like API tokens and so on, that you don't want to hard code or expose in your code repositories. These are the exception to that rule. But that may have changed with Gemini here. And so, you know, these keys have as one person that I heard put it, they've grown teeth. It uses the same key format as the legacy of identifiers. They're getting hard coded. They are prone for exfiltration and vulnerability. And in fact, if we move on to our next story, what we do find is that a stolen Gemini key racked up eighty two thousand in Gemini costs in forty eight hours for a one person development shop.

Now, there's no direct proof that this came directly from an exposure of a key in a GitHub repo that was that was ignored by a secret scanner. But you can see the logical connection between the two stories. This is something we're going to have to really keep an eye on and look at this for other platforms as well. If the pattern is true for Google, there are chances that it might pop up on other platforms as well.

All right. Uh, last thing I did want to mention is a rapid AI driven development make security unattainable. This is a warning out of Varicode. Nothing super interesting or super novel to report here, but it does tie into one of the most interesting observations from the unprompted conference. So thematically, there were about four or five things that I observed over the course of the last couple of days, and I'll get through them one by one.

But one of the big ones was there was a lot of researchers who shared information about how they've used Llms to identify whether it's zero days, whether it is malware in different operating system platforms, whether it's source code vulnerability or what have you. The zero day clock has been published by a number of people who collaborated to put it out there. It was introduced by the CISO or CTO of Sysdig, Sergey Epps, who announced it. And what it really highlighted is that the mean time to availability of an exploit from the publication or the announcement, or the discovery of a vulnerability has come down by orders of magnitude. You know, from the point that it used to be kind of multiple months to thirty days to several weeks to now hours, and there are projections that that is just going to actually accelerate and accelerate any number of kind of prompt persuasion techniques can be used in getting an LLM to VBA code and exploit for a vulnerability for you, and if that fails, you can also use jailbroken Llms that you might run locally, where you remove ethical guardrails and constraints and you can, you know, get that same LLM. Now to write you stuff and it won't have any objections to it.

So that is just this kind of call to arms for the cybersecurity industry to think about, you know, what is the implication of all of that happening? So just as Varicode said, you know, this is really a situation that we have to be very, very aware of. And one of the, I would say, most interesting sideline conversations that I had was with a lot of other longtime security practitioners like myself, who look at the vulnerability problem as a problem that the industry as a whole have faced for twenty plus years at this point. And it's been the statistic for a long time that the mean time to patch is still on the order of months. For a long, long time it was one hundred and eighty days, six months and that kind of time frame. And I think recently in the last couple of years, that's gotten generally better when it comes to high severity CVEs and especially those that are in production environments and publicly accessible.

But what happens when the mean time to availability of an exploit is down to the order of minutes? How do we respond to that? You know, is automatic patching required at this point? As I've argued, for instance, on LinkedIn recently, and I had a pretty interesting conversation with folks in the comments about how to think about automatic patching. I got a number of perspectives over the course of the week. Is probably something that we're going to take some time to reflect on and put out a blog post around, but kind of the use of LMS for discovering vulnerabilities was one of the key topics.

So zero day discovery, probably four or five talks on this topic, everything from just, you know, zero days in code, zero days in massive software projects that humans have looked at for decades. You know, one of the researchers even pointed out a vulnerability that was discovered by an LLM, and it existed so long that it predated git and GitHub. So it was really interesting that, you know, these things that have actually flown under the radar for a long time are probably going to come to the surface sooner. So we saw everything from hardware vulnerabilities to Mac operating system malware detected all in the under the general banner of using Llms to find vulnerabilities and problems.

One of the other major thematic areas that I thought was interesting to listen in on was kind of defensive automation and agentic infrastructure. So a number of companies Google, OpenAI, meta, and others included, got up and talked about their own security strategies, their own kind of tooling, and what they're putting out there, whether internally or open source or via publicly commercial products in terms of like helping organizations secure and leverage AI and AI agents to automate, improve, facilitate certain tasks that are long running things.

Another area that I found to be really, really interesting was new attack surfaces. And this is something we've talked about in any number of stories on the last several weeks of this week in AI security, where you can think about things like indirect prompt injection, where malicious instructions might be embedded in environment variables and Readme files and what have you, whether that's across Ides or so on. But a couple of other areas that were brought up was things like, um, KYC pipelines that are increasingly being automated using AI tools or image recognition that's being automated with OCR embedded in Llms and things like that. AI note takers, you know, these meeting assistants that are pretty much ubiquitous right now and also have some shadow deployment and also maybe have some legal implications in terms of, uh, first one party versus two party consent for recording meetings and what might, you know, what data might they have access to as well as where might they be vulnerable to injection in various meeting environments?

A couple of other things. Some good real world case studies from Trail of bits, from Wiz, from Sysdig in terms of proof of concepts, in terms of automation of different tasks from the Wiz team post-mortem on using a multi-agent triage to kind of uncover massive breaches and data leaks from the Shai Hulud incident. Um, so a lot of really good stuff in real world case studies around that one that I really want to call out from my own experience. And one of the ones that I really enjoyed was from Nicholas over at Google on the Gmail team, who talked about how they're thinking about, you know, the embedding of Gemini into so many Google Workspace products and then the defense mechanisms that they're having to put in place.

Thinking about the prompt as code, um, was one of the I think, to me, one of the just the most interesting kind of conceptual things to think about is that, you know, your prompt really is your set of instructions. Now that's going to tell something to go execute based on on the context, based on the data available, based on what is in that prompt. And so something you have to think about when you might have looked at code hygiene and good coding and secure coding practices in the past. Well, apply some of that same kind of logic to your prompt, because that is where your instruction set is going to live going forward.

So some great stuff there and, you know, great community turnout. Kudos to the team over at agnostic who put the event on. Highly recommend it. I'll have some more thoughts. That'll probably come out as a blog post over the course of the next couple of days. And also, kudos to them for, uh, live streaming and capturing a lot of this stuff. So a lot of the content will be available post-event for people who didn't have the opportunity to get out here to San Francisco and attended live in person.

All right. That's it for this week's episode. We'll come to you next week with another episode of This Week in AI security. Any questions? Any suggestions for stories? As always, please don't hesitate to reach out in the meantime. Like subscribe, rate review. Remember, sharing is caring and we'll talk to you next week. Bye bye.

‍