Kenneth Ellington of Ellington Cybersecurity Academy

All right. Welcome back to another episode of Modern Cyber. We've got a conversation today that I know is super relevant as we record here in early twenty twenty six. This is the year that what I hear from a lot of my security leader, compatriots and colleagues is this is the year when Agentic AI goes from being a concept to being something in production. Today's guest is going to share a lot of his experience and expertise in that domain. I am delighted to be joined today by Kenneth Ellington.  

Kenneth is a senior SIEM engineer and cybersecurity entrepreneur and the founder of the Ellington Cyber Academy, where he trains the next generation of engineers and threat hunters. And we're going to get a lot into his background, his experience. What is detection engineering these days? What is threat hunting? He previously served as a senior consultant at EY supporting enterprise security ops and SIEM engineering initiatives across complex environments. He specializes in detection, engineering, threat hunting, XDR architecture, deep hands on experience across Splunk, Elastic, Sentinel. I think those probably are the big three of SIEM platforms these days, right? So we've got a lot of expertise that we're going to be able to get into. Kenneth has spoken at BSides St. Pete sharing insights. He also holds a degree in Management Information Systems, MIS and cybersecurity. He's certified across Splunk. I'm sure I could go on. And one of the interesting things for me is outside of cyber, he trains Muay Thai and kickboxing. Time permitting, we'll ask a little bit about how you blow off steam from cybersecurity because it can be a very stressful job. Kenneth, thank you so much for taking the time to join us on Modern Cyber today.  

Of course. Happy to be here.  

Awesome, awesome. First of all, is that the big three of SIEM platforms these days? Kind of Sentinel, Splunk and Elastic?  

Are at this point, yes. Um, I would probably say I guess up comers would, might be like, um, CrowdStrike, next gen SIEM, and then I think XSIAM from Palo Alto. But Elastic, Sentinel are the biggest ones right now.  

Makes a ton of sense. Makes a ton of sense. Do you see big differences between the platforms in terms of like, not from a technical perspective? I know they've got technical differences, but in the overall philosophy of, let's say like when I think SIEM, I think aggregate your logs into a log stream, do some level of data normalization, present the raw data, and then apply detections on top of that data stream. That's pretty consistent across those three, right?  

Yes, they all are all different because their parent firms are different, right. You have Splunk which is now owned by Cisco. You have Microsoft Sentinel, which is Microsoft, and you have Elastic, um, for Splunk because Splunk was designed for on prem environment, they're trying to transition towards more of a cloud based model. So there's still some tweaks they're trying to do with that. Sentinel was initially designed to be on prem. So it works extremely well in that, uh, format or like or like for us, like infrastructure as a service for like AWS or Azure, Microsoft. Um, Microsoft is not a hardware company whatsoever. They are a software company. So everything they do is based off of that, which present challenges in that regard, then Elastic is open source. It's not free open source. Big difference. Um, so there's a lot of, uh, good community support in regard towards their platforms.  

Fantastic. Fantastic. Great. Just kind of background information from our audience. Probably a lot of our audience might know that, but I always think it's a good refresher to kind of think through some of that. So we have our SIEM, we have our log files flowing in all this telemetry observability data coming in from all the places. But the thing that I hear pretty consistently is that, you know, detection systems still generate too much noise, still too many false positives, still too much investigation to be done. This has been the case, I think, as long as SIEMs have been a thing. So probably going back, you know, ten, fifteen years at this point. So I know automation is something that everybody wants. And in the age of AI, I'm really curious, your kind of first initial reaction. One is AI helping that situation? Before we get into actual, let's say, automated playbooks, but just at a high level, is AI helping or are we still kind of at the earliest days where we know it can help, but we're not quite at a place where it's effectively helping yet?  

I would say it can be helpful if it's implemented correctly, which like any tool or platform that's been the case since the dawn of time isn't implemented correctly. It'll just add on more tech debt towards your organization. People will get confused as to as to what is going on. So I think the biggest thing for most firms is having a plan as to how they want to use AI within their organizational structure and how will it be set up. It's the first question you should ask yourself.  

Okay, okay. And what would you say are like best practices as you think about kind of laying out that foundation or putting that architecture in place?  

I would say a big question is like, like, what is your budget? Right? Because money always is everything for most organizations. Um, after you understand what your budget is, understand what type of industry are you working in? Are there certain compliance policies that you have to try to go up against and HIPAA policies? Um, uh, data sovereignty. Are you based in the US? Are you based in the UK? Are you based in the EU or you're based in Asia? Right. If you're using a certain LLM model, who is the owner of it? Right. Is it you? Is this purely internal? Do you have the actual proper staff to actually manage and develop that, or is it external towards a different company? If it's external towards a different company, again, who is managing that? Are they transferring data from one country towards the other? What type of policies and costs will be associated with it? Um, are the questions I ask my clients a lot of times.  

Okay. And it's that question. I mean, you said in there like, you know, what is your budget? I always hear about from a project management perspective, it's always kind of speed, quality and, uh, crap speed versus quality and scope, I guess. And you can pick two and that determines the third one. Uh, so is that the same way to think about this?  

Um, yes and no. I again, like it just depends. Um, some firms depends, depends on industry and field. You're in. Right. Some firms are much more like big tech is much more um build things and fix it. I'm like after they break some firms like either finance or like, um, medical fields are much more cautious at the things they'll open in because they have much, much more of a global impact on the world. Or for example, for like medical field people might die if these systems don't work properly and end result, then for finance, I feel I work for the most of my career. Um, if certain things don't happen right, a lot of money can be lost. So there are a lot more cautious things they might want to build or take out.  

Yeah, yeah. So you mentioned finance and healthcare. And I think that's a great segue into something I want to ask about, which is, you know, these are two highly regulated industry, one with lives on the line, one with money on the line, right? So rightfully they're regulated. They're very sensitive about things being accurate. Correct. You know, data privacy, all the things that go into both of those industries. What does responsible AI usage look like in an environment like that?  

Um, I'll probably say, as I mentioned before, I'm making sure you're following the proper compliance policies. Um, so if you're in finance, you're going to have a PCI credit card compliance, right? Um, so making sure if you're inputting in certain people's information for your own models, I don't think you should in any sort of way. But if that has happened, just to make sure it's stored within your own model and there's no data, there's no data transfer between different regions or different, um, countries, which would, which would go against the policies which you can get audited for um, in that regard. So it's often, I think try to just make people understand, okay.  

But, you know, one of the frustrations that I hear, you know, and I work in AI security. One of the frustrations points that I hear is, yeah, but we don't really have great standards or regulatory frameworks around the usage of AI in those environments. You know, PCI as an overall industry standard. Got it. HIPAA same thing. But if you dig into the HIPAA, current HIPAA regulations, there's almost nothing around healthcare. Uh, sorry, AI usage in a healthcare context. And, you know, at the same time, we see these headlines about a, you know, AI helped to detect cancer in a particular patient. So I know there's like experimentation happening. How do you advise leaders to think about it when maybe the regulation or the regulatory framework doesn't cover this new technology just yet?  

That's part of the problem. A lot of this is really new. Um, over the past, I want to say what like four years since like twenty twenty two when I'm open AI, or at least like ChatGPT is kind of like the Wild West. So there aren't that many rules and regulations as to like how to manage, like how to design this. So people are just building and putting things in place, which makes it kind of hard for me to advise some of my clients because there aren't, I should say there aren't there aren't there aren't like a lot of rules. It's just kind of you build things. You hope you cross your fingers, you hope you don't get sued or something happens towards your clients.  

Yeah. And so then I guess you just fall back to kind of like, okay, well, let's implement the other parts of the regulatory framework, make sure we're not stepping out of line there. And then just, you know, good cyber best practices around making sure that you're doing everything you can to protect the data, protect the environment, etc.. Is that kind of the right way to think about it for now?  

Yeah, like security best practices overall. I mean, AI is still just a machine on a, like a Linux server inside a data center, right? So the same principles will still apply towards it for the most part. Um, that's, that's what I tell my clients all the time. I know some firms have just kind of come up with their own standards as well because they can't find any. They try to match against something.  

Yeah, yeah. So on that note, I want to go tap on some of your experience in working in SOCs at scale, in complex environments. First, I just want to validate the thing that I've heard consistently is that most SIEM environments, the detections are still too high. False positive rate, and most SOC analysts still spend too much of their time chasing down false positives. Is that still true in twenty twenty six?  

Yes, I can attest to this firsthand. Um, very much so. So a lot of it is for false positives, right? Um, let's say you get called at two a m in the morning for a certain malicious threat. Um, a false positive would mean it's just someone logging in that normally does maintenance on those servers at that time. Right. And you know, for a fact you see the alert on your phone and you're like, okay, this isn't actually serious. The reason that's bad is because analysts will get burnt out from that, right? And then when actual real threat happens, they won't take it seriously because you're getting inundated with so many different false positives. Um, the other part of it is like the human element, right? Um, there's so many alerts you have to tune. It's just a lot of work so people don't have the time to actually properly manage their environment, which adds on to more of the tech debt. And then it kind of falls out of control.  

Yeah, yeah, it makes a ton of sense. So with that in mind, where do you think AI can genuinely improve that situation? Is it on kind of reducing the number of alerts that go to the human for evaluation? Is it on enriching alerts with more context? So the human can kind of assess them more quickly? Is it on the data investigation? Is it all of the above? What are the things that that you think can really be impactful?  

I would say if implemented properly, like all of the above. I think it's really good at like, um, analyzing very common alerts that get, get put inside your client environment. So I think most firms technically have been using ML AI for a decent while now and a lot of like marketing buzzwords. Um, but any and any who, a lot of the times you can use it to trim down on those false positives for certain type of baseline, like metrics you want to use. So if certain users log in at a certain time and access certain applications like, like for example, using that for machine learning data, you're able to actually create a baseline as to what is normal and what is abnormal within a client environment, which will help you streamline your investigation process as well.  

Yeah, makes a ton of sense. You know, I hear a lot about kind of, let's say, using AI more in incident investigation and response where, for instance, maybe like log gathering or log collection and then log correlation are a big part of the process that's very time consuming. And, you know, even if we automate it with our current tools, meaning the non-ai tools, at best, we can kind of collect the data, but then we still have to go through and do the analysis ourselves. Is that an area where you think AI has a, has a chance to really like, do a better job faster than a human can do?  

I think in the right circumstances it can. Um, okay. It's going to be kind of like what, what the models are trained from what models they're using, right? Because you want to base it off of what specific towards your own environment, right? Because those are your own specific needs. Even though some cemeteries have very similar type of threat factors, right. And some type of threats that might want to attack them, you want to base it off of your own environment because if you just automated using AI to detecting certain things that based off of like what Walmart's doing right from its database that it's pulling from, and then that is really going to probably tell you if you're a smaller mom and pop shop, right? So like the complexity of your environment is going to be different. Your network configuration is going to be different, you know?  

Yeah. Gotcha. Gotcha. Makes a ton of sense. So when you think about kind of automated response today, you know, the term SOAR became a big buzzword, I would say probably about like ten years ago, kind of the twenty fifteen twenty sixteen time frame. Um, maybe like stretching into twenty seventeen eighteen. I saw that bubbling up in a lot of the enterprise customers that I was working with at the time. And, you know, and there were a couple of companies that grew very quickly, got acquired by the likes of Splunk and Palo Alto Networks and so on, and kind of were some of the biggest names in that space at the time. But then I kind of feel like that space stalled a little bit. I didn't feel like I felt a lot or saw a lot of innovation and automated response. Uh, until maybe now when I think, you know, kind of LLM powered SOAR has kind of become a thing again. How do you think about it? What do you think is currently possible? What do you think is like real SOAR today? Or are we just saying like, no, actually, like AI powered, let's say SOC work is actually just like SOAR playbooks with a little bit automation. What's real?  

I, so I have a pet peeve about this. So I heard first— All right, let's hear. I first heard about SOAR back in twenty nineteen when I was an intern at Publix supermarkets. I'm doing threat intelligence work for them. Um, my first time I heard about it and like, as I was starting to get really good at it, everyone was like, oh, SOAR is dead. You're not gonna use this anymore. They start talking about like AI and I'm like, I'm looking at it. I'm like, I'm like, this is basically just like some AI rapper on it. It's still the same thing. So like, this is just marketing buzzwords, right? Like, like a lot of these features we have already had for years, we're just throwing, slapping on the AI, um, label on it to sell like, um, whatever you want to sell towards a client. Right? So I do think there can be some uses for AI platform to help streamline the process. Because a lot of times if you get a platform, it's not built out correctly. You'll have to hire like a lot of developers to actually build the playbooks out and then integrate them in. And sometimes firms don't actually set up the workflow action properly. So it still feels very disjointed to, to actually run like an investigation, which means no one's going to use it. So you spend a bunch of time and money on a platform that no one's going to use, right? So I think like having a proper workflow, again, having a very structured organization for the things you actually want to understand on how to actually do for your clients is super and super important. Like that's like the baseline thing because everyone is on the same page. This isn't going to work.  

Yeah. So I mean, if I'm hearing you right, basically it still comes back to we got to sit down, talk about what our environment looks like, understand the threat model, both for our industry and for our organization relative to, let's say, our architecture and then design what we think our response playbook should be like. And then and only then start thinking about using AI to automate some of that or augment the humans in that process. Is that maybe the right way to think about it?  

Yeah. I mean, it's still like garbage in, garbage out, right? If your processes aren't good, your crappy data, it's going to give you incorrect information.  

But but like crappy data should never be the case, right? Like, isn't that part of the point of a SIEM? Is that like, you know, we're collecting all the relevant data? Or do you still see organizations getting that wrong where they're like pumping in maybe the wrong data feeds or not getting all the right ones or what, what, what's going wrong in garbage in, garbage out.  

I mean, firm, firm view a SIEM as like a data lake. And those two things are different, right? A SIEM is just for security use cases that are relevant towards your organization, meaning like actual hot data should be stored inside of SIEM, right? You shouldn't, you should ideally for like in twenty twenty six, shouldn't, you shouldn't use SIEM directly to store data like longer than a year. Um, most SIEM should just search data that is probably twenty four hours old or like up towards thirty days old. And the rest of it, you can reroute it and put it either in a data lake on long term storage, S3 bucket, Azure blob storage, like whatever the case may be. And if different platforms that will help do data pipeline, like for example, I'm really good at that. Yep. Um, so I've seen a lot of issues where clients will just ingest everything, right? And so we're having to filter through like a lot of stuff that really isn't relevant towards like security cases specifically and makes it seem a lot slower and cost more money too.  

But then, but then like this begs the question in my mind, which is that like, the way that I always, was always pitched. SIEM, is that like, this is your audit trail. Like this is when your regulators come calling, you point them at your Splunk, you point them at your Azure Sentinel, you point them at your Elastic instance, because that is the record of all your security events. So are you saying that like, you know, it's actually more of a operations environment as opposed to like a historical record? Like what's the, what's the trade off or what's the, the kind of way to think about that?  

You're correct. It used to be when we invented and designed, it used to be a historical record. But over the past, like ten, ten to five years or five to ten years, technically speaking, it's evolved to where you have other platforms that can more properly and cheaply manage long term storage. So if you have like an AWS data lake, right, for security or. Yeah, um, you can look if, if our company looks for something that's older than a year, you can just search that, right? That, that isn't like an investigation, right? Where time is of the essence, right? Because it's an audit, you probably have a week to do it. Um, I think Yes. Right. Those are for like very permanent, very critical events are happening within the organization.  

Okay. Okay. So then what's the primary use case for the security data lake? Is it, you know, post threat hunting forensics? Is it this audit trail or is it like more general purpose? And you can do all of the above on it.  

It depends. Um it can be very good for threat hunting if you're looking for like longer term threats over a longer, over longer time horizon for your organization. But normally threat hunting is much more proactive. So normally you're working in the background to look for threats that mapped out towards your specific environment, your specific industry, right? So for example, a threat or an apt or advanced persistent threat actor isn't going to target the same type of industries, right? So if someone targets Walmart, then they're not going to target, um, LG or like AT&T, right? Because the threat model will be different and type of attacks will be different attack vectors. So if you have like a data lake, right, um, you can do much longer term storage. You can store much more data, and it's a lot cheaper because the speed to actually search it is lower because you're paying a lower cost for it.  

Okay, so that goes back to that kind of question about time sensitivity, right? Like your SIEM is optimized for like real time queries. And so it consumes more CPU as a simple way to think about it. So it's going to be more expensive to kind of operate store data and etc.. Whereas security data lake to the point that you said earlier, like maybe we've got longer to look at this. It's not as time sensitive. So we can, you know, we can actually run this on, let's say like slightly lower resourced kind of hardware or virtual hardware environments. Makes a ton of sense when you think about threat hunting. Today, I got to think that there's a big AI use case here as well, primarily just in the sense of like helping SOC engineers or SOC analysts who may not be programmers by background, but let's say like in writing structured queries to go, look, you know, if I've got a hypothesis about something that I really want to investigate, you know, I've got to think that an AI is very good at helping me write the queries or figuring out which you know, which data sources to go look at from that threat hunting perspective. Is that correct?  

Yes. Because this is what I do for my job and I use AI for, um, it's very helpful for building the initial thought process to get started. And then you can pivot and go from there, especially if you have like a lot of clients or there's a lot of environments and you simply don't have the time to like look through like forty different data sets and different data tables, right? It's just not practical. So you can, as you can ask, like Claude, like, hey, I'm looking for a type of threat. What, what table would this reside in? And it'll give you, give you a good baseline search and you can kind of tweak it from there and then go about doing the rest of your, your threat hunt to go from there. So it's very helpful with the preliminary searches to get a better sense as to how this environment actually look, what data does it actually hold, and then what type of threats might be inside of it.  

Okay. So, you know, when we think about software engineering these days, you very often hear that like, you know, the value is not in the code being written because, you know, AI engines are at least as good at as a human at writing the actual code. The value is in the human is in sitting down and thinking about what is the thing that I'm trying to build and how am I trying to build it? You know, do I want this monolith? Do I want a collection of microservices? What do I want each microservice to do? I hear a very direct parallel to kind of detection, engineering and threat hunting these days, where the value in the SOC analyst or in the threat hunter is not in the ability to write the, you know, Python scripts or to write the SQL queries. But in to think about what is the thing that I'm trying to detect or the thing that I'm trying to hunt? Does that parallel hold in your mind?  

I think so. So like the boat is like, like, right. Syntax has been lowered because of like AI chat boxes and, and LLMs. Right. Especially because I'm learning, I'm learning K8s stuff for Kubernetes. Um, so I use a lot of time to write, like write my Yaml scripts. Because for me, like I remember I send text at this point, isn't that beneficial? Like I'm thinking on a much grander, but like higher level scale, like how to design it, right? Like, um, what should the law go and how should look? So I'm trying to focus on that and then I'll like. Write the most basic basic queries, for example. So I think it's going to be helpful for doing that. Um, so for example, detection engineering is more reactive from the standpoint of you normally write a rule, then you wait, you wait for it to fire, then you respond to it's alert compared to threat hunting, which is much more proactive. Um, so there are so for a lot of firms that are like medium size, the detection engineers and the threat hunter will be one person. For larger firms like let's say JP Morgan or Citibank or Chase, they'll be separate roles. So it just depends.  

Got it. Got it makes a ton of sense. But then this begs a question how do you train somebody coming up through the ranks nowadays to think about that higher level, because I feel like that higher level you only develop by going through, you know, some of the experience, right? Having the the couple of years of working in a SOC, having a couple of years working on incident response investigations, etc.. So how do you get somebody from, you know, somebody who's come up coming up right now who never learned syntax, but now they've got to like land in an environment. They've got to learn what real world incidents look like. How do you think about that? In the modern AI equipped SOC and the modern AI equipped threat hunting environment?  

I would probably say, I mean, from my experience, it's always been trial by fire, just like, hey, kind of we have, we, so we have a new tenant that we're calling you. You're going to be for one hundred for them. Oh my God. I'm like, I don't, I don't know what that is. Very cool. Then learn. And you kind of just like you do it or you don't keep your job has been my personal experience from it. Um, so I tried to, for whatever organizations I've worked for, like I build processes and systems in place as best as I can to make sure whoever comes in behind me, if I win a lottery or if I get hit by a bus, they can do better than I did. So a lot of times, not a lot of times I always, always take really good documentation. I'll take screenshots, I'll take notes, I write down my thought processes, and I write down the queries that I'm trying to actually run and trying to do as well. So they know like exactly like step by step, like what I did, like my like litmus test is if, um, I gave this documentation to my grandma, if she could do this and not like, not like cry at the end. Uh, I mean, it's, it's like good enough to do this. My, like, my litmus test for that.  

Yeah, makes a ton of sense. I want to close on a segue from exactly this topic. How do you get this trial by fire? How do you get people into this? SOC analyst has been notoriously one of the hardest roles in cybersecurity long hours. You know, we talked about false positives, high stress, you know, often a very important but underappreciated role within organizations. And so I think consequently, and probably kind of rightfully, it's a high turnover position. So I know a lot of organizations, especially, let's say like MDR providers or large, you know, big, big companies that have large SOC teams, twenty four over seven operations, etc., where this role, they're always hiring for SOC analysts. So I think I worry a little bit about kind of people breaking into cyber security. Are there enough people interested? How do you get into it? And you hear this tension about like, everybody wants the entry level salary cyber security analyst who has five years of practical experience and a bunch of certifications. What's been your experience? What are some of your observations from having worked both, you know, in a SOC and as a consultant for a number of firms? What do you think is the right way for us as an industry to think about encouraging people into this role, getting them in, training them up. How should it all work in an ideal world?  

I think I think the big thing is just being honest with them, with people that want to do this right. Making them understand how hard this is and the things they can get from it. Right? Because my, my personal thought process has always been for most of my life, my parents are Jamaican immigrants is everything in life that's worthwhile is going to be hard. There are no shortcuts towards it. If you want to be a doctor and be a lawyer, you be in any high paying career field. It's going to take a lot of work and effort. That's just the reality of it. I don't know any shortcuts around it unless you like, like, unless you're like a trust fund baby, which I'm definitely not. It's going to be a lot of work, right? It's just kind of how it's set up. So I think, um, back in the past, I think a lot of people were lied to. So the expectations were one thing like, hey, like get a job within six months if you do this certain boot camp or like whatever. And so they have that expectation of like working hard for six months, then they're not getting a job right now. Now, if someone was honest with them and told them the expectation that, hey, this might take upwards of like two to three years to get in through various different roles. Then their thought process for those that stick on that path won't be as sad or depressed if they don't get a role within like a couple of months. It's just kind of like how it's set up. So I think the biggest thing is looking at it from a couple lenses, right? The technical hard skills, like soft skills and branding and marketing, technical hard skills, everything we talked about so far on this session is like scripting, learning how to knowing how to use AI, putting queries, doing threat hunting stuff, all that fun jazz you see in TV. Those are technical hard skills, soft skills, and the ability to actually like translate the things that, um, we're talking about to actual English. So someone that is not technical can actually understand it. And the last part would be like branding and marketing or being able to actually sell yourself towards somebody.  

Yeah. Makes a ton of sense. Makes a ton of sense. Kenneth, thank you so much for talking through all this with us today. I wanted to know if you've got any final thoughts or where people can find more about the work that you're doing. If they want to look, look you up and kind of, you know, reach out and communicate with you.  

Yeah. So we're on, um, LinkedIn, me personally, Kenneth Ellington or Ellington Cyber Academy, my website. You can also find us at Ellington cyber academy dot com as well. And then we have a new cyber range as well that's on my home page. Or you can go on school and put cyber range and it'll pop up as well.  

Awesome, awesome. And we'll have those linked from the show notes for today's episode. Kenneth, thank you so much for talking through what is again, one of the most kind of misunderstood, but still mission critical activities within cyber security. And I think one of the areas that there is the most speculation around where AI can have the biggest impact in terms of AI in cyber, and that is in the SIEM, in the SOAR in incident response, all of those areas. Thanks so much for taking the time to join us on Modern Cyber today.  

Thank you. Thank you for having me.  

Awesome, awesome. And to our audience, as always, rate and review, like and subscribe, all that good stuff. We'll talk to you next time. Bye bye.  

‍