Elastic Security app enables affordable threat hunting

Finding the appropriate threat hunting tools on a budget isn't easy, especially for newer infosec pros.

The Elastic Stack may solve that problem, said Andrew Pease, principal security research engineer at Elastic and author of Threat Hunting with Elastic Stack.

In his book, Pease introduces companies and beginner security researchers to the benefits of using the Security app included in the Elastic suite of open code tools.

He has been using the Elastic Stack for nearly 10 years. In his military career, he has taught security operations, threat hunting and incident response using it. He also co-founded a company, Perched, that did security workloads on the Elastic Stack and was acquired by Elastic in 2019.

Pease wrote Threat Hunting with Elastic Stack as a primer on threat hunting and the theory around it. The book also explains how to set up the Elastic Stack and its Security app to examine log data for security events. From there, threat hunters can visualize and present the relevant security data to executives.

Here, Pease gives further insights into the book and using the Elastic Stack for security.

Read an excerpt from Chapter 8 of Threat Hunting with Elastic Stack that provides step-by-step instructions on creating detection rules in the Elastic Stack Security app.

Editor's note: The following interview was edited for clarity and length.

Why did you write a book focused on using the Elastic Stack for threat hunting?

Learn more about Andrew Pease's
Threat Hunting with Elastic
Stack here.

Andrew Pease: There are tons of how-to guides out there. I wanted to write a book that covered theory and critical thinking and how to leverage tools to solve problems. It's around the operator aspect of being an analyst -- their ability to identify what they need to do and then to think critically about data to solve problems. I've been doing threat hunting and cyber and incident response for a long time, and there isn't a direct manual explaining that, if A happens, then you need to do B. Being able to critically think about each situation is crucial.

What convinced you to try the Elastic Stack when starting out in security?

Pease: [When in the military], we didn't have money but needed a platform for incident response tasks. We needed a way to send the alerts collected from a network platform into something we could have analysts look at. While the Elastic Stack wasn't meant to be a traditional database, that's what we used it for. SQL, at the time, was too cumbersome for the volume and velocity of data we were collecting.

I'm sure there were better ways to do it, but at the time, you could funnel data into Elasticsearch and get rapid insights into what we were doing from an incident response perspective. From there, we built tools on top of the stack to fit our use cases.

We had no money, and the Elastic Stack was free and had an active open source community. It wasn't a situation where Elastic just had a free tier that does the basics and then you need to buy up to do exciting tasks. People run entire companies based off the Basic license version of the Elastic Stack. That's what brought us in and what kept us there.

We tried some of the other big platforms but struggled with how, once you've ingested a certain amount of data, you had to pay for it. It's a legitimate business model, but when you're in security, don't have any money and you're trying to solve real problems, it doesn't work well to have to make decisions about what data to collect or not collect to keep using a free license.

Who will benefit the most from reading Threat Hunting with Elastic Stack?

Pease: Intelligence analysts who are either new or not involved in cyber yet. You only need an understanding of the basics to follow my book. For example, know what an IP address is and how the internet and networks operate. There isn't a tremendous amount of knowledge required, certainly not on the Elastic Stack. I wanted to start at ground zero. The first three chapters have nothing to do with technology, instead they build on process and theory. From there, I worked into the Elastic Stack from the ground up.

The book is written for people who want to use the Elastic Stack to do security operations. When I say security operations, I'm referring to traditional security monitoring, proactive threat hunting and intelligence analytics. I want people to be able to look at the data and identify how to create better data-collecting methods. I wanted to approach the subject that threat hunting is really the merging of intelligence analysis, collecting and analyzing data, and understanding the technology.

How should readers start using the Elastic Stack for threat hunting?

Pease: Start with the Discover app. It provides an unfiltered view of all your data. Once you understand what kind of data is in there, you can create dashboards and visualizations. One of the things that makes Elastic powerful is how the collected data can be used in various ways. For example, ingesting web proxy logs is valuable from a threat hunting perspective, but you can use that data to see the wait times for users trying to connect to the internet, too. That data is all in Discover.

Next, you can examine the data in Elastic Observability or the Security app. It's the same data but available in a different context and displayed differently. Once you understand the data coming into your Stack, you can stitch a lot of it together and develop a theory about it.

You don't have to use each Elastic app to succeed -- I call that out in my book. Discover is probably the most valuable tool, since you can see all your data. Then, you can apply filtering techniques to shape it into something interesting. Once you understand the big buckets, you can zero in using the visualizations. Plus, the Security app does a really good job of presenting relevant security data.

Do you use any tools alongside the Elastic Stack?

Pease: Yes. As a full-time threat researcher, the amount of data we collect is tremendous. We use a variety of sources to collect that data, which then goes into the Elastic Stack. We also have tools for creating honeypots and for digital forensics works. There are plenty of tools out there. But I've found that, for security, the Elastic Stack serves as a single search platform to get you enough information to determine if something 'interesting' is happening on an endpoint or the network.