SIEM evaluation criteria: Choosing the right SIEM products

1. How much native support does the SIEM provide for relevant log sources?

A SIEM's value is diminished if it cannot receive and understand log data from all of the log-generating sources in the organization. Most obvious is the organization's enterprise security controls, such as firewalls, virtual private networks, intrusion prevention systems, email and web security gateways, and antimalware products.

It is reasonable to expect a SIEM to natively understand log files created by any major product or cloud-based service in these categories. If the tool does not, it should have no role in your security operations.

There are many SIEM systems available today, including light SIEM products designed for organizations that cannot afford or do not feel they need a fully featured SIEM added to their current security operations.

In addition, a SIEM should provide native support for log files from the organization's operating systems. An exception is mobile device operating systems, which often do not provide any security logging capabilities.

SIEMs should also natively support the organization's major database platforms, as well as any enterprise applications that enable users to interact with sensitive data. Native SIEM support for other software is generally nice to have, but it is not mandatory.

If a SIEM does not natively support a log source, then the organization can either develop customized code to provide the necessary support or use the SIEM without the log source's data.

2. Can the SIEM supplement existing logging capabilities?

An organization's particular applications and software may lack robust logging capabilities. Some SIEM systems and services can supplement these by performing their own monitoring in addition to their regular job of log management.

In essence, this extends the SIEM from being strictly a centralized log collection, analysis and reporting tool to also generating raw log data on behalf of other hosts.

3. How effectively can the SIEM make use of threat intelligence?

Most SIEMs are capable of ingesting threat intelligence feeds. These feeds, which are often acquired from separate subscriptions, contain up-to-date information on threat activity observed all over the world, including which hosts are being used to stage or launch attacks and what the characteristics of these attacks are. The greatest value in using these feeds is enabling the SIEM to identify attacks more accurately and to make more informed decisions, often automatically, about which attacks need to be stopped and what the best method is to stop them.

Of course, the quality of threat intelligence varies between vendors. Factors to consider when evaluating threat intelligence should include how often the threat intelligence updates and how the threat intelligence vendor indicates its confidence in the malicious nature of each threat.

4. What forensic capabilities can SIEM products provide?

Forensics capabilities are an evolving SIEM evaluation criteria. Traditionally, SIEMs have only collected data provided by other log sources.

However, recently some SIEM systems have added various forensic capabilities that can collect their own data regarding suspicious activity. A common example is the ability to do full packet captures for a network connection associated with malicious activity. Assuming that these packets are unencrypted, a SIEM analyst can then review their contents more closely to better understand the nature of the packets.

Another aspect of forensics is host activity logging; the SIEM product can perform such logging at all times, or the logging could be triggered when the SIEM tool suspects suspicious activity involving a particular host.

5. What features do SIEM products provide to assist with performing data analysis?

SIEM products that are used for incident detection and handling should provide features that help users to review and analyze the log data for themselves, as well as the SIEM's own alerts and other findings. One reason for this is that even a highly accurate SIEM will occasionally misinterpret events and generate false positives, so people need to have a way to validate the SIEM's results.

Another reason for this is that the users involved in security analytics need helpful interfaces to facilitate their investigations. Examples of such interfaces include sophisticated search capabilities and data visualization capabilities.

6. How timely, secure and effective are the SIEM's automated response capabilities?

Another SIEM evaluation criteria is the product's automated response capabilities. This is often an organization-specific endeavor because it is highly dependent on the organization's network architecture, network security controls and other aspects of security management.

For example, a particular SIEM product may not have the ability to direct an organization's firewall or other network security controls to terminate a malicious connection.

Besides ensuring the SIEM product can communicate its needs to the organization's other major security controls, it is also important to consider the following characteristics:

• How long does it take the SIEM to detect an attack and direct the appropriate security controls to stop it?
• How are the communications between the SIEM and the other security controls protected so as to prevent eavesdropping and alteration?
• How effective is the SIEM product at stopping attacks before damage occurs?

7. Which security compliance initiatives does the SIEM support with built-in reporting?

Most SIEMs offer highly customizable reporting capabilities. Many of these products also offer built-in support to generate reports that meet the requirements of various security compliance initiatives. Each organization should identify which initiatives are applicable and then ensure that the SIEM product supports as many of these initiatives as possible.

For any initiatives that the SIEM does not support, make sure that the SIEM product supports the proper customizable reporting options to meet your requirements.

Do your homework and evaluate

SIEMs are complex technologies that require extensive integration with enterprise security controls and numerous hosts throughout an organization. To evaluate which tool is best for your organization, it may be helpful to define basic SIEM evaluation criteria. There is not a single SIEM product that is the best system for all organizations; every environment has its own combination of IT characteristics and security needs.

Even the main reason for having a SIEM, such as meeting compliance reporting requirements or aiding in incident detection and handling, may vary widely between organizations. Therefore, each organization should do its own evaluation before acquiring a SIEM product or service. Examine the offerings from several SIEM vendors before even considering deployment.

This article presents several SIEM evaluation criteria that organizations should consider, but other criteria may also be necessary. Think of these as a starting point for the organization to customize and build upon to develop its own list of SIEM evaluation criteria. This will help ensure the organization chooses the best possible SIEM product.