Security log management and logging best practices

What are log files?

Log files are detailed, text-based records of events within an organization's IT systems. They are generated by a wide variety of devices and applications, among them antimalware, system utilities, firewalls, intrusion detection and prevention systems (IDSes/IPSes), servers, workstations and networking equipment.

Log files provide a vitally important audit trail and can be used to monitor activity within the IT infrastructure, identify policy violations, pinpoint fraudulent or unusual activity and highlight security incidents.

Because logs contain details of what has happened and what is happening, security teams can use them to detect and respond to indicators of compromise, investigate and analyze where an attack is coming or came from, and establish how an attack has affected IT resources.

What is log management and why is it important?

Security log management comprises the generation, transmission, storage, analysis and disposal of security log data, ensuring its confidentiality, integrity and availability.

This process is so important that the Center for Internet Security lists log management as one of its critical security controls. It's critical because organizations that fail to collect, store and analyze system events leave themselves open to attack. This is also why log management is required for compliance and reporting by various laws and standards, such as Federal Information Security Modernization Act, ISO 27001, HIPAA, Sarbanes-Oxley Act, Gramm-Leach-Bliley Act, National Industrial Security Program Operating Manual and PCI DSS. Logs are also needed to perform general audits, establish baselines, and identify operational trends and longer-term problems.

Security log management challenges

Security log management is not easy. Even if only recording events that cover the most important metrics, a small organization still generates a large amount of loggable data. Large enterprises can produce hundreds of gigabytes of logs. Handling this continuous and voluminous supply of data presents its own challenges.

Because logs come from multiple endpoints and different sources and formats, they require normalizing. Transforming the information into a uniform format for easy searching, comparison and readability is key. The systems and media used to share and store logs must be highly secure with tightly controlled access. In addition, they must be capable of processing large amounts of data without impairing overall system performance.

Which events should be logged?

The security events an organization captures depends to some extent on industry-specific needs and relevant legal requirements. That said, there are several events that should always be captured and logged to ensure user accountability and to help companies detect, understand and recover from an attack. Such events include the following:

• Authentication successes and failures.
• Access control successes and failures.
• Session activity, such as files and applications used, particularly system utilities.
• Changes in user privileges.
• Processes starting or stopping.
• Changes to configuration settings.
• Software installed or deleted.
• Devices attached or detached.
• System or application errors and alerts.
• Alerts from security controls, such as firewalls, IDSes/IPSes and antimalware.

Fault logging -- that is, faults generated by the system and the applications running on it -- is also important as the data can be used to find out what is wrong with a system or application and identify trends that may indicate faulty equipment.

What constitutes a log entry?

Information that should be recorded in a log entry includes the following:

• Date and time.
• User and/or device ID.
• Network address and protocol.
• Location when possible.
• Event or activity.

Compromised or inaccurate logs can hamper investigations into suspicious events, undermine their credibility, and invalidate disciplinary and court actions.

One way to ensure logs are trustworthy is to use synchronized system clocks, giving every log entry an accurate timestamp. This involves obtaining a reference time from an external source, combined with a network time protocol, to sync internal clocks. Always record the time of an event in a consistent format, such as Coordinated Universal Time. For additional security, add a checksum.

Security log rules and log data integrity

Strict rules should govern how and when logs are deleted, with controls designed to ensure ample log storage media is available. Otherwise, events may not be recorded or past events will be overwritten.

While capturing as much data as possible and storing it for as long as possible appears to be a pragmatic approach, in reality, this isn't feasible. Logging levels should match the level of risk. The table below is an example of prudent logging configuration settings, according to NIST's "Guide to Computer Security Log Management." Note, certain laws and standards may require different settings to those in this table.

The integrity of log data is paramount. Controls must protect against unauthorized changes to log files -- the first step most hackers take is to alter log files to hide their presence and avoid detection. To protect against this, record logs both locally and to a remote server located outside the control of regular system managers. This framework provides redundancy and offers an extra layer of security as the two sets of logs can be compared against one another, with any differences indicative of suspicious activity.

A lower-cost alternative to a dedicated log server is writing logs to write-once media to prevent an attacker overwriting them. Remember that sensitive and personally identifiable information can be captured in event logs. As a result, proper privacy controls must be implemented as well, using such tactics as anonymization or pseudonymization.

Other security logging best practices

Beyond capturing the proper events, including the necessary info in a log entry, implementing log rules and ensuring log integrity, here are three other best practices to follow.

1. Remember, logging is only the first step

Even if appropriate volumes of the correct data are being collected, it is worthless unless that data is monitored, analyzed and the results acted on. Logging and auditing ensure users are only performing the activities for which they are authorized. These processes also play a key role in preventing inappropriate activity, as well as ensuring hostile activities are spotted, tracked down and stopped.

2. Give admins, system operators extra scrutiny

One area of log management that requires extra consideration is administrator and system operator activities. These users have powerful privileges, and security teams need to carefully record and check their actions. To that end, these users should not be allowed physical or network access to logs of their own activities. Additionally, those tasked with reviewing logs should be independent of the people, activities and logs being reviewed.

3. Use logging tools

Due to the volume of incoming data organizations confront every day, most need a dedicated log management system to make management, event correlation and analysis easier. A specialized system also improves the quality of dashboard data and reports.

SIEM is a common approach used to aggregate log data from multiple sources. SIEM systems can parse and analyze data in real time to identify deviations from established baselines. If an anomaly is detected, SIEM systems can generate alerts, possibly activating additional security mechanisms. They can be rules-based, often employing a statistical correlation engine to establish relationships between event log entries. Advanced systems also rely on user and entity behavior analytics, as well as security orchestration, automation and response tools.