Network anomaly detection: The essential antimalware tool
Traditional perimeter defenses are no longer enough; network anomaly detection tools are now essential in the battle against advanced malware.
In part 1 we learned why firewalls, IDS and IPS cannot adequately protect against advanced malware; advanced malware is specifically engineered to defeat those devices. Firewalls, IDS and IPS are based on signatures and rules that define "known bad" activity, and while they can be effective in preventing some data exfiltration after malware has permeated a network, they are inherently ineffective in detecting advanced malware during initial entry to a network.
As a result, a change of focus is needed. Security teams must go beyond trying to detect and deny malware at the border. The planning assumption has to be made that it is not possible to detect and deny all advanced malware threats at the border. Instead, security tools must now focus on the interior of the network and possess network anomaly detection capabilities.
An expanded focus means identifying all mission-critical information assets (as identified through an IT risk assessment) and then monitoring these assets to detect unwanted behavior. When tens of thousands of customer transaction records and credit cards are streaming from a database and out through the firewall, isn't that something that should be noticed and stopped?
Network traffic analysis with IP Flow
Network traffic analysis, based on the IETF Internet Protocol Flow and Information Export (IPFIX) protocol, is perhaps an underutilized tool that can help identify unwanted behavior by advanced malware inside the network despite the encryption advanced malware uses to conceal itself.
Computer network traffic analysis, or network flow analysis, is a method of network traffic analysis based upon the concept of an Internet Protocol Flow (IP Flow).
Each packet that is forwarded within a router or switch is examined for a set of IP packet attributes. These attributes are the IP packet identity, or fingerprint, of the packet and determine if the packet is unique or similar to other packets.
Traditionally, an IP Flow is based on a set of five to seven IP packet attributes. IP Packet attributes from an IP flow are:
- IP source address
- IP destination address
- Source port
- Destination port
- Layer 3 protocol type
- Class of Service
- Router or switch interface
All packets with the same source/destination IP address, source/destination ports, protocol interface and Class of Service are grouped into a flow. Then packets and bytes are tallied. This methodology of fingerprinting, or determining, a flow is scalable because a large amount of network information is condensed into a database of information called the cache.
Computer network traffic analysis, therefore, is based upon collecting and analyzing IP flows to determine the characteristics of network communication that is taking place.
This flow information is extremely useful for understanding network behavior for a number of reasons:
- Source address allows the understanding of who is originating the traffic.
- Destination address tells who is receiving the traffic.
- Ports characterize the application utilizing the traffic.
- Class of Service examines the priority of the traffic.
- The device interface tells how traffic is being utilized by the network device.
- Tallied packets and bytes show the amount of traffic between any point on the network.
- Flow information allows for operating system identification, including the identification of rogue operating systems.
- Flow information also allows for identifying network traffic from common applications.
- Network traffic from unwanted applications can also be identified.
- Flow information can monitor bandwidth utilization and identify unexpected or excessive bandwidth utilization.
Using a network traffic baseline for malware detection
This network flow information makes it possible to establish a baseline of normal network traffic behavior. Such a baseline makes it easier to then identify unexpected or unwanted behavior, including malicious activity caused by advanced malware. In other words, it becomes possible for an organization to determine whether a user's account is transmitting a large amount of email directed by a botnet, or large amounts of intellectual property data are attempting to exit the network; when compared against the activity baseline profile, data transmissions like these will look anomalous.
Similarly, if a user were to reboot a workstation into a new operating environment -- for instance, a virtual machine or a different operating system -- the network traffic analysis system could detect that behavior as well.
Availability
Today, all recent routers and many Layer 3 switches support Cisco's NetFlow protocol or the IPFIX protocol, with one or both being a prerequisite to creating the network traffic baseline described above; increasingly, routers and switches support both protocols.
From there, collection and analysis systems need to be implemented to allow for network traffic analysis based on the protocol(s) the network infrastructure supports. The cost to put these collection and analysis systems in place ranges from zero (for open source systems) to several hundred thousand dollars (for commercial products); the cost of the commercial technology depends upon its scope and scale.
Trusted gateway
Network traffic cannot be monitored if it can't be seen. Consider consolidating all enterprise Internet traffic through a single trusted gateway system and then monitor that traffic continuously.
The executive branch of the U.S. government did exactly this through its Trusted Internet Connections (TIC) initiative. The 110 executive branch departments and agencies have consolidated more than 8,000 Internet connections into fewer than 100 continuously monitored Internet gateways worldwide. Fewer ingress points make it easier to monitor what enters the network, making it harder for malware to slip through.
In addition to requiring a trusted gateway to the Internet, TIC also required numerous security management and technical controls, including the following:
- Application proxy servers for bi-directional inspection of Web, mail and file transfer applications.
- A security information and event management system (SIEM) to aggregate, correlate and analyze all security event alerts across all security devices.
- The Security Operations Center, which responds to alerts from the gateway devices and the SIEM and acts as a front end to the incident management capability.
You too must create a single trusted gateway to your network, and then carefully monitor the traffic that traverses it. Remember, though: You must search it continually for both everyday malware and advanced malware. It's essential, in other words, that you extend your security controls beyond the perimeter, adding network anomaly detection capabilities to your other existing security tools.
About the author: Peter Sullivan began his career in network operations, information security and incident response 20 years ago with the U.S. Army. For the last ten years, Sullivan has been a visiting scientist at the Software Engineering Institute, Carnegie Mellon University, where he teaches courses in risk management, information security and assurance, computer security incident response and digital forensics. He is also a partner with InfoSecure Solutions, LLC, a Massachusetts based consultancy specializing in IT risk management and incident response planning. Sullivan holds a CISSP certification and a CERT/CC Computer Security Incident Handling (CSIH) certification.
