carloscastilla - Fotolia
How to use Packetbeat to monitor Docker container traffic
Docker containers can help secure cloud applications, but malicious traffic can still move to and from those containers on a network. Dejan Lukan explains how to use Packetbeat to monitor such threats.
Most cloud applications today are deployed using containers. One of the most widespread examples of this is Docker. But while containers can bolster cloud application security, there could still be threats lurking in Docker container traffic.
For example, security researchers last year discovered how Docker REST APIs can be misused by threats actors to hide malware on systems. In addition, the Meltdown vulnerability affecting Intel CPUs, which was disclosed earlier this year, can lead to data leaks even when popular containers, including Docker's, are being used.
Let's look at how to move Docker container traffic off the network interface in order to feed it into Elasticsearch/Kibana, and later analyze it.
Every cloud application works on top of some protocol that allows the clients to communicate with it in a standardized way; otherwise, the clients and servers wouldn't be able to talk to each other. Cloud applications run on the same protocols as normal applications, so this is nothing . There are many application-based protocols, but the most common are HTTP, domain name system (DNS) and the Simple Mail Transfer Protocol.
One tool that can be used to monitor, or sniff, the traffic coming into and going out of the container is Packetbeat, which listens to the traffic on a specific interface and sends it to Amazon Elasticsearch, where the data is stored.
Cloud service providers such as AWS are adding support for Docker containers into their environments, which further proves that analyzing Docker container traffic is an important aspect of container and cloud security practices. Traffic should be inspected and monitored further to detect possible threats to the applications running inside the containers.
The docker0 network interface
Before setting up Packetbeat, it's important to understand the basics of the docker0 network interface, which Packetbeat uses to listen for container traffic. A single interface has to be specified in the Packetbeat configuration file so it can listen for incoming and outgoing packets and send them to Elasticsearch for storage. The packets can then be viewed as graphs in the Kibana web interface.
When a Docker container starts, a veth* network interface is created for it. This allows the application running inside the container to communicate with the rest of the world through that interface.
It's also important to understand that Docker, by default, creates a docker0 network interface that acts like a network bridge to other veth* interfaces. Therefore, if you can directly access the docker0 network interface, you can monitor the traffic from all veth* interfaces automatically and see the incoming and outgoing traffic of every Docker container.
Because all veth* interfaces are joined to the docker0 network bridge, all the containers can communicate with the host and each other. This allows one service to talk to another that is running in a different container, but still on the same host. Note that all interfaces belonging to the same bridge are in the same collision domain, so when one interface broadcasts a request, it is sent over the bridge to every other network interface, any of which may come back with a packet response.
A virtual veth* interface is created for Docker containers and is then bound to the docker0 network interface on the host. In the container, this network interface has a standard name of eth0, which has a unique IP address from the bridge's range of available network addresses.
Every Docker container can accept an optional --net parameter, which allows for the specification of the following options:
none: The container has its own unconfigured network stack that allows it to build its own network inside the container.
container:name/id: The container shares the network with another container running on the same host, which is identified with NAME/ID. Note that the containers linked this way are able to communicate with each other through a loopback interface.
bridge: This is the default option and it connects the container interface veth* to the bridge docker0 interface.
host: This option allows the container to share the network with the Docker host in order to access all of its network interfaces.
In order for Packetbeat to sniff all the incoming and outgoing packets of every network interface belonging to running containers, it has to be configured with the --net=host option so it has access to the docker0 network interface. This allows Packetbeat to sniff every network packet coming from the internet to any Docker container.
Setting up Packetbeat to monitor network traffic
If Docker is already installed and ready to use, start Packetbeat in its own container. It can also be installed directly on the host, but with access to Docker, it's better to install it directly in the Docker container. This also enables you to automate this part of the infrastructure.
The Docker image for Packetbeat is already written, so it's simple to install it with one command:
# docker run --name packetbeat -d --net=host -d proteansec/packetbeat app:start
This will install Packetbeat in the container, which assumes that Elasticsearch is running on the local host and is accessible through port 9200. There are customizable options due to many parameters that can be passed to the container it starts.
By default, the configuration file sniffs the following network protocols running on specified ports:
DNS: 53
HTTP: 80, 8080, 8000, 5000, 8002
Memcache: 11211
MySQL: 3306
PostgreSQL: 5432
Redis: 6379
Thrift: 9090
MongoDB: 27017
Next, connect to the Kibana web interface and start monitoring the packets flowing into and leaving Docker containers. There are a lot of graphs that present different container traffic characteristics.
Conclusion
This just scratches the surface of how to detect malicious traffic in containerized cloud applications. Because many cloud service providers are integrating Docker into their architecture, detecting threats perpetuated by attackers is imperative. Despite the protections that containers offer, it's still beneficial to monitor Docker traffic, especially in light of recently disclosed threats and vulnerabilities.
Enterprises can defend against attacks on containerized applications by using Packetbeat to log traffic data and Kibana to analyze it. Packetbeat can also be used to detect what traffic is being sent over a network.
Normally, what is being exchanged between clients and an application -- let alone where the application is running -- is unknown, but if an application is running in a Docker container, Packetbeat can sniff the packets in order to analyze them. It's also possible to pipe the traffic to IDS/IPS, which could be beneficial because it can detect known threats inside container traffic that could have been blocked prior to being delivered to the actual application.
It's important to keep an open mind and think about possible defense mechanisms as new technologies emerge. While attackers are continuously discovering new ways to attack an infrastructure, security professionals must continuously look at options to protect their organizations, and Packetbeat is one of the tools that can help.
