Installing the ModSecurity Web application firewall on Red Hat Enterprise Linux

ModSecurity provides an open source Web application firewall (WAF) solution. It is based upon the popular Apache server, and can be easily deployed by most Linux based Web shops. WAFs are application-level firewalls that monitor access to Web servers to ensure that requests are both valid and not attempts to break into the Web server. The technique used for the inspection is referred to as “deep packet inspection” because the firewall is looking much further into the packet. 

A typical firewall examines the IP address, the protocol (TCP or UDP) and the port to determine if the traffic is authorized. A WAF looks beyond the traditional IP and TCP/UDP and examines the payload to determine if the traffic is malicious. Deep packet inspection detects anomalies in two ways: protocol awareness and signatures. With protocol awareness, deep packet inspection may look at things such as protocol versions, protocol inconsistencies, invalid requests, domain name resolution checking, and other methods that are easily identified as being out of the norm. With signatures, the payload within the payload of the packet (for example http) is inspected for specific signatures of hacks. For example, a request to a specific compromised version of a cgi program may come in as http://www.domain.com/cgi/hacked.cgi?a=4, which can be blocked based upon its signature.

Why use WAF?
Besides the additional security afforded by application aware firewalls, may organizations need to deploy WAFs to support payment card industry (PCI) projects to become compliant with the PCI DSS as required by Visa and Master Card.  The PCI DSS requires either the deployment of WAFs or manual procedures to ensure security is being applied.

Open source WAF solutions
There are two well known open source WAFs:  AQTRONIX and ModSecurity. AQTRONIX lends itself to deploying in IIS environments and mod_security in Apache environments. The mod_security product covered here can be placed in front of any Web server (Apache is not required).

ModSecurity requirements
I recommend using Apache 2.2.x.  or higher for your Web server. The steps below assume that Apache has not been installed and will use the newly added repository to ease installation for you.

DNS is fundamental for Web traffic and the use of the Web application firewall. Therefore, ensure DNS is functioning correctly on the Red Hat system running the WAF. Red Hat version 5.4 was used for this tutorial, but the instructions should work equally well under 5.3.

ModSecurity istallation
For these instructions we will install ModSecurity on the same server being protected and direct the traffic from port 80 to port 8080.  The web server’s address is 192.168.1.5.

ModSecurity can be installed from source code and detailed instructions are available. But, I prefer a modular approach to the install, and use package management to install ModSecurity.

To use package management a repository is necessary for the installation. I use Jason Litka’s repository for the install. To use Jason’s repository, add a repository to Yum to have it recognize Jason’s repository.  For example, edit /etc/yum.repos.d/utter.repo and add the following;

 [utter] name=Jason's Utter Ramblings Repo               baseurl=http://www.jasonlitka.com/media/EL$releasever/$basearch/ enabled=1 gpgcheck=1 gpgkey=http://www.jasonlitka.com/media/RPM-GPG-KEY-jlitka

Now that Yum has a way to recognize the repository Apache can be installed with:M

 yum install httpd

 yum install httpd-devel

ModSecurity can then be installed using:

 yum install mod_security

Unfortunately this method will also require some compilation for mod_proxy which is required by ModSecurity, so the gcc compiler will be installed (and later removed):

 Cd /root

 wget http://www.manticmoo.com/articles/jeff/programming/proxy/mod_proxy_html.c

 yum install libxml2-devel   yum install gcc apxs -c -I /usr/include/libxml2/ -i mod_proxy_html.c

The extras can be removed that were installed with the install httpd-devel:

 yum erase kernel-headers yum erase cpp yum erase libgomp

ModSecurity requires the libxml module, so dd the following lines to httpd.conf:

 LoadFile /usr/lib/libxml2.so LoadModule proxy_html_module modules/mod_proxy_html.so

Above the line:

 LoadModule proxy_balancer_module modules/mod_proxy_balancer.so

Now the proxy can be activate with these parameters in httpd.conf

 <Proxy *>

     Order deny,allow

     Allow from all

 </Proxy>

 SSLProxyEngine On ProxyPass / http://192.168.1.5:8080/ ProxyPassReverse / http://website.domain.com/

 <Location>

 ProxyHTMLExtended On </Location>

The parameter ProxyPass and ProxyPass reverse define where to send the actual traffic to. Typically this will be the fully qualified domain name of the Web servers load balanced address.

ABOUT THE AUTHOR: Ronald McCarty is a freelance writer and consultant specializing in systems, network, and information security. He received his bachelor's degree in Computer and Information Systems at the University of Maryland's international campus at Schwaebisch Gmuend, Germany and his master's degree in Management with a specialization in information technology at Capella University. Ron's company, Your Net Guard offers IT consulting and integration services in the Dallas/Forth Worth area. He can be reached at [email protected].