If you have decided to use Linux or another Unix-like operating system for your dedicated server, you probably factored in security into your consideration. Linux, BSD, and similar OSes are renowned for their inherent security features. Nevertheless, is still important to be diligent and make sure your server is as secure as possible. A unprotected Linux server can fall victim to security flaws just as easily as any other. This is especially true for servers connected to the Internet.
There is unfortunately no single security tool that will make Linux rock solid for you, but with the right combination of free and open source tools you can find online, you can make your server virtually impenetrable.
1. APF (Advanced Policy Firewall)
APF is based on the default Linux kernel firewall called iptables, but it adds a user-friendly configuration system and a comprehensive rules management feature. Features include static rule-based policies, connection-based stateful policies, and sanity-based policies. APF has a boatload of features and has been released to the open source community R-fx Networks, which provides a host of other security tools, including BFD (Brute Force Detection).
2. OSSEC – Open Source Host-Based Intrusion Detection System
OSSEC is an intrusion detection system developed by Trend Micro, the company responsible for several widely-used antivirus and security tools for Windows. OSSEC is designed for servers running any number of operating systems. From its rootkit scanner to its detailed log file analysis, OSSEC keeps you covered in terms of security intrusion protection. Features include file integrity checking, log monitoring, rootkit detection, and active response.
A network firewall is only one line of defense when protecting your server. Even if you block out attackers who wish to exploit network vulnerabilities, your web applications may still present an avenue for them to attack. ModSecurity is an application firewall that sits between your scripts/web applications and the network. Most servers host dynamic websites using server-side scripting, such as PHP, Perl, Python, or Ruby. Attackers can exploit these scripts and use them to launch attacks, infiltrate your server, or even gain root access. While writing air-tight scripts and locking down certain features may help, ModSecurity takes it to the next level and gives you full customization over security rules and settings.
For a system administrator, spam is evil. While there are plenty of things that could harm your server, few are as annoying and persistent as spam. SpamAssassin is a complete anti-spam system that is used on a great portion of the servers all over the world. Quite simply, when spam arrives at the system’s mail server, SpamAssassin will check the mail based on preset rules and the training initiated by the sysadmin and/or user. If the mail fails to pass a certain level of spam filtering, it is flagged and dealt with according to system settings.
5. OpenVas (vulnerability scanner)
While many security tools are designed to prevent certain attacks or to find and neutralize malware or security breaches, OpenVas is a tool designed to find any weaknesses in your server. Before your server is ever even subjected to an attempted attack, OpenVas will tell you which components of your server are likely targets based on vulnerability scanning.
6. SELinux – Mandatory Access Control (MAC)
By default an application running under a particular user has access to everything that user owns because it is a virtual representation of the user. When running server programs, this can be a serious security risk. One workaround is to run the software under a different user unique to that program. When that is not enough, a tool like SELinux may suffice. Originally developed by the U.S. National Security Agency (NSA), SELinux (Security Enhanced Linux) provides access control security policy features for users. It behaves as a set of kernel add-ons that can be applied to Unix-like operating systems.
Your log files can provide a wealth of information about your server, both for security and monitoring. The kernel itself has a log, and each application that runs on top of it generally keeps a log as well. Logwatch is a tool that monitors your server’s logs and contacts you when there is a security problem. Logwatch anyalzes your logs based on your judgement and creates a report that you can use to track security issues and anything else that concerns your server’s health.
There are plenty other security tools on the market, but these seven free and open source projects are among the most useful for finding vulnerabilities, scanning for attackers, and stopping all sorts of intrusion. Many of the projects are actively maintained and may even be available in your Linux distribution’s repository.