Intrusion Prevention, Detection and Unified Threat Management

When I first began my IT career the only computer link we had to the outside world was a modem hooked up to the telephone which plinked away when dialling and broadcast a bunch of white noise around the room when connected. At that point we knew we had a live link to a local bulletin board and could upload some files.

Quite frankly the only intrusion detection system we ever needed was a rather loud receptionist who controlled access to our building.

Inevitably, as the internet took hold and business realised the benefits of being online with email and the worldwide web, modem connectivity was quickly replaced by ISDN lines and finally broadband. Switching connections on and off just didn’t figure anymore, and from the early 1990’s onwards organisations were hooked up to the internet 24×7.

With this nascent “always on” computing it was soon apparent that the door to businesses computer networks was open for all and sundry to enter and steal or damage data. Something clearly had to be done and quickly, so the IT security experts turned their minds to systems that could prevent and detect intruders.

Intrusion Prevention, Detection and Unified Threat Management
Intrusion, in the context of IT security, is the attempted or actual entry into a computer system by an unauthorised person. Occasionally this would be an attempt to steal data or, more often, a way of causing damage or propagating a virus or other malware. Sometimes this may be a denial of service (DoS) attack designed, in most cases, to overwhelm an IT infrastructure. In practice, most intrusions are self-propagating malware that search the worldwide web looking for vulnerable systems.

There is evidence that some system intrusions are now being initiated by organised criminals. Some have blackmailed online service providers, such as betting operations, with a threat to launch a DoS attack on busy sporting days. There are even indications that some governments are actively targeting systems belonging to countries they consider a threat—a sort of online cold war.

In an effort to prevent or defeat such attacks we have Intrusion Prevention Systems, Intrusion Detection Systems and now Unified Threat Management.

Introducing Intrusion Prevention Systems (IPS)
IPS works on the principle that prevention is better than cure. In fact many intrusion prevention activities can be undertaken without investing in expensive hardware or software. Creating and adhering to a good IT security policy is a great way of preventing intruders, as is running up-to-date and well configured anti-malware software on each client endpoint in your organisation. Of course you do need to have in place technology, of which a firewall, well configured, will be the mainstay.

Introducing Intrusion Detection Systems (IDS)
Intrusion Detection Systems are normally technology-based and used to detect if a system is being targeted. The system will monitor network traffic as it enters and leaves an organisation with a view to sounding an alarm if an unusual event occurs, which, in turn, may indicate a potential intruder. Often an IDS will have a pre-set action to take when an intruder is detected to minimise any possible damage from an attack.

Unified Threat Management (UTM)
Clearly the notion of having separate systems to detect and protect intruders can be inefficient. As is having separate systems to manage anti-virus or other anti-malware activities. To that end there is a considerable move in the market away from pure IPS/IDS to Unified Threat Management (UTM). With UTM, defence systems are aggregated in single management consoles and the overall control of threats is coordinated from one place. That way system duplication is eliminated and the ever-important cost of ownership reduced as much as possible. Over the coming years the differentiation between intrusion detection and prevention will become less important and we will be using the terms less and less. Instead unified threat management will become the catchall phrase.

Intrusion Detection Systems in Detail
The simplest way of thinking of an IDS is to think of a burglar alarm. As a burglar enters a property an alarm is sounded so that the police can be summoned. With IDS an alarm is raised (often via email or pager) and administrators informed of an intruder.

Associated with such a system are false negatives and false positives. Worse case scenario is a false negative, when your expensive IDS fails to trigger an alarm when an event occurs. The first you may know about it is users complaining about off line websites that have been nobbled in a denial of service attack. False positives, on the other hand, may be more irritating but are less problematic. These occur when the IDS believes that an attack has happened. On investigation it transpires that the event was not an intruder, rather an unusual business activity, but nothing to worry about.

More advanced Intrusion Detection Systems will raise an alarm along with a confidence factor based on an immediate assessment of the problem. This will be determined by the system logic and may be based on heuristics or learned behaviour once the system has monitored routine business traffic. Alert thresholds can be set, such as those with a 90%+ confidence factor will alert via a pager and probably those of a lesser confidence factor alert via email.

Clearly there is a lot of responsibility on the security team to ensure that the IDS system has been correctly set up

In practice Intrusion Detection Systems work to protect the network, a host server or an application. Each system requires a different approach to protect it which led to the evolution of Network IDS (NIDS), Host based IDS (HIDS) and Application IDS (AppIDS) systems. In reality, vendors soon realised there were benefits and drawbacks of each approach and current best-of-breed solutions, under the Unified Threat Management banner, will monitor all three areas using a single product.

For the sake of simplicity we will look at each of these areas in isolation to understand how IDS works in practice.

Network-Based Intrusion Detection Systems (NIDS)
A NIDS will often be an appliance solution and will be connected to a network segment with the job of monitoring network traffic as it passes up and down the wire. Packets are analysed to determine if there is any odd or out of character behaviour which may indicate an attack. An example may be a sudden influx of packets that appear to be related, which, in turn, could indicate an imminent denial of service attack. Other packet patterns could indicate a port scan in progress, where common ports are explored to see if common network services are running which could be exploited. We cover this in more detail later.

Generally, network-based intrusion detection systems can detect a lot more attacks than host-based intrusion detection systems, as they are closer to the network traffic and can see more of what is happening from minute to minute. The downside is they require additional, and often complicated, setup and maintenance.

The type of monitoring a NIDS undertakes depends on the network topology and the type of attack you are trying to test for. Often a system will be used to monitor a group of computers or a specific network segment. Before the widespread adoption of network switches, intrusion detection systems could be connected to a network hub and be guaranteed to be able to monitor all network traffic that passes through.

Unfortunately the downside was that hubs represented a security risk as, once compromised, it was easy to monitor all traffic that was being processed. Network switches create a more secure network as they create point-to-point links between their ports, but this in turn makes traffic interception far more difficult.

To overcome this, network intrusion detection systems are normally attached to a monitoring socket called the SPAN or switched port analysis port to capture passing traffic.

NIDS use a number of techniques to determine if an attack is underway or not.

Signature matching looks for attack patterns by comparing activity on the network with known signatures in their databases. This uses clever techniques to reassemble packets using protocol stack verification, where packets are examined for their structural integrity and application protocol verification where packets are examined for their specific use.

Protocol stack verification will monitor for malformed packets that do not meet the standard rules for the TCP/IP protocol. This can be useful in preventing denial of service attacks which often rely on the creation of malformed packets, which, in turn, can take advantages of weaknesses in the operating system or application.

With application protocol verification, protocols such as HTTP or FTP can be monitored to check for strange packet behaviour as some attacks can take the guise of valid protocol packets but in very large numbers.

Like most IT solutions, network intrusion detection systems have advantages and disadvantages:

Pros

Few devices can be used to monitor large networks
Little disruption when deployed as NIDS are passive devices

Cons

Some NIDS may be overwhelmed by the volume of network traffic
NIDS cannot analyse encrypted packets
NIDS have to use monitoring ports which are not present on all switches

If network intrusion detection is not suitable then there is an alternative—host based intrusion detection of HIDS.

Host-Based Intrusion Detection Systems (HIDS)
HIDS are host-based as they sit on a specific computer or server being monitored, rather than at the segment level found in NIDS. Their role is to monitor a host and detect if an intruder is attempting to make changes to system files or attempts to change specific monitored parts of the system, such as the Windows registry. HIDS use a change-based approach to security. Monitored files are initially checked as to their size, creation dates and any other measurable attribute. Any subsequent change to one of these files will create an alert to the systems administrator. Likewise system logs will be monitored to determine who is accessing which of these files and appropriate alerts raised. Often system logs themselves will be attacked by more sophisticated hackers trying to hide their activities. To overcome this most HIDS will create their own, well hidden, log files for monitoring.

HIDS will also monitor system directory files on a server and their own file structure in case there is an attempt to disable the HIDS as a precursor to a coordinated attack.

A major advantage of host-based intrusion detection is that it can often be configured to sit on a host computer and access information that would otherwise have been encrypted as it travelled over the network. How the network data actually reached the host computer is irrelevant, all HIDS worry about is the integrity of their host system.

To improve manageability, some HIDS can be deployed across multiple hosts and monitored from a central location with data being reported back to a single console. Criteria can be set up to determine what events trigger an alert and the way in which the alert should be communicated; normally via email or pager/SMS.

On the average host computer there may be thousands of files. Some of these will need active monitoring whilst others are not so important. During setup the administrator needs to determine which files are vital and therefore need constant monitoring; for example system files.

To assist with this, some HIDS allow files to be triaged using a red, yellow and green colour code. Red files are the most actively monitored, yellow files may be less so and green files not monitored by the HIDS. Other HIDS allow a numerical ranking of files according to their system importance.

Similar to network intrusion detection systems, host-based systems have advantages and disadvantages:

Pros

As HIDS work on a host computer they can be used to monitor previously encrypted traffic
The network architecture is irrelevant to a HIDS as they do not need to monitor ports on switches

Cons

HIDS often need more management than NIDS as they monitor individual files and logs, each of which are capable of raising an alarm
HIDS have a poorer ability to deal with some denial of service attacks
HIDS can consume large amounts of disk space with their monitoring services and logs

Application-Based Intrusion Detection Systems (AppIDS)
AppIDS take the notion of host-based intrusion detection one step further. Instead of monitoring an entire host system they will monitor a specific application that may be running on the host. During this monitoring, the AppIDS system will be looking for any out-of-course activity or other anomalous behaviour that could indicate an attack. An AppIDS can be tuned to monitor specific user activity and determine who is doing what on a system. Similar to a HIDS, AppIDS sit above any encryption that may be in place. Typically an AppIDS will monitor file reads and writes, configuration settings and the use of application execution space in the system memory.

Advantages and disadvantages of AppIDS include:

Pros

An AppIDS can be finely tuned to monitor specific application attributes
AppIDS sit above any encryption algorithms being used
AppIDS will work irrespective of the network topology

Cons

AppIDS are more susceptible to attack as they sit at the application layer
AppIDS can be fooled by some forms of spoofing and Trojan Horses

Common Intrusion Threats—Port Scanning
This is the computer security equivalent of a burglar checking to see what doors or windows may be left unlocked in your house. With the TCP/IP protocol there around 65,000 ports that can be used for services, applications or for programs to communicate on. The first 1024 TCP ports are referred to as the well-known ports and host services such as FTP and HTTP. A port scan is a process of automatically scanning each of a system’s ports to determine which ones may have been left open either deliberately or accidentally. This open port can then be probed further to see if there is an underlying weakness in the system waiting to be abused.

As can be seen port scans are a crude way of checking for vulnerabilities and are one of the first attack vectors that a decent Intrusion Detection System will prevent.

Common Intrusion Threats—Denial of Service (DoS) Attack
This is another reasonably crude way of attacking a computer system and can come in a number of forms, each of which is designed to slow down or stop a computer operating. In fact, the simplest Denial of Service attack may be someone locking your office door—if you can’t get in you can’t do any work.

One of the most common technical DoS attacks tries to prevent users accessing a system by overwhelming it with data. A ping flood is sometimes used to overload a computer with ping packets, which are used in legitimate circumstances to see if a computer is present on a network. If the sending computer has more bandwidth than the computer under attack then an unprotected computer is likely to collapse under the volume of ping packets.

A SYN flood uses a feature of TCP/IP to connect to computers on a network. A message is sent from one computer (often hijacked using malware) to the computer under attack asking for a connection. The computer under attack responds to say it is ready to communicate but never receives confirmation of the connection request. That way the computer under attack sits waiting with half opened connections. With lots of these hanging half connections the attacked system will be unable to respond to legitimate connection requests and be unable to work normally.

The good news is that both port scans and denial of service attacks can be prevented by using intrusion detection systems.

Strengths and Weaknesses of Using Intrusion Detection Systems
Few would doubt that adding an intrusion detection system to your security portfolio is probably a good idea, but there are some drawbacks as well as advantages.

An intrusion detection system is a very useful adjunct to good security practices and policies. There is no point in having an expensive IDS if you allow your users to download malware or copy files from USB thumb drives. Education is vital, as is leadership to demonstrate that the business takes security very seriously. Any violations of a well communicated policy need to be taken seriously. An IDS is a useful way of creating a security baseline and then detecting any deviations from that which may indicate an attack. In this way an IDS will allow you to act before any damage is done and prevent loss to the business.

On the downside, installing and configuring any form of IDS will take time and effort. During the learning phase there may quite well be a lot of false positives and false negatives as the administration team get to fully understand what the system can do and how to tune it. No IDS can 100% guarantee that all attackers will be deterred and, in fact, a determined, educated attacker will probably succeed whether you have an IDS or not. That said, a very large percentage of untargeted attacks will be prevented with the most basic intrusion detection system.

In Summary
IT Security is a tough gig. It needs administrators to work with developers, database administrators and the business to get the correct balance of security across the organisation. The simplest option can at times appear to be the easiest—unplug your systems from the internet and you will no longer need to worry about intrusion detection systems. We all know the reality is very different and in today’s modern, connected world internet connectivity is mission critical for most businesses. To that end it is important that we get security right from the start. Putting in place an intrusion detection system, as part of your unified threat management strategy, is now as vital as installing Office productivity software on user’s PCs. We can no longer rely on receptionists with shrill voices to protect our organisations from intruders.