Learning Resource

Intrusion Detection Systems (IDS)

Introduction to Intrusion Detection Systems

An IDS (intrusion detection system) is the term for a mechanism which quietly listens to network traffic in order to detect abnormal or suspicious activity, thereby reducing the risk of intrusion.

There are two distinct major families of IDSs:

  • The NIDS group (network-based intrusion detection system), which handle security at the network level.
  • The HIDS group (host-based intrusion detection system), which handle security at the host level.

An NIDS needs dedicated hardware and forms a system that can check packets traveling on one or more network lines in order to find out if any malicious or abnormal activity has taken place. The NIDS puts one or more of the dedicated system's network adapters into promiscuous mode. This is a sort of "stealth" mode in which they have no IP address. They no longer have a protocol stack assigned to them, either. It is common to find multiple IDSs on different parts of the network, and particularly to place a probe outside the network in order to study attempted attacks, as well as an internal probe to analyze requests which either passed through the firewall or were made from the inside.

Diagram of an N-IDS on a network

The HIDS resides on a particular host, and its software therefore covers a broad range of operating systems, such as Windows, Solaris, Linux, HP-UX, Aix, etc.

The HIDS acts as a daemon or standard service on a host system. Traditionally, the HIDS analyzes particular information stored in logs (such as syslogs, messages, lastlog, and wtmp) and also captures network packets entering/leaving the host in order to check for signs of intrusion (such as denial-of-service attacks, backdoors, Trojan horses, unauthorized access attempts, malicious code being run, or buffer overrun attacks).

Detection Techniques

Network traffic is generally (over the Internet, in any case) made of IP datagrams. An NIDS can capture packets as they travel over the physical connections to which it is attached. An NIDS contains a TCP/IP stack that reassembles IP datagrams and TCP connections. It can apply the following techniques for recognizing intrusions:

  1. Checking the protocol stack: Some forms of intrusion, such as "ping of death" and "TCP stealth scanning" use IP, TCP, UDP, and ICMP protocol violations in order to attack a machine. A simple protocol check can reveal invalid packets and detect this commonly-used tactic.
  2. Checking application layer protocols: Some forms of intrusion employ invalid protocol behavior, such as "WinNuke," which uses invalid NetBIOS data (adding out-of-band data). In order to effectively detect such intrusions, an NIDS must have implemented a variety of application layer protocols, such as NetBIOS, TCP/IP, etc.
    • This technique is fast (the NIDS doesn't need to sift through the entire signature database for particular byte sequences) and eliminates some false alarms, and is therefore more efficient. For example, by analyzing protocols, an NIDS can tell apart a "Back Orifice PING" (low danger) and a "Back Orifice COMPROMISE" (high danger).
  3. Recognizing "pattern matching" attacks: This intrusion recognition technique is the oldest NIDS analysis method, and still a common one.
    • It involves identifying an intrusion just by examining a packet and recognizing, within a series of bytes, a sequence which corresponds to a specific signature. For example, searching for the string of characters "cgi-bin/phf" indicates an attempt at exploiting a hole in the CGI script "phf". This method is also used as a supplement to filters on IP addresses, destinations used by connections and source and/or destination ports. This recognition method can even be refined by combining it with a succession or combination of TCP flags.
    • This tactic is widespread on "Network Grep" NIDSs, which are based around capturing raw packets on a monitored connection, and comparing them using a "regular expression" parser, which will attempt to match sequences in the signature base byte-for-byte with the content of the captured packet.
    • The primary advantage of this technique lies with how easy it is to update, and of course in the large quantity of signatures found in the NIDS base. However, quantity doesn't necessarily mean quality. For example, the 8 bytes “CE63D1D2 16E713CFâ€, when placed at the start of a UDP data transfer, indicate Back Orifice traffic with a default password. Even if 80 percent of intrusions use the default password, 20 percent will use personalized passwords and won't necessarily be recognized by the NIDS. For example, if the password is changed to "evade," then the series of bytes will become "8E42A52C 0666BC4A," which automatically protects it from being caught by the NIDS. The technique also unavoidably leads to a large number of false alarms and false positives.

There are other methods for detecting and reporting intrusions, such as stateful pattern matching, and/or auditing dangerous or abnormal network traffic.

In conclusion, a perfect NIDS is a system which uses the best parts of all the techniques mentioned above.

What IDSs do

The primary methods used by NIDSs to report and block intrusions are:

  • Reconfiguring third-party devices (firewall or ACLs on routers): Command sent by the NIDS to a third-party device (like a packet filter or firewall) to immediately reconfigure itself so as to block an intrusion. This reconfiguration is made possible by sending data explaining the alert (in the packet header(s)).
  • Sending an SNMP trap to a third-party hypervisor: Sending an alert (and details on the data involved) in the form of an SNMP datagram to a third-party console like HP OpenView, Tivoli, Cabletron Spectrum, etc.
  • Sending an e-mail to one or more users: Sending an e-mail to one or more inboxes to report a serious intrusion.
  • Logging the attack: Saving the details of the alert in a central database, including such information as the timestamp, IP address of the intruder, IP address of the target, the protocol used, and the payload).
  • Saving suspicious packets: Saving all raw network packets captured, and/or only the packets which triggered an alert.
  • Opening an application: Launching an outside program to perform a specific action (such as sending an SMS text message, or playing a sound to indicate an alert)
  • Sending a "ResetKill": Constructing a TCP FIN packet to force a connection to end (only valid for intrusion techniques that use the TCP transport protocol).
  • Visual notification of an alert: Displaying an alert on one or more management console(s).

IDS challenges

The computing media is starting to use the term IPS (intrusion prevention system) more and more, as a replacement for "traditional" IDSs or to make a distinction between them.

The IPS is a prevention/protection system for guarding against intrusions, and not just recognizing and reporting them like most IDSs do. There are two main characteristics which distinguish a (network) IDS from a (network) IPS:

  • The IPS sits inline on the IPS network and does not just passively listen to the network like an IDS (traditionally placed as a sniffer on the network).
  • An IPS has the ability to immediately block intrusions, no matter what transport protocol is used and without reconfiguring a third-party device, which means that the IPS can filter and block packets in native mode (using techniques such as dropping a connection, dropping offending packets, or blocking an intruder).

Licenses and Attributions

Intrusion detection systems (IDS) is available under a Creative Commons Attribution-NonCommerical-ShareAlike 3.0 Unported license. UMGC has modified this work and it is available under the original license.