Intrusion Detection Techniques

4.IDS Taxonomy

A distributed intrusion detection system is one where data is collected and analyzed in multiple host, as opposed to a centralized

detection systems may use host- or network-based data collection methods, or most likely a combination of the two.

--Audit information analysis can be done generally in two modes. Intrusion detection process can run continuously, also called in real-time. The term "real-time" indicates not more than a fact that IDS reacts to an intrusion "quick enough". Intrusion detection process also can be run periodically

components:

used to determine whether an intrusion has taken place. These

sources can be drawn from different levels of the system, with

network, host, and application monitoring most common.

organizes and makes sense of the events derived from the

information sources, deciding when those events indicate that

intrusions are occurring or have already taken place. The most

common analysis approaches are misuse detection and anomaly

detection.
Response – the set of actions that the system takes once it detects

intrusions. These are typically grouped into active and passive

measures, with active measures involving some automated

intervention on the part of the system, and passive measures

involving reporting IDS findings to humans, who are then expected

to take action based on those reports.

IDS are arranged with respect to each other.

According to one study [Axelsson, 1998], an IDS can be thought of as consisting of an Audit Collection/Storage Unit, Processing Unit and an Alarm/Response unit.

The Audit Collection/Storage Unit collects data that is to be analyzed for signs of intrusion.

The Processing Unit analyzes the data received from Audit collection/Storage Unit to find the intrusions.

Alarm/Response Unit triggers an alarm on detecting an intrusion and it may execute defensive action too.

7.Information Sources or targets:
The most common way to classify IDSs is to group them by information source. Some IDSs analyze network packets, captured from network backbones or LAN segments, to find attackers. Other IDSs analyze

information sources generated by the operating system or application

software for signs of intrusion.

7.1.Network-Based IDSs(NIDS):

The majority of commercial intrusion detection systems are networkbased.

These IDSs detect attacks by capturing and analyzing network packets. Listening on a network segment or switch, one network-based IDS can monitor the network traffic affecting multiple hosts that are connected to the network segment, thereby protecting those hosts.Network-based IDSs often consist of a set of single-purpose sensors or hosts placed at various points in a network. These units monitor network traffic, performing local analysis of that traffic and reporting attacks to a central management console. As the sensors are limited to running the IDS, they can be more easily secured against attack. Many of these sensors are designed to run in “stealth” mode, in order to make it more difficult for an attacker to determine their presence and location.
Advantages of Network-Based IDSs:
A few well-placed network-based IDSs can monitor a large network.

The deployment of network-based IDSs has little impact upon an existing network. Network-based IDSs are usually passive devices that listen on a network wire without interfering with the normal operation of a network. Thus, it is usually easy to retrofit a network to include network-based IDSs with minimal effort.

Network-based IDSs can be made very secure against attack and

even made invisible to many attackers.

Many of the advantages of network-based IDSs don’t apply to more modern switch-based networks. Switches subdivide networks into many small segments (usually one fast Ethernet wire per host) and provide dedicated links between hosts serviced by the same switch. Most switches do not provide universal monitoring ports and this limits the monitoring range of a network-based IDS sensor to a single host. Even when switches provide such monitoring ports, often the single port cannot mirror all traffic traversing the switch.

Network-based IDSs cannot analyze encrypted information.

Most network-based IDSs cannot tell whether or not an attack was successful; they can only discern that an attack was initiated. This means that after a network-based IDS detects an attack, administrators must manually investigate each attacked host to determine whether it was indeed penetrated.

Some network-based IDSs have problems dealing with network-based

attacks that involve fragmenting packets. These malformed packets cause the IDSs to become unstable and crash.

Individual computer system. This vantage point allows host-based IDSs to analyze activities with great reliability and precision, determining exactly which processes and users are involved in a particular attack on the operating system. Furthermore, unlike network-based IDSs, host-based IDSs can “see” the outcome of an attempted attack, as they can directly access and monitor the data files and system processes usually targeted by attacks. Host-based IDSs normally utilize information sources of two types, operating system audit trails, and system logs. Operating system audit trails are usually generated at the innermost (kernel) level of the operating system, and are therefore more detailed and better protected than system logs. However, system logs are much less obtuse and much smaller than audit trails, and are furthermore far easier to comprehend.

Host-based IDSs can often operate in an environment in which network traffic is encrypted, when the host-based information sources are generated before data is encrypted and/or after the data is decrypted at the destination host

Host-based IDSs are unaffected by switched networks.

When Host-based IDSs operate on OS audit trails, they can help detect Trojan Horse or other attacks that involve software integrity breaches. These appear as inconsistencies in process execution.

Since at least the information sources (and sometimes part of the analysis engines) for host-based IDSs reside on the host targeted by attacks, the IDS may be attacked and disabled as part of the attack.

Host-based IDSs are not well suited for detecting network scans or other such surveillance that targets an entire network, because the IDS only sees those network packets received by its host.

Host-based IDSs can be disabled by certain denial-of-service attacks.

When host-based IDSs use operating system audit trails as an information source, the amount of information can be immense,requiring additional local storage on the system.

Host-based IDSs use the computing resources of the hosts they are monitoring, therefore inflicting a performance cost on the monitored systems.

and application, which often allows them to trace unauthorized activity to individual users.

Application-based IDSs can often work in encrypted environments, since they interface with the application at transaction endpoints, where information is presented to users in unencrypted form.
Disadvantages:
Application-based IDSs may be more vulnerable than host-based IDSs to attacks as the applications logs are not as well-protected as the operating system audit trails used for host-based IDSs.

As Application-based IDSs often monitor events at the user level of abstraction, they usually cannot detect Trojan Horse or other such software tampering attacks. Therefore, it is advisable to use an Application-based IDS in combination with Host-based and/or Network-based IDSs.

.

8.IDS Analysis:
There are two primary approaches to analyzing events to detect attacks:

misuse detection and anomaly detection. Misuse detection, in which the

analysis targets something known to be “bad”, is the technique used by most

commercial systems. Anomaly detection, in which the analysis looks for

abnormal patterns of activity, has been, and continues to be, the subject of a

great deal of research. Anomaly detection is used in limited form by a

number of IDSs. There are strengths and weaknesses associated with each

approach, and it appears that the most effective IDSs use mostly misuse

detection methods with a smattering of anomaly detection components.

There is also another technique which combines the two approaches.
8.1.Misuse Detection:
Misuse detectors analyze system activity, looking for events or sets of

events that match a predefined pattern of events that describe a known

attack. As the patterns corresponding to known attacks are called

signatures, misuse detection is sometimes called “signature-based

detection.” The most common form of misuse detection used in

commercial products specifies each pattern of events corresponding to an

attack as a separate signature. However, there are more sophisticated

approaches to doing misuse detection (called “state-based” analysis

techniques) that can leverage a single signature to detect groups of

attacks.

Misuse detectors can quickly and reliably diagnose the use of a specific attack tool or technique. This can help security managers prioritize corrective measures.

Misuse detectors can allow system managers, regardless of their level of security expertise, to track security problems on their systems, initiating incident handling procedures.
Disadvantages:

Misuse detectors can only detect those attacks they know about –therefore they must be constantly updated with signatures of new attacks.

Many misuse detectors are designed to use tightly defined signatures that prevent them from detecting variants of common attacks. State-based misuse detectors can overcome this limitation, but are not commonly used in commercial IDSs.
8.2.Anomaly Detection:

Anomaly detectors identify abnormal unusual behavior (anomalies) on a

host or network. They function on the assumption that attacks are

different from “normal” (legitimate) activity and can therefore be

detected by systems that identify these differences. Anomaly detectors

construct profiles representing normal behavior of users, hosts, or

network connections. These profiles are constructed from historical data

collected over a period of normal operation. The detectors then collect

event data and use a variety of measures to determine when monitored

activity deviates from the norm.

Figure:-

* The measures and techniques used in anomaly detection include:

Threshold detection, Statistical measures, Rule-based measures

Advantages:
IDSs based on anomaly detection detect unusual behavior and thus have the ability to detect symptoms of attacks without specific knowledge of details.

Anomaly detectors can produce information that can in turn be used to define signatures for misuse detectors.

Disadvantages:
Anomaly detection approaches usually produce a large number of false alarms due to the unpredictable behaviors of users and networks.

Anomaly detection approaches often require extensive “training sets” of system event records in order to characterize normal behavior patterns.

8.3.EXAMPLE:A TEXT BASED TECHNIQUES WITH A BWC TECHNIQUE:

The starting point of the method is the observation that any normal execution of a process follows a pattern and hence the normal behavior of a process can be profiled by a set of sequences of system calls. Any deviation in this pattern of system calls is termed an intrusion in the framework

of anomaly-based IDS. The problem of intrusion detection thus boils down to detecting anomalous sequence of system calls, which are measurably different from the normal behavior. We propose a new scheme in which we measure the similarity between two processes using a metric that considers two factors - occurrence of system calls shared by the two said processes

and the frequency of all system calls in the processes. Due to the way it is constructed, we term this similarity metric Binary Weighted Cosine (BWC) metric.

8.3.1.FEATURE VECTOR AND SIMILARITY MEASURE:

Let S (say, Card(S) = m) be a set of system calls made by all the processes under normal execution. From all the normal processes a matrix A = [aij] is formed, where aij denotes the frequency of ith system call in the jth process. We also form a matrix B = [bij] where, bij = 1, if ith system calls is present in the jth process, otherwise bij = 0. Thus the binary representation of

process P, namely Pbj, is defined by the m-vector Pbj = [0,1]m as a column in B. For example,

Let S = {access audit chdir close creat exit fork ioctl}.

Let the two normal processes be

P1 = access close ioctl access exit

P2 = ioctl audit chdir chdir access

Then we have

The rows of A (and B) correspond to the elements of S in the same order and columns of A (and B) correspond to processes P1 and P2. Thus the first entry in A is calculated by counting the frequency of system call access in the process P1 that is 2. Similarly the first entry of the second column of A is calculated by counting the frequency of the system calls access in the process P2 which is 1, and so on. Similarly the first entry of the first column of B is 1 because the system call access is present in P1 whereas the second entry is 0, which shows that the system call audit is absent in P1.

8.3.2.BINARY SIMILARITY MEASURE:

We define a similarity score between any two processes Pbi and Pbj (ith and jth columns of B) as follows:

where the summation runs over n, which is a subscript on the elements of the processes Pbi and Pbj and m is the length of each process vector.

It may be noticed that 0 ≤ μ ≤ 1. The value of μ increases when there are more shared system calls between the two processes (due to the numerator) and value ofμ decreases when the number of system calls, not shared by both the processes, is more than the shared ones (due to the denominator) in Pbi and Pbj.

8.3.3.FREQUENCY SIMILARITY MEASURE:
Another similarity score, known as cosine similarity measure between the processes Pi and Pj (ith and jth columns of A) is defined as follows:


Now we define our new similarity measure, termed as Binary Weighted Cosine (BWC) metric, Sim(Pi, Pj) as follows:


The motive behind multiplying μ and λ is that measures the similarity based on the frequency and is the weight associated with Pi and Pj. In other words, tunes the similarity score according to the number of similar and dissimilar system calls between the two processes. Therefore, the similarity measure Sim(Pi, Pj) takes frequency and the number of shared system calls into consideration while calculating similarity between two process.
8.3.4. PROPOSED SCHEME:

As discussed , the matrices A=[aij] and B=[bij] are constructed using normal processes and the set S. For every new process P, if P contains a system call that is not in S, the process P is classified as abnormal; if not, it is first converted into a vector for further processing. The binary equivalent P_b of this vector is then calculated. Next, the similarity score is calculated for every normal vector Pj by using equation (2). If = 1, P is classified as normal. Otherwise, using equations , the values of and Sim(P, Pj) are calculated. Values of Sim(P, Pj) are sorted in descending order and the k nearest neighbors (first k highest values) are chosen. We calculate the average value (Avg_Sim) of the k nearest neighbors. The kNN classifier categorizes the new process P as either normal or abnormal according to the rule given below.

If Avg_Sim > Sim_Threshold, classify P as normal, otherwise P is

Abnormal where Sim_Threshold is a predefined threshold value for similarity measurement. The pseudocode for the proposed scheme is shown in Figure 1.

Given a set of processes and system calls S, form the matrices A=[aij] &

B=[bij]

for each process P in the test data do

if P has some system calls which does not belongs to S then

P is abnormal; exit;

else then

for each process Aj in the training data A do

calculate Sim(P, Aj);

if Sim(P, Aj) equals 1.0 then

P is normal; exit;

find first k highest values of Sim(P, Aj);

calculate Avg_Sim for k nearest neighbors so obtained;

calculate Avg_Dist for k nearest neighbors;

if Avg_Sim is greater than Sim_Threshold then

P is normal;

else then

P is abnormal;

Figure 1. Pseudo code for the proposed scheme

.
8.4.Specification-based detection:

They distinguished between normal and intrusive behaviour by monitoring the traces of system calls of the target processes. A specification that

models the desired behaviour of a process tells the IDS whether the actual observed trace is part of an attack or not. With this approach, they attempt to combine the advantages of misuse and anomaly detection. It should reach the accuracy of a misuse detection system and have the ability to deal with

future attacks of anomaly detection. Their systems managed the detection by inspecting log files.


Advantages:

More or less the same as for misuse detection. However these systems manage to detect some types/classes of novel attacks. Additionally, they are more resistant against subtle changes in attacks.

Disadvantages:
Usually for every program that is monitored, a specification has to be designed. Furthermore, the modelling process can be regarded as more difficult than the design of patterns for misuse detection systems. Additionally some classes of attacks are not detectable at all.

Their systems managed the detection by inspecting log files.

9. Tools that Complement IDSs:

Several tools exist that complement IDSs and are often labeled as intrusion detection products by vendors since they perform similar functions. This section discusses four of these tools, Vulnerability Analysis Systems, File Integrity Checkers, Honey Pots, and Padded Cells, and describes how they can enhance an organization’s intrusion detection capability.

10. Deploying IDSs:

Intrusion detection technology is a necessary addition to every large organization’s computer network security infrastructure. However, given the deficiencies of today’s intrusion detection products, and the limited security skill level of many system administrators, an effective IDS deployment requires careful planning, preparation, prototyping, testing, and specialized training.

10.1.Deploying Network-Based IDSs:
One question that arises when deploying network-based IDSs is where to locate the system sensors. There are many options for placing a network-based IDS with different advantages associated with each location:



Figure – Locations of Network-based IDS sensors
10.1.1.Location: Behind each external firewall, in the network DMZ

(See Figure – Location 1)

Advantages:

Sees attacks, originating from the outside world, that penetrate the network’s perimeter defenses.

Highlights problems with the network firewall policy or performance

Sees attacks that might target the web server or ftp server, which commonly reside in this DMZ

Even if the incoming attack is not recognized, the IDS can sometimes

recognize the outgoing traffic that results from the compromised server

10.1.2.Location: Outside an external firewall

(See Figure – Location 2)

Advantages:

Documents number of attacks originating on the Internet that target the

network.

Documents types of attacks originating on the Internet that target the network

10.1.3. Location: On major network backbones

(See Figure – Location 3)

Advantages:

Monitors a large amount of a network’s traffic, thus increasing the possibility of spotting attacks.

Detects unauthorized activity by authorized users within the organization’s

security perimeter.

10.1.4.Location: On critical subnets

(See Figure – Location 4)

Advantages:

Detects attacks targeting critical systems and resources.

Allows focusing of limited resources to the network assets considered of

greatest value

.

10.2.Deploying Host-Based IDSs:

Once network-based IDSs are in place and operational, the addition of host-based IDSs can offer enhanced levels of protection for your systems. However, installing host-based IDSs on every host in the enterprise can be extremely time-consuming, as each IDS has to be installed and configured for each specific host. Therefore, we recommend that organizations first install host-based IDSs on critical servers. This will decrease overall deployment costs and allow novice personnel to focus on alarms generated from the most important hosts. Once the operation of host-based IDSs is routine, more security-conscious organizations may consider installing host-based IDSs on the majority of their hosts. In this case, purchase host-based systems that have centralized management and reporting functions. These features will significantly reduce the complexity of managing alerts from a large set of hosts. Another consideration when using host-based IDSs is that of allowing operators to become familiar with the IDS in a sheltered, but active environment. Much of the effectiveness of any IDS, but particularly a host-based IDS depends on the operator’s

ability to discern between true and false alarms. Over a period of time, an operator, working with an IDS in a particular environment, will gain a sense of what is normal for that environment, as monitored by the IDS.

It is also important (as host-based IDSs are often not continuously attended by operators) to establish a schedule for checking the results of the IDS. If this is not done, the risk that an adversary will tamper with the IDS in the course of an attack increases..

11.Strengths and Limitations of IDSs:
Although Intrusion Detection Systems are a valuable addition to an organization’s security infrastructure, there are things they do well, and other things they do not do well. As you plan the security strategy for your organization’s systems, it is important for you to understand what IDSs should be trusted to do and what goals might be better served by other types of security mechanisms.
11.1. Strengths of Intrusion Detection Systems
Intrusion detection systems perform the following functions well:

Monitoring and analysis of system events and user behaviors

Testing the security states of system configurations

Base lining the security state of a system, then tracking any changes to that

Baseline

Recognizing patterns of system events that correspond to known attacks

Recognizing patterns of activity that statistically vary from normal activity

Managing operating system audit and logging mechanisms and the data they generate.

Alerting appropriate staff by appropriate means when attacks are detected.

Measuring enforcement of security policies encoded in the analysis engine

Providing default information security policies

Allowing non-security experts to perform important security monitoring

Functions.
11.2. Limitations of Intrusion Detection Systems
Intrusion detection systems cannot perform the following functions:

Compensating for weak or missing security mechanisms in the protection

Infrastructure. Such mechanisms include firewalls, identification and

Authentication, link encryption, access control mechanisms, and virus

Detection and eradication.

Instantaneously detecting, reporting, and responding to an attack, when there is a heavy network or processing load.

Detecting newly published attacks or variants of existing attacks.

Effectively responding to attacks launched by sophisticated attackers

Automatically investigating attacks without human intervention.

Resisting attacks that are intended to defeat or circumvent them

Compensating for problems with the fidelity of information sources

Dealing effectively with switched networks.

12.Challenges with IDS Techniques:

• There exist over 100 Intrusion Detection Systems

– Both open source and commercial

– Can be network based or host based or combination

• Main problem

– Too many false positives

– System administrators tend to ignore warnings after a while

– Difficult to determine a good IDS policy

• Other problems

– Protecting the IDS itself against attack

13.Conclusion:
IDSs are here to stay, with billion dollar firms supporting the development of commercial security products and driving hundreds of millions in annual sales. However, they remain difficult to configure and operate and often can’t be effectively used by the very novice security personnel who need to benefit from them most. Due to the nationwide shortage of experienced security experts, many novices are assigned to deal with the IDSs that protect our nation’s computer systems and networks. Our intention, in writing this document, is to help those who would take on this task. We hope that this publication, in providing actionable information and advice on the topics,serves to acquaint novices with the world of IDSs and computer attacks. The information provided in this bulletin is by no means complete and we recommend further reading andformal training before one takes on the task of configuring and using an intrusion detection system.


REFERENCE

1. 
   ID using Text processing with BWC metric:
   

Rawat,Pujari,Gulati,Vemuri.

2. 
   ID systems:Rebecca Bace,Peter mall.
   
3. 
   IDS with Snort:Rafeeq ur Rehman.
   
4. 
   Datamining for ID:Shamhu,Pei,Upadhyaya,Farooq,Govindaraju.
   
5. 
   Using text categorization techniques for ID:Liao,vemuri.
   
6. 
   Undermining an anomaly-based detection systems:Tan,Killourhy,Maxion.
   
7. 
   An application of pattern maching in ID:Kumar,Spafford.
   
8. 
   An introduction to ID:A.Sundaram.
   
9. 
   Using CSP to detect insertion and evasion possibilities within ID area:Rohrmair and Lowa.
   
10. 
    Anomaly detection using call stack information:Feng,Kolesnikov,Fogla,Lee,Gong.
    
11. 
    Realtime user identification:Steven Eschrich.
    

Directory: research
research -> Charter School Enrollment Data Annual Report
research -> Lyra Viol Composers
research -> Geographies of Race and Food
research -> Author (s) title
research -> The Media and Intercollegiate Sports
research -> Project Title: Research and Development for goes-r risk Reduction Principal Investigator
research -> Why Everyday Life Matters: Class, Community and Making Life Livable Les Back
research -> Ecocritique and the Materialities of Animation

Download 93.19 Kb.

Share with your friends: