Active and Passive Scanning Intrusion Detection in Web Applications

Intrusion Detection Systems (IDS) are defensive solutions for threat detection. There are multiple types of IDS and are widely used in both network traffic and host intrusion detection. For web applications, IDS still present ineffectiveness when monitoring and acting against potential threats. This work presents how passive and active network scanning against a web application behaves when an IDS is protecting it. It presents which events triggered alerts and their relevance to web application security.

The internet overgrowth poses a harsh challenge to security specialists as the surface of attacks allows cybercriminals to find and exploit flaws and challenge everyday technology. Due to these events, various defense solutions need to be continuously improved.

These solutions prevail when an attack is well-known to the defender, as the attack has been cautiously analyzed before. Usually, knowledge bases are used to store such information and allow it to be accessed publicly by the cyber defense community. Therefore, many tools to find standardized attacks are improved daily to meet the community’s security requirements.

Intrusion Detection Systems (IDS) are often used as a security measure to monitor and protect against malicious activities or policy violations. Usually, these activities are then reported to a centralized security information and event management system (SIEM).

There are two most common types of IDS, network intrusion detection systems (NIDS) and host intrusion detection systems (HIDS). Network IDS monitor and analyze network traffic for suspicious behaviour and real threats in the content of all packets moving across the network. Host IDS monitors and analyzes system configuration and application activity across the network, looking for permissions unexpected changes, files being overwritten, unauthorized access and so on. Sometimes the ideal scenario is to incorporate both HIDS and NIDS since they complement each other.

This work analyzes the behaviour of IDS against automated and manual scans while monitoring a vulnerable web application. This work is organized into five different sections. The first section introduces the context and the objectives. The second section provides insight on related work on intrusion detection systems. The third presents some of the existing IDS technology. The fourth section presents the architecture of a virtual testing laboratory containing a vulnerable web application running on a virtual machine and the alerts generated by both the IDS combined. The fifth section provides a deeper analysis of such results. Finally, the sixth and final section presents the conclusions and explores the potential these intrusion detection systems have.

Related Work and Intrusion Detection Models

Signature vs. Anomaly-Based Intrusion Detection Systems

Signature-based

Signature-based detection systems identify known threats. Such systems operate based on attack patterns and sequences that match known threats. Signature-based detection uses a list of indicators of compromise (IOCs) containing such patterns and sequences. The main limitation of these systems is the fact that they rely on known attack signatures to trigger alerts, meaning novel attacks cannot be detected.

Anomaly-based

Anomaly-based intrusion detection systems analyze malicious or unusual patterns of behaviour and build a statistical model describing the normal network traffic. This method is based on defining the network behaviour, and matching it with the accepted one or else it triggers the alert for such events. There is an intrinsic early phase of working and learning the predefined behaviour, but when this process is complete, it may offer the best protection solution for a system as the goal of defence is concise and well-defined.

Policy-based

Policy-based techniques establish boundaries between the allowed and not allowed events. Although this approach solves major problems such as detecting unknown attacks and noticing unusual behaviour not respecting policies, the stored policy rules are defined by a security specialist and it may be the major drawback of this method alongside having consistent and logically correct policies.

Hybrid systems

Hybrid systems combine different intrusion detection approaches. These systems provide better results as they combine multiple detection methodologies overcoming the limitations of individual techniques.

State-of-the-Art Technology & Laboratory

IDS in Action

To provide a better understanding of how intrusion detection systems can be useful to protect web applications against threats, a simple virtual laboratory was deployed. The stack consists of a virtual machine running Docker where two containers were deployed. (i) One running the OWASP JuiceShop, a modern and sophisticated vulnerable web application, which is usually used in security training; (ii) A dockerized Wazuh version, used as the Wazuh manager, later on, to receive information from agents. Then, a Wazuh agent was deployed on the host machine to monitor possible intrusions to the host. And finally, Suricata was also deployed to monitor the network interface of the host to collect and analyze traffic data, so alerts could be triggered when launching automated and manual scans against the vulnerable web application. Another virtual machine running Kali Linux was used to launch the OWASP Zed Attack Proxy (ZAP) automated scans and the port scanning described in the evaluation section. The virtual laboratory is represented in the following figure.

Wazuh

Wazuh provides powerful out of the box analysis and detection tools. To visualize alerts and detections, pre-generated dashboards for agents on hosts and security events are presented in the main interface.

Suricata Integration

Wazuh is a great HIDS. It provides rule-based analysis of events and performs great insight into the security of its hosts. However, having a NIDS working alongside Wazuh allows network traffic to be successfully inspected. This laboratory uses Suricata as the NIDS complement to the HIDS. Suricata’s installation is pretty straightforward and easily integrated with Wazuh’s agents. The following figure shows an example of Suricata’s detections and their following representation in Wazuh’s data visualization dashboards.

Nmap.org, THE network mapper

Nmap is a free and open-source utility for network discovery and security auditing. Nmap successfully discovers what hosts are available in a network, as what services they have running.

Different Scan Strategies

One of the main objectives of Nmap was always to have a fast and reliable scan mechanism, it has also been rewritten before to optimize performance and accuracy. Besides the different predefined scanning algorithms, the goal was to understand how we could evade IDS alerts while scanning with different timings and pokes around the network. To test this and how well Suricata IDS performs against different crafted network scans, Nmap timing templates came to good use as different test parameters.

According to Nmap’s documentation, timing templates allow users to specify how aggressive their scans should be. Currently, there are six templates to carefully adjust these scans. Paranoid (0) and sneaky (1) modes are usually used for IDS evasion, adding increased duration to each scan.

root@kali:~# 
nmap -p3000 -T0 192.168.1.154
Nmap scan report for unknown782b464.lan 
(192.168.1.154)
PORT     STATE SERVICE
3000/tcp open  ppp
MAC Address: 08:00:27:95:48:4B
(Oracle VirtualBox virtual NIC)

Nmap done: 1 IP address (1 host up) 
Scanned in 600.28 seconds

As we can see, the scan took about 10 minutes (600 seconds) to finish. During this time no alerts were triggered by the Suricata IDS.

The second test with the timing scan set to sneaky (1):

root@kali:~# 
nmap -p3000 -T1 192.168.1.154
Nmap scan report for ubuntu2004.lan 
(192.168.1.154)
PORT     STATE SERVICE
3000/tcp open  ppp
MAC Address: 08:00:27:95:48:4B 
(Oracle VirtualBox virtual NIC)

Nmap done: 1 IP address (1 host up) 
Scanned in 30.34 seconds

The execution took 30 seconds, which is a lot faster than the first scan. One of the major differences found is that the hostname was not retrieved during the first scan.

OWASP Zed Attack Proxy

Using the traditional ZAP spider, an active scan was executed. The ZAP mode used was the attack mode with the following analysers represented in the following figure.

According to the generated report for the last 25 minutes before the scan execution, Suricata triggered 33 security alerts, 28 of which are forbidden access related events.

Notes

Intrusion Detection Systems safeguard web applications by alerting and evicting potential threats. Most of the IDS need to have methodically preconfigured rules that suit the host they are protecting for their needs, as there are many types of IDS and their usage should be combined so each of these IDS fill the gaps others may have. A well-timed passive scan against a host will not be detected by the IDS used in this virtual laboratory. On the other hand, active scans will trigger alerts, but if the rule set for the IDS that is protecting the host isn’t concise, false alerts will flood it and real threats might be able to go unnoticed and potentially cause a system failure. With this work, it was also noticed that most of the triggered alerts were against crawlers with unauthorized access on sub-paths of the web application.

Intrusion detection in web applications remains a challenge as web applications frequently interact with users, so the IDS should be able to understand what these interactions are and how they behave during each session.