Slovak University of Technology in Bratislava Faculty of Informatics and Information Technologies FIIT-100241-102918 Jakub Gašparín Expanding system focused on monitoring safety events Bachelor thesis Study program: Informatics Degree Course: 9.2.1 Informatics My institute: Institute of Informatics and Software Engineering, FIIT STU, Bratislava Supervisor: Ing. Ján Laštinec, PhD. May 2024
iii
I hereby affirm that I worked independently on this work, with the guidance and support of my advisor and through extensive reference to the relevant and cited literature. In Bratisla, iv
Anotácia Slovenská Technická Univerzita Fakulta informatiky a informačných technológií Štúdijný program: Informatika Autor: Jakub Gašparín Bakalárska práca: Rozšírenie systému monitorovania bezpečnostných udalostí Vedúci bakalárskej práce: Ing. Ján Laštinec, PhD. Máj 2024 Kyberbezpečnostný monitoring nám pomáha detekovať potencionálne hrozby a zraniteľnosti v reálnom čase. Zároveň nám pomíha chrániť citlivé dáta a integritu monitorovaných systémov a sietí. Cieľom tejto práce bolo zanalyzovať rôzne defenzívne systémy, ktoré sa bežne používajú v modernej kyberbezpečnosti, porovnať ich s monitorovacím systémom SecMon a následne navrhnúť a implementovať vylepšenie monitorovacieho nástroja SecMon. Keďže existujú viaceré, rôzne funkcie a technológie, táto práca sa zamerala na Ochranu do hĺbky (Defense in Depth, alebo skrátene DiD) a sys- témy SIEM ako jej základy na hlbšiu analýzu nástroja SecMon návrhy na jeho vylepšenie. Koncept DiD bol analyzovaní a opísaný tak, aby bolo jasné, prečo sa využíva. Podobne som sa zameral na SIEM systémy, hlavne na pochope- nie toho, čo to SIEM je, na čo sa používa, aké sú hlavné benefity SIEM systémov a ako by mohli byť vylepšené. Následne som spravil hlbokú analýzu archi- tektúry SecMon, vysvetlil som ako funguje, aké sú jeho benefity a nedostatky a ako by sa dal vylepšiť. V implementácii som sa zameral na napísanie troch pravidiel na spoluprácu s IDS Suricata. Vytvoril som 1 normalizačné pravidlo ktoré normalizuje všetky logy zo Suricaty, vrátane tých vytvorené z vlastných Suricata pravidiel. Potom som vytvoril 2 korelačné
pravidlá ktorý korelovali mnohostranné útoky z jednej IP adresy. Na záver som ešte vylepšil frontend SecMon stránky Filter, kde som zmenil ako funguje funkcia filtrovania. vi
Annotation Slovak University of Technology Bratislava Faculty of Informatics and Information Technologies Degree Course: Informatics Author: Jakub Gašparín Bachelor Thesis: Expanding system focused on monitoring safety events Supervisor: Ing. Ján Laštinec, PhD. May 2024 Cybersecurity monitoring helps us detect and mitigate potential threats and vulnerabilities in real-time, protecting sensitive data and preserving the integrity of systems and networks. The goal of this final project was to analyze various defensive systems used in modern day cybersecurity, compare them with security monitoring system SecMon and suggest and implement an improvement to it. Various functions and techniques exist and this thesis focsed on Defense in Depth (DiD) and SIEM systems as its basis for deeper analysis of SecMon and suggestions for its improvement. DiD and SIEM systems as a concept was properly analyzed and explained to fully understand why it is used. Subsequently I took a deep dive into the inner workings of SecMon, explained its architecture and shortcomings and how they can be improved. For the implementation itself, I have created 3 rules that work together with IDS Suricata that ensure event normalization and correlation of Suricata logs. Normalization rules normalize every Suricata log, including those created from custom Suricata logs. Correlation rules were created to correlated events related to multifaceted attacks from a specific host IP. And lastly I improved the filter section of SecMon by upgrading how its frontend filter functionality works.
Acknowledgment I would like to express my sincere gratitude to Ing. Jánovi Laštincovi, PhD. for their support and guidance throughout the research process. Their expertise, encouragement, and feedback were invaluable in helping me to complete this project.
Contents 1 Introduction 1 2 Analysis 3 2.1 Defense in depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1.1 Characterization of perpetrators . . . . . . . . . . . . . . . . 3 2.2 Barriers of defense in depth . . . . . . . . . . . . . . . . . . . . . . 4 2.2.1 DiD strategy . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2.2 Active protection . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.3 Real time detection . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.4 Intrusion tolerance attacks . . . . . . . . . . . . . . . . . . . 7 2.3 SIEM Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.1 Correlation rules . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3.2 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3.3 User and entity behavior analysis . . . . . . . . . . . . . . . 10 2.3.4 Network security with SIEM . . . . . . . . . . . . . . . . . . 10 2.3.5 Data storage . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.6 Data visualization . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4 Forensic capabilities and data examination and analysis . . . . . . . 12 2.4.1 Limitations of SIEM tools . . . . . . . . . . . . . . . . . . . 12 xiii
Contents 2.4.2 Modern day SIEM vendors . . . . . . . . . . . . . . . . . . . 15 2.4.3 FortiSIEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4.4 ArcSight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3 SecMon analysis 19 3.1 SEC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Normalized events . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3 Correlation events . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4 Normalization rules . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.5 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.6 Data model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.7 Front end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.8 Possible improvements . . . . . . . . . . . . . . . . . . . . . . . . . 26 4 Suricata analysis 29 5 Solution proposal 33 6 Description of implementation 37 6.1 Implementation enviroment . . . . . . . . . . . . . . . . . . . . . . 37 6.2 Suricata configuration . . . . . . . . . . . . . . . . . . . . . . . . . 38 6.3 Suricata normalization rules . . . . . . . . . . . . . . . . . . . . . . 40 6.4 Correlation rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 6.4.1 Multifaceted attack from the same IP directed at various hosts 43 6.4.2 Multifaceted attack from the same IP directed at a specific host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 6.4.3 CEF format . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 6.5 Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 xiv
Contents 7 Testing 51 7.1 Testing enviroment . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 7.2 Normalization Rule Testing . . . . . . . . . . . . . . . . . . . . . . 52 7.3 Correlation rule testing . . . . . . . . . . . . . . . . . . . . . . . . . 52 8 Conclusion 57 8.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 8.2 Deficiencies and observations . . . . . . . . . . . . . . . . . . . . . . 58 8.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 A Work plan A-1 A.1 Winter semester . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 A.2 Summer semester . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 B Technical documentation B-1 B.1 Implementation of new rules . . . . . . . . . . . . . . . . . . . . . . B-1 B.1.1 Normalization rules . . . . . . . . . . . . . . . . . . . . . . . B-1 B.1.2 Correlation rules . . . . . . . . . . . . . . . . . . . . . . . . B-5 B.1.3 array_push() error . . . . . . . . . . . . . . . . . . . . . . . B-10 B.2 Using the rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-11 C Contents of the zip file C-1 xv
Contents xvi
Chapter 1 Introduction Cybersecurity is one of the most important spheres in informatics and one where the demand for tools that reach as close to perfection as possible is very common. We will find various security tools surrounding us everywhere„ without us even knowing. Best tools work constantly without break, observing and monitoring various networks and systems in real time, without interrupting their ongoing processes. A security tool can loudly announce itself to the attacks as a psychological detriment or it can silently sound an alarm to the security team so they can address the issue. We need these tools to protect us, computers allow us to target issues that humans themselves are not capable of resolving. Networks have to process thousands, if not millions of communications at all times. For a human, the task of analyzing such large amounts of data, processing it and then addressing potential issues is far beyond their capabilities. We need computers and the tools we develop on them to properly defend our belongings. Risking leak of information can cause massive monetary losses and loss of privacy. 1
Chapter 1. Introduction 2
Chapter 2 Analysis 2.1 Defense in depth Defense in depth, sometimes referred to as Rings of depth, is a widely used concept in cybersecurity practically everywhere that utilizes any form of networking and networked environment. Because so much information is processed through network communications, it is vital to achieve a high level of security. Defense in depth is a “best practices” strategy relying on layered security techniques, where each layer is meant to function as a back-up should one or multiple of the other layers fail to protect from outside attacks. Originally conceived by the US National Security Agency, defense in depth is a comprehensive approach to cybersecurity and protection of information[1]. 2.1.1 Characterization of perpetrators In order to build a strong cybersecurity and layered methods of protection, a facility first needs to understand its adversaries, reasons for their attacks and forms in which the attack can occur. 3
Chapter 2. Analysis Some of the most common motivations for attacks include information and intelligence gathering, theft of intellectual property, denial of service, blackmail, ushering public shame upon a target, or purely out of curiosity if the attacker has the skills to breach security of a facility, website or database. Types of attacks include monitoring of communications, direct attacks on a network, exploitation of insiders, direct physical breach of facility. 2.2 Barriers of defense in depth The principles of Defense in depth (DiD) are built upon several layers, or barriers, of defensive mechanisms that each surround a given area we wish to protect. These barriers protect from unwarranted access by intruders and attackers and deny them access to the protected assets. Each barrier must have these functions: [2] • Deterrence: warning against unauthorized access - uses signs, bright and stark lighting, sometimes loud noises and psychological cues. • Detection: should defense be breached, system must take notice of the intruders and register them as soon as possible • Delay: should defense be breached and unauthorized person(s) gain access to the facility, other physical barriers should slow down the intruders and delay progress of their attack before a positive response occurs. • Response: after the physical barriers delay intruders enough, a sufficient response should be the apprehension of perpetrators [3] These techniques are then utilized in the following barriers when we are designing a system to protect our assets: • Psychological barriers: barriers clearly announce a boundary that unau4
Chapter 2. Analysis thorized person(s) has no right to cross; these warnings can take form in signs, chains, fences, gates, lighting etc. • Technological barriers: purpose of these barriers is detection of presence of perpetrators using technologies such as cameras, intrusion detection systems and access control systems. • Physical barriers: these barriers provide sufficient obstacles to slow down progress of the attackers or can serve as a deterrent to the attack, these barriers can be walls surrounding a building, fences, gates, locked doors, barred windows, chained containers, locks using password or numerical combination used in safes etc. • Proccedural barriers: operations that help secure integrity of assets and maintain sufficient security of the facility; include security guards, badges, access cards, identification cards. [3] 2.2.1 DiD strategy Because we are building a multi-layered mechanism, we must ensure that all layers are properly secured. However, only securing the network of a facility is not enough, as security vulnerabilities and flaws can be found in every single part of the overall architecture of a facility. Operating systems, applications, databases are all subjects to attack and must be protected as well. Additionally, in the public’s eye, an often overlooked aspect of security is the education of the staff on proper etiquette, securing their individual PCs, as well as forcing them to use strong login credentials. Proper training of security guards is also important to ensure stability and maintenance of physical barriers. Individual layers of security go far beyond just ensuring a secure network: a proper DiD strategy involves creation of full-time protection of all its systems, ranging from individual desktop 5
Chapter 2. Analysis PCs and the staff to the main server and database. Should one layer be damaged, there is going to be another defensive mechanism in place to protect the company’s assets. If an attacker does not get discouraged by physical barriers and invades a company, technological barriers and staff should be there to quickly respond. If an attacker manages to steal an internal worker’s desktop computer password (or the login information gets leaked), the database should still be safely secured by administrator-only access. As such, a proper DiD system strategy must be designed with 3 core parts in mind. Active protection, real time detection and intrusion tolerance attacks. Figure 2.1: Figure 2.1: Network defense in depth strategy [4] 2.2.2 Active protection The Information Technology sphere is constantly evolving. New technologies and systems are invented and they become more and more complex. Because of this, no matter how strong a security is, it will eventually become outdated, as new vulnerabilities and exploits are found and potentially abused by attackers. New forms of attacks and various websites and tools offering different attack styles and types are developed very rapidly, so even if a technology is developed with the most advanced and available defensive measures, it will fall victim to brand new and unknown attacks that have yet to be documented and accounted for, and as 6
Chapter 2. Analysis such these unknown attacks have a potentially simpler time penetrating protective layers of a DiD strategy. [4] Active protection is a term that addresses this problem. To keep up with all the newly developed attacks the defensive side also needs to constantly monitor all new found ways to exploit weaknesses and come up with new solutions and defensive measures. 2.2.3 Real time detection Despite having strong multi-layer defense, there is still a possibility it will be overcome. As such, there needs to be real time detection to monitor all attacks which have not been successfully defended against by the layers. A big downside of relying purely on automated detection systems is false positives or lack of detection of an attack in the first place. Additionally, it is very difficult to account for every technical downside or mistake created by a detection system[4]. 2.2.4 Intrusion tolerance attacks Intrusion tolerance is a fault-tolerant design that aims to ensure continuous operation of a service, even while under an attack or in the event of one of the components of the DiD strategy system failing, including failures that come from the inside of a facility. A well known intrusion tolerance system is the Byzantine fault-tolerant system that proposes that n components can still function and operate at their intended (or slightly reduced) level even if f < n components suffer from a fault, be it an outside attack or a failure coming from the inside.[5] 7
Chapter 2. Analysis 2.3 SIEM Systems Security Information and Event Management (SIEM) is a system designed for event log analysis, captured and compiled from users’ devices, servers and intrusion detection and prevention systems. SIEM is a solution to help organizations to detect, analyze and respond to threats to security before they can cause harm to business operations. A big benefit of SIEM systems is that they provide passive protection in the background, while leaving the main operations of servers and networks intact, without interrupting them in the majority of cases. Poor implementation of SIEM tools however will increase risks of system breakthroughs and successful outside attacks. [6] Figure 2.2: Figure 2.2: Basic components of SIEM [7] SIEM tools have very specific functions within the cybersecurity system of any given organization. They aim to collect, aggregate and analyze event logs collected from users and use this information to provide real time security analysis, so the security teams can respond to potential threats quickly. SIEM systems function on a set of predetermined rules nowadays, unlike in the past, when they heavily relied on signs and signatures to detect potentially harmful behavior. Modern SIEM tools come with their own correlation rules and models to provide proper monitoring. Additionally, they can be customized and changed, removed or added to the rule set to create a model fitting an organiza8
Chapter 2. Analysis tion’s needs. 2.3.1 Correlation rules Correlation rules are logical expressions that force the system into an appropriate response to a certain event. A very basic example of a correlation rule would be password policy enforced on all network systems - “If the user fails to login 5 times, time out the account”. It is a safety precaution that takes place when a condition to trigger this precaution happens. The rules function automatically on their own, without properly considering past events, conditions or characteristics of the system at a time when a prior attack happened, unless otherwise specified. Additionally correlation rules can be joined together to create groups of rules to achieve more varied and complex correlation behavior of the SIEM system[8]. These groups’ rules consist of simpler correlation rules, but are created to prevent attacks or irregular activities that cannot be handled by a rule covering only a single event. 2.3.2 Models A Siem module is a compilation of behaviors by users, assets and devices, sounding an alert when their behavior differs from their usual behavior. When a model detects behavior deviating from the norm, it will use certain rules to assess the situation and respond appropriately. The main difference between correlation rules and models is that models don’t have to always be activated as a response to an event that triggers them, unlike correlation rules which are much more aggressive with their responses, activating every time their trigger happens. This is because a model consists of logical classification expressions and its own set of rules to dictate its behavior. 9
Chapter 2. Analysis SIEM systems can consist of multiple types of attack model security evaluation processes. Their components allow for error correction in networks, revealing possible attacks on the system, decide how to handle critical cases, where an organization’s resources are at risk, and secure their safety, using appropriate mechanisms fitted for current threats. All these parameters are handled by the security administration, whose responsibility is to ensure the provision of necessary security. Examples of procedures used in models: • networking interface supporting interactions with external environment • interactive decision support module • malefactor modeler responsible for keeping track of all possible attacks[9] 2.3.3 User and entity behavior analysis User and entity behavior analysis (UEBA) is a model created by using normal user behavior of individual users and network components like IP addresses or application communicating over a network. Many SIEM systems make use of EUBA additional layer of security to accompany, or sometimes even replace correlation rules. Popular SIEM tools that use UEBA are IBM QRadar or ArcSight [10] 2.3.4 Network security with SIEM Globalization is a part of today’s business, many organizations and business enterprises have moved large parts of their operations online. As such, proper cybersecurity is a big requirement and an ever growing demand. SIEM tools are widely used in network security because of this reason, as they provide all the tools needed to combat cyber attacks. Without them, these online-only organizations would be very vulnerable to various forms of targeted attacks. Real-time analysis of the network and alerting the security administrators is the main responsibility of SIEM 10
Chapter 2. Analysis tools, all while using correlation rules and models to detect, collect, store and respond to all possible threats that the SIEM tool detects, while ensuring smooth running of the system without possible interruptions. This means that whatever attack is launched against a system, the implemented SIEM tool should be able to detect it, respond to it appropriately, report it to the security team and log it as an event for future precautions, all in real-time, without potentially disrupting the system it protects. For example, if a SIEM detects a potential breach in employee etiquette, such as downloading potentially harmful software on their machine, the SIEM tool should ensure the whole system would not shut down, it should only address the breach in rules. Proper implementation of SIEM results in far better real-time analysis of the network and quicker responses to attacks. There are several types of SIEM tools out there, available as a license or as open source. 2.3.5 Data storage Low scalability is one of the most well known issues that SIEM tools have. This limitation restricts them heavily in their usage in more advanced networks and infrastructures, such as power plants or water dams. This flaw is heavily reflected in data storage as one of key components of all SIEM systems. The data repository is an attempt at keeping all relevant data stored for future processing. All SIEM systems must keep track of data for a certain period of time for forensics operations, for example the type of attacks that took place in recent time, and keep them in mind in future threat prevention. This storage keeps track of event logs or source devices. For all possible SIEM solutions out there, each one treats data storing slightly differently and each SIEM offers the ability to customize them to the needs of the user. [11] 11
Chapter 2. Analysis 2.3.6 Data visualization A very important feature that, due to its absence, can slow down the analysis and solution of a threat, is the lack of its visualization. Data visualization is an important part of the interactivity of SIEM tools and helps understand the capabilities of the system, how it collects and treats its data and what sort of threat the security team is dealing with. [11] 2.4 Forensic capabilities and data examination and analysis Some SIEM systems provide very quality forensic capabilities. For example, the ability to perform a full packet capture and its analysis for potential threats. Despite the fact that many of these analysis processes are automated by some of the best SIEM tools out there, many aspects of data analysis still need to be done manually by an experienced team. However, the best SIEM tools can provide tools to speed up the examination of this data and various security events. A standard feature SIEM tools provide support this human examination process. This feature of modern SIEM tools is part of data visualization, but not all systems provide this functionality. While many open systems offer full documentation for their way of handling data, some do not share this information. [12] 2.4.1 Limitations of SIEM tools Unfortunately, despite the numerous upsides, current SIEM systems are limited in many of their aspects. Oftentimes their responses are limited and countermeasures are chosen based on a very primitive set of boolean rules and instructions. Some of the more notable limitations that this paper has touched upon include: 12
Chapter 2. Analysis • Incomplete data It is simply not cost effective to capture, store and process all the data that is required for fully functional SIEM. Some data will always be missing. Because of this, the data storage in many SIEM tools is limited. Some results of incomplete data are fragmented user profiles (therefore modules that depend on user behavior to fully function will be lacking in some areas). Current SIEM systems capture all traffic through VPNs, but very few try to keep track of connections established through DNS traffic or through SMTP connections (email transfers, email logs etc.). As a consequence of this incompleteness, the system cannot know for certain about everything happening on the network. As such, some level of human input is required to ensure total security. Despite that, a foolproof system is not possible to achieve. What is worse, systems ignore data that has been archived for forensic purposes. Current systems store the data for only a set period of time, usually 90 days or 6 months. Afterwards the data is either deleted or transfered elsewhere, usually to another database. This is done to primarily limit the storage space. Many viruses, worms or other forms of harmful software utilize timed payloads that only trigger at a specific day or time. If this happens after the storage period is over, the SIEM tool can fail to properly defend against this attack. [12] Some modern day systems aim to implement an ontological data repository system for SIEM. Ontological data approach can ensure easier scalability, as well as better storage of security vulnerabilities. [11] • Correlation rules are too simple 13
Chapter 2. Analysis Many of the current correlation rules do not cover enough range to ensure proper security on all fronts. Many of them are too weak, simple and are built for alerts for each case. As a result, threats are treated as individual cases and solved in isolation, independent from one another. [13] While they exist, almost none of the current SIEM tools try to incorporate more complex correlation rules or mixed groups of rules that are built to handle more complex situations and potential threats. A very notable feature that is still lacking is the proper view at historical data and trying to determine better solutions at handling modern day threats.[13] • Dependent on humans As it was mentioned many times in this paper, certain aspects of SIEM are made with human input in mind. As such, they need to be built with proper interfaces to support human interaction. While at first glance this does not sound like a massive flaw, because the systems cannot be fully automated and they still require some level of human input, an experienced employee must be hired to handle security alongside SIEM tools. This is a costly process and requires an experienced security administrator that respects and understands the potential threats and knows how to handle the SIEM tool. Depending on the tool, this process of educating the user can be more costly and lengthy than one would expect from a tool that is supposed to be almost fully automated. Additionally, human error is a very common occurrence in security. Many security breaches happen because of errors on the side of employees. This tendency to make mistakes is prevalent even in high level security administrators. It cannot be written out and must be accounted for. Modern day research is trying to automate the process fully, but not many SIEM tools 14
Chapter 2. Analysis come close to this future vision of total automation of security. [12] 2.4.2 Modern day SIEM vendors SIEM is not a singular entity, it is a concept that many organizations applied in creating their own SIEM tool. All tools have the basic ability of collecting and using data to fight potential threats. This section will focus on a few of the most well known SIEM tools that are used in modern day organizations. 2.4.3 FortiSIEM Developed by Fortinet, FortiSIEM [14] is a SIEM tool that utilizes many modern day advancements in the industry to fix some of the most common issues with SIEM systems. One of the most notable ones is the use of machine learning to detect unusual user and entity behavior (UEBA), without the need of an administrator to write complex correlation rules. Additionally, it is able to monitor all its users and rank them using risk scores. The risk scores are calculated by observing the users, their behavior and usage of a device. The score is stored and displayed on the user interface. Some of the other notable features of FortiSIEM: • Real-Time operation context • Performance monitoring • Scalability of its analytics and data • Scalability of log collection • Customer file logging • Observation of DNS and DHCP 15
Chapter 2. Analysis • Easy to use thanks to its web-based GUI and protection by HTTPS [15] 2.4.4 ArcSight ArcSight, developed by CyberRes, is a SIEM that prides itself on its 360 degree security analytics that implement aspects of real-time detection with machinelearning behavior for its analysis and gathering of data of various threats. The basis of this technique is to preemptively catch potential threats, even if their tool cannot “see” it directly. Because the amount of data needed for each SIEM to analyze is so massive, attackers can easily hide amidst the overwhelming amount of information. ArcSight combines aspects of artificial intelligence and machine learning and combines them together to highlight, spot and preemptively prevent abnormal activities. This results in faster and more actionable responses while attempting to minimize the time the company needs to address potential threats. On top of this ArcSight provides a modern graphical interface and views on its analysis and data gathering. One of the key features of ArcSight is its ability to properly detect threats preemptively while providing a user-friendly GUI.[16] 16
Chapter 2. Analysis 17
Chapter 2. Analysis 18
Chapter 3 SecMon analysis SecMon’s name is derived from the words “SEC”, an open source event correlation tool, and “Mon”, short for “monitoring”. It’s a tool developed to function as a “controller” for using Sec more effectively. In other words, SecMon is simply a monitoring tool and a GUI window that aims to make working with Sec easier. Like many other SIEM tools, it aims to provide the basic functionalities of a security monitor tool, while displaying various information that the user can filter through. It collects gathered event logs and displays them in an easy to understand format. The user can browse through these logs, filter them according to their needs and display what they need for identification of threats [17]. In this chapter I will describe some of the basic functionalities of SecMon and how it utilizes SEC for event gathering and processing. 3.1 SEC SEC is an open source tool used as an event correlator and normalizator. It is written in the programming language Perl and brought to the public on March 19
Chapter 3. SecMon analysis 23, 2001. Unlike many other traditional event log managers, SEC is a lightweight and platform-independent UNIX tool. This means that a user can run it on any operating system, be it Windows or Linux, however it was primarily tested on Linux and SOlaris. It manages to deterministically gather and process event logs in a single-thread manner and, because it is so non-demanding to run, it is possible to have several SEC instances active at the same time, on the same machine or system, with each instance having its own independent roles and rules [18]. Start and installation of SEC is relatively simple and straightforward. It is possible to activate it as a daemon and execute it in a terminal and then run many SEC processes simultaneously. As it was mentioned earlier, this is very non-demanding because of SEC’s size[19]. In the following figure 4.1 you can see the overall architecture of SecMon 3.1: Figure 3.1: SecMon architecture [17] 20
Chapter 3. SecMon analysis 3.2 Normalized events When a tool processes logs of events, they will be returned in the form of raw input events. Working with data formatted in this way is not easy. Taking this data and extracting each individual field from it is called normalization. For example, our system detected an unusual activity coming from various machines, but this suspicious activity was connected to a single IP address. Normalization allows us to extract that singular information field alongside other potentially important attributes like name, source, destination, timestamp, UID of the communication or protocols used in the communication. The very obvious benefit of normalization is that we can take any large amount of data a tool produces and parse it through appropriate filters to extract our desired information. Without a feature like this, working with the processed data can be very difficult even for computers. A big disadvantage of normalization is that it is very difficult to process data if it is spread across multiple event sources. It’s not optimal to extract fields while ignoring the dependencies or influence it may have on other entries in separate event logs. It’s needed to find a similarity or a union between the data to create a sensible scheme our parsing will follow. This can get exceptionally difficult if we want to parse millions of records and create a normalized log out of them. This is a scalability issue and one that is very difficult to solve. Last big disadvantage is that it is human dependent. Chances of human error are low with an experienced user, but it is still there and it cannot be presumed that all logs created by a user will be perfect, something that, in a sensitive sphere like cybersecurity, is practically needed. One smaller disadvantage is if older data is needed for parsing, it needs to be stored somewhere. As I have described earlier in this work, many SIEM tools limit the amount of storage a singular tool has. Sometimes older data can be unavailable, unless it is properly 21
Chapter 3. SecMon analysis archived [20]. To create these normalized event logs and stores them independently in another tab, SecMon uses normalized utilities as another part of SEC correlator [17]. It uses Common Event Format (CEF)[21] for displaying normalized data. These rules are stored in the secmon/rules/Ubuntu/normalization directory. The files of the rules are found as *.rule files [22]. Normalizer in SecMon works by first reading the raw events it receives from the aggregator. Then, it starts its event normalization using the SEC tool and converting it into the appropriate CEF format. Normalized events are then sent to the next module for addtional event enrinchment [17]. Normalizator is written in PHP and its source code is in the secmon/controllers/ [22] directory in the EventsNormalizedListController.php and EventsNormalizedController.php files[22]. 3.3 Correlation events SEC is a tool used for advanced log monitoring, processing and even correlation gathering, used primarily for network and security management.[18] As such, it is a perfect fit for event correlation. Event correlation is a process that tries to find similarities between events (hence correlation). It then tries to establish and analyze their relationships. Tools that make use of this concept then send alerts based on what the event demands, for example - sending alerts because of a hardware or application failure. All of this must happen within a predefined time frame, otherwise the appropriate response will not trigger. In more complex environments, an event can be triggered millions of times in a short period of time and these events can range from unimportant, to false positives to very critical. 22
Chapter 3. SecMon analysis Such large amounts of data with so many different variables are impossible to be analyzed by a human, hence why this technology was invented and is used in modern day security tools[23]. There are numerous benefits of using event correlation. The very obvious one is the introduction of automation to security. As discussed earlier in this paper, many aspects of SIEM tools still largely depend on human review. Event correlation allows for real-time threat analysis and can alert security administrators practically instantly in case of a threat. From this we can also automatically deduce that event correlation allows for non-stop network monitoring without break. As such, our tool would always provide reports and analysis that the team can view and review at any time. Correlator is written in PHP and its source code is in the secmon/controllers/ in the EventsCorrelatedController.php file [22]. 3.4 Normalization rules Just like correlation rules, normalization rules can also be created and customized to suit the user’s tastes and needs. • iptables: normalizes events related to iptables configured on the system. • apache: normalizes events monitored on the apache server. • su and sudo: normalizes events related to the su and sudo commands, primarily to observe possible abuse of them. 3.5 Framework SecMoM was created in YII framework using PHP language. YII framework offers multiple packages but the main package that SecMon works with is called Mate23
Chapter 3. SecMon analysis Figure 3.2: SecMon Normalization rules rialize CSS. It is a very responsive framework built on Material Design. To use it for SecMon, the package was imported using Composer and its current version is defined in the repository composer.json. The package comes with javascript, widgets and css[17]. 3.6 Data model SecMon works with its own data model and it consists of many various elements connected together. • User: user is strictly defined with a clear user name, password and role that 24
Chapter 3. SecMon analysis enables them to enact actions within permission. • Roles and permission: a series of commands that differ between various roles in the system and gives different users with different roles their appropriate permissions, thus allowing or prohibiting them from enacting certain roles in the system. • View: customizable graphical interface that each user can change according to their tastes and needs. • Filters: allows user to change the way that data is displayed in the SecMon tool. Filters allow user to find and focus only on the data they themselves deem the most important • Security_events: displays all normalized and correlated events, with the option to display additional information such as source and destination IP addresses and ports, CEF Name, protocols, security level etc. • SEC_Rule: SEC rules, the cornerstore of SecMon that the system was built upon. It is possible to change and add rules, but not to remove them [17]. 3.7 Front end SecMon’s front-end consists of multiple parts: • Main menu: main menu of the software contains various information about the logged in user (username, their role) and window of each individual SecMon function. It should be noted that this window is separated into two parts the bottom one is only visible to the administrator. • Header: contains name of the site and a few buttons that serve as basic functionality of the website. 25
Chapter 3. SecMon analysis • Main window: displays functionality and options available to the user of their selected SecMon function. • Footer: contact window and copyright text[17]. All of front-end was programmed using YII framework and Materialize CSS package. The source code for each part of the front-end can be found in the /views repository [22]. 3.8 Possible improvements Below is a list of possible improvements that could be implemented or altered in SecMon: • Normalization rules for SSH pubkey authentication • Optimization of a process that is responsible for getting parents events, making it possible to use all SEC rules available • Joining of correlated and normalized events in the database • Automatic update of normalization rules, currently SecMon creates a new rule instead of automatically detecting new rule with the same name and updating the old one • Add more correlation rules and normalization rules • Add support for multiple Linux distros • Planned better collaboration with GitHub and Docker • Alerts when a threat is detected • User-friendly GUI for rule writing 26
Chapter 3. SecMon analysis • Reputation database for known harmful IP addresses • Updated online documentation • Implemented results from the penetration test • Possible rework of the database using NoSQL Secondly here is a list of various bugs found in SecMon: • GUI error related to updating contents of a component • GUI error when creating column graph • GeoIP module spamming error.log with pointless error messages 27
Chapter 3. SecMon analysis 28
Chapter 4 Suricata analysis Suricata is a high-peformance open-source intrusion detection and intrusion prevention system (IDS/IPS). Its purpose is to monitor network traffic and detect threats and attacks in real time. While Suricata is often used as a standalone security tool, it can also be integrated into other systems as part of a larger security structure [24]. IDS and IPS are technologies used in security used for monitoring network traffic and detecting unauthorized access, malicious activities or security breaches. IDS is more of a passive observer. It analyzes network traffic and raises an alert if it detects anything suspicious. While it does not do anything to mitigate the attack, it instead alerts the human administrator who then takes appropriate action [25]. IPS, on the other hand, is an active tool that specializes in both detecting and blocking any suspicious activity [26]. Suricata utilizes qualities of both of these systems for its full functionality. Suricata uses multi-threaded architecture to achieve high network throughput and low latency. Because of its ability to monitor multiple networks at the same time, 29
Chapter 4. Suricata analysis Suricata can analyze traffic in real-time and block threats as they occur. It is also highly customizable and can be configured to fit the specific needs of its user. Additionally it can be integrated with other security tools and platforms, making it very useful and versatile network security tool. On top of all these qualities, Suricata can also be used to extract files and analyze network traffic and flow, which allows it to provide detailed view of network traffic. Currently Suricata can be deployed and configured on variety of platforms, mainly on Linux, Windows and macOS. In this work I will be focusing on its configuration on Ubuntu, which is a Linux OS. While Suricata is not part of SecMon, SecMon is capable of receiving logs generated by Suricata for further event enrichment. Using syslog, the IDS can forward the logs to any SIEM if properly configured. 30