Section 8.1 Security Information and Event Management (SIEM)
As you study this section, answer the following questions:
- What are the benefits of using a SIEM system?
- What are two functions of a SIEM system?
- What is the SIEM process?
In this section, you will learn to:
- Evaluate network security with Kibana
- Evaluate network security with Hunter
The key terms for this section include:
Key Terms and Definitions
| Term | Definition |
|---|---|
| Security information and event management (SIEM) | A system that automates the collection, analysis, and response to security-related data. This automation helps simplify identifying, analyzing, and responding to security threats, especially for events contained within log data. |
| Security event management | The security event management software or appliance consolidates all monitored data in one location. |
| Security information management | The security information management function correlates the data gathered by security event management to identify events that raise concerns from a security perspective. |
This section helps you prepare for the following certification exam objectives:
| Exam | Objective |
|---|---|
| CompTIA CySA+ CS0-003 | 1.2 Given a scenario, analyze indicators of potentially malicious activity
1.3 Given a scenario, use appropriate tools or techniques to determine malicious activity
2.2 Given a scenario, analyze output from vulnerability assessment tools
|
| TestOut CyberDefense Pro | 1.1 Monitor networks
1.3 Implement logging
|
8.1.1 Security Information and Event Management (SIEM) Overview
Click one of the buttons to take you to that part of the video.
Security Information and Event Management (SIEM) 00:00-01:21 In this video, I'm going to talk about Security Information and Event Management, or SIEM. Infrastructure-related security events can happen anywhere.
Workstations, servers, switches, routers, and other network-based devices are all places to keep an eye on. By default, the events generated by each of these systems are stored locally, so you would need to access each device in each physical location to view the security-related events. Depending on the network size, this could take hours to accomplish. Besides that, data in this form is extensive. Relevant security events are scattered throughout thousands of mundane, unimportant events that typically clutter up a device's log file.
SIEM systems help to increase efficiency at detecting security issues. SIEM systems are sold as software applications or as standalone security appliances and combine two functions, which are event management and information management, into a single management system.
The event management function aggregates security events in the form of log messages. The information management component allows you to view the information that's been gathered and analyzed. A dashboard provides you with a quick overview of your monitored system's current state.
SIEM Process 01:21-03:54 A SIEM system makes things much more efficient and accurate than a manual analysis. But how does it work? You can implement SIEM in several ways, and there are many software and hardware solutions that provide either partial or comprehensive solutions.
Most SIEM implementations start by installing a collection agent on each of your network-based devices. These collectors forward security-related events from each device's log files to a centralized SIEM management console. The SIEM system aggregates the data from your sources, where it's stored in a database and organized.
The data goes through a correlation process after it's been aggregated. During correlation, the data is categorized based on a set of rules. This allows the SIEM to form relationships between the log data and actual threats.
Security analysts are often tasked with rule-writing, and these rules can sometimes get complicated. Luckily, there are many software solutions with default rules that can help save you time. But it's a good idea to spend a bit of time configuring custom rules that reflect what your organization needs most. These rules could be simple, such as creating a list of known bad IP addresses. This is basically just a list of IP addresses that've been blacklisted for known or suspected behavior. You could also make complex rules that involve behavioral patterns. But rules should be well-balanced. They shouldn't be so general that you become overwhelmed with false alerts, but they shouldn't be so specific that you miss a security breach.
After all the data has been aggregated, converted, and correlated, the next step is to do something with the information output. One of the key features of a SIEM solution is automated alerts and triggering based on collected data. For example, when the system identifies a threat, you need to have a way to notify the security team so that they can address the threat immediately. This could be in the form of automatically generated emails or text messages to administrators. It could even be a ticket created for
the help desk. Because this is all automated, be sure to include an escalation path if the people who receive the alert don't have the ability to resolve it.
The nice thing about SIEM reporting is that, when it's configured properly, it's much more informative and comprehensive than traditional logs. It can provide an explanation for a threat and might help you find short- or long-term solutions for the issue.
Summary 03:54-04:05 That's it for this lesson. In this lesson, we discussed SEIM and how the SIEM process works.
8.1.2 SIEM Review Facts
Security information and event management (SIEM) automates the collection, analysis, and response to security-related data. This automation helps simplify identifying, analyzing, and responding to security threats, especially for events contained within log data.
This lesson covers the following topics:
- Security information and event management (SIEM)
- SIEM process
Security Information and Event Management (SIEM)
Security information and event management (SIEM) solutions provide near real-time analysis of security alerts generated by a wide variety of network hardware, systems, and applications. SIEM platforms enhance incident detection and response capabilities by providing expanded insights into operational activity through the collection, aggregation, and correlation of vast volumes of event data across the entire enterprise environment. Security analytics by means of log analysis and activity auditing is significantly increased, as SIEM platforms help analysts quickly and effectively identify suspicious activity that would otherwise take considerable time and effort to perform.
SIEM systems help to increase the efficiency and effectiveness of detecting security issues. SIEM systems are sold as a software application or as stand-alone security appliances. Not all SIEMs are the same, but most include similar features, including the functions described in the following table.
| Function | Description |
|---|---|
| Security event management | The security event management software or appliance consolidates all monitored data in one location. Security event management:
|
| Security information management | The security information management function correlates the data gathered by security event management to identify events that raise concerns from a security perspective. It works as follows:
|
SIEM Process
A SIEM system can be implemented in several ways. Many software and hardware solutions provide either partial or comprehensive solutions. Most SIEM implementations follow this general process:
- A security analyst establishes rules to gather and aggregate data. The rules allow the SIEM to form relationships between the log data and threats or events.
- Data is pulled from locations, such as workstations and servers, that provide logs.
- Data is stored and collected. It is ready for translation or normalization.
- Automated alerts and triggers based on preconfigured settings keep your security team aware of any existing threats.
The following table identifies several SIEM concepts:
| SIEM Concepts | Description |
|---|---|
| Log file | A log message is a message generated when a specified event occurs on a network device. A log file is a file of log messages created on the device. Log files are stored on a local device. Log files can be a valuable resource when you are troubleshooting network problems. |
| Aggregation | Aggregation is the process of combining log files and other data from various sources into a common repository. Data must be aggregated so that it can be pulled from. After the data is collected and stored, it is ready for translation or normalization. |
| Correlation | Event correlation gathers data, analyzes it, and compares it against known malicious behavior. This allows the system to link events across the entire enterprise architecture to form a more complete picture of important events. |
| Automated alerts and triggering | Preconfigured settings trigger automated alerts and send them to security personnel. These alerts can be customized based on the needs of the organization. |
| Rule writing | Security analysts are often tasked with writing the rules for the SIEM. Many software solutions have preset or default rules that can save time. Consider configuring custom rules that reflect the needs of your organization.
|
| Known bad IP addresses | Known-bad IP addresses is a list of IP addresses that have been blacklisted for known or suspected malicious behavior. |
| Event deduplication | Event deduplication is the practice in SIEM that tries to reduce the number of events by eliminating duplicate events. |
| Retention | The retention function provides long-term storage of collected data to meet government compliance requirements. |
| SIEM dashboard | The dashboard is a common component of all SIEM systems. The dashboard consists of customizable information screens that show real-time security and network information. The real-time information allows the IT security team to monitor and respond to events on the network effectively. |
| Compliance | SIEM facilitates compliance by producing activity reports designed to meet governance and auditing requirements. |
| Data retention | SIEM platforms have the capability to store historical data, which is critical for deep event analysis, digital forensics, data retention, and compliance requirements. |
The following table describes a couple of SIEM tools:
| Tool | Description |
|---|---|
| Splunk | Splunk is a collection tool you can use to search and analyze large collections of data in multiple formats. |
| Security Onion | Security Onion is an open-source intrusion detection system that also provides SIEM-type services. It includes search and analysis tools to help index collected data. Security Onion is a Linux distro that is based on Ubuntu. |
8.1.3 Use Security Onion v2 – Hunter
Click one of the buttons to take you to that part of the video.
Use Security Onion 00:00-00:31 Security Onion is a free, open-source intrusion detection system, or IDS. It also provides network security monitoring and log management. Security Onion can provide many layers of protection. We're going to use several of the tools available in Security Onion to parse a PCAP file.
Security Onion can be installed with an included OS or on top of Ubuntu or Fedora. Our installation is on Ubuntu.
Import a PCAP File 00:31-01:17 For this demonstration, we'll import a saved PCAP file, which allows Security Onion to parse the data and flag anything suspicious. First, I need to import the file. From a shell or command line, I can use the 'sudo so-import-pcap' command with the name of the file. After a few moments, the file has been uploaded, and a URL is returned. I'll copy and paste it into a browser. The login credentials were created during the installation of Security Onion.
Parse a PCAP File with Security Onion 01:17-06:26 Okay, I'm logged in. I'm interested in the Hunt section, where I can view the results of the PCAP parsing. Let's review the interface here. First, I need to tell Hunt which time portion I want to parse. Our sample PCAP is from July 10th, 2020. I'll change the date to match. I'll give it a full day to work with. Notice that it's already parsed the file and created 337 alerts, as found in the upper right hand corner. There are also graphs. I can sort the filter the results to give me an easier list to work with. I'll start by changing the Onion Query Language string at the top left to include the events found in the dataset. The interface has been updated to reflect the change. Now I have two categories of occurences, zeek and suricata.
zeek is a free, open-source software network analysis framework that parses live or recorded data to generate events when something happens to trigger zeek's notice.
suricata is also a free and open-source software network analysis tool. It identifies specific attacks and can flag them as related to specific known threats.
Our Security Onion installation has used zeek and suricata to identify specific items that are suspicious and should be reviewed by a security professional. If we scroll down the page, we can see individual events listed. I'm going to expand the number presented per page, then sort by timestamp so we can follow the timeline. I'm going to update the query again to list the events by rule name. This narrows results even more.
Let's take a look at the Events section. If I look at the rule.name field, the first listed event is an Emerging Threat (that's the "ET" at the beginning of the field) with a Dotted Quad Host DLL Request. This means that instead of a hostname, there was a request from the 10.7.10.101 address, our local machine, to download a DLL from a remote IP address. That is somewhat suspicious. Let's look at the details. The destination IP address is in Germany. That's not suspicious by itself. But the fact that it's just an IP address, not a hostname, is something to consider. If I scroll down, I can see the rule.name.
Let's look at the raw packet data and see what's inside. If I click on the event in Security Onion and select PCAP under Actions, I can open this specific event and view the frame data. Playing with the views, I can see that this is a downloaded DLL and that it was specific to a Microsoft executable as referenced by the MZ at the start of the data, here. Also, notice that the Host: field is an IP address, not a hostname. Together, these caused Security Onion to generate alerts.
If we wanted to, we could download the PCAP from security onion and use another tool, such as NetworkMiner, to rebuild the file and examine it further, or perhaps reverse-engineer it.
That's only the first alert that Security Onion flagged in our PCAP file. We also need to examine the others. The communication between the exploited system is encrypted via a forged certificate that's recognized as part of the Trick bot malware. We can pivot to the PCAP data itself to examine this more closely. Notice that the certificate includes example.com as part of its address. That isn't done by legitimate certificate owners. This particular certificate is identified as being used by Trick bot.
Now that this host is compromised by a DLL from a questionable host, the malware will begin to harvest information about the host for further exploitation. In the Events section, there's a reference to a suspicious post to a dotted quad address. The first event appears to be a username and password taken from chrome passwords. A second event appears to post data to an IP address in Cambodia. If we look at the PCAP data from that event, we can see that the running process list and the host network configuration were transferred.
Summary 06:26-06:52 That's it for this demo. I've walked you through some of the features of Security Onion. There's a great deal more that can be done with Security Onion, but its flexibility and power are easy to see through a PCAP of an Windows machine exploitation. As with all security tools, practice increases proficiency, and you'll learn to do more and more as you use all the features.
8.1.4 Use Security Onion v2 – Kibana
Click one of the buttons to take you to that part of the video.
Use Kibana 00:00-00:49 Kibana is an open-source, browser-based visualization tool. It is primarily used to analyze a large number of logs, outputting data in line and bar graphs, pie charts, region maps, heat maps, etc. When used in conjunction with Security Onion, a professional can sort security-related data and see trends and tendencies in a visual form.
I'm going to introduce you to a few features of Kibana. Because it's such a complex tool and has many features and functions, I'll only present a few items. As with other security tools, practice and training will be of great use in building proficiency.
The version of Kibana I will be working with comes as part of a Security Onion 2 installation, with a focus on security. Kibana is a data visualization tool and is not only for security, although that's how we'll use it today.
Use Kibana to Explore Security Events 00:49-03:47 I'll start by logging into Security Onion. Previously, I imported PCAP files that show the exploit of various machines over the past several months. I will be using those PCAP files to showcase Kibana's capabilities. Once logged in, I click on Kibana on the left menu. It opens Kibana in a new tab. Nothing is displayed in Kibana yet, as the time filter is set too narrowly. I'll expand it to include data from the last 90 days, which will add in some of my imported data.
Kibana is designed to be flexible and allow complex searches and filters to further refine which data is available. I can use the search bar to find terms I'm interested in using plain search terms, or I can use KQL, the Kibana Query Language, to perform complex data refinement.
I have several thousand log events to look through and a graph of when that data came into the system. I have a graph in the Data Overview section that shows which tools parsed the data for me, and I can click on the sections of that graph to further filter by those tools. For instance, if I click on the suricata ring represented in the graph, and click apply, it'll use suricata as a search filter. The graphs have updated as a result, giving us a visualization of the data. If I look up near the top left, I can see which filters are being used. In this case, I'm filtering on suricata and network. I want to focus on just the suricata events, so I will remove the network filter.
If I scroll down, I can find individual log events. I'm going to sort by time, looking for the first event in the last 90 days. If I expand the first log event, I can see that this was a communication event between some host on my local network and a host in Russia. During that time, an executable was downloaded by a host on my network. That's a security event I'll want to examine. If I filter on that rule name entry, I can sort by other events where an executable was downloaded. If I scroll up to the top of the Kibana dashboard, I can see that the number of events that match my filters which I can see here at the top is 18.
Let's view which file was downloaded. If I open the _id field in another tab, it shows the actual packet data. I can see that the name of the file was tender.exe. Looking at another logged event the last one in my list and opening the _id field in another tab, I can see that the file requested was an image file, but what was returned was an executable of some kind. I know this because of the MZ and the phrase "This program cannot me run in DOS mode".
Once again, if I look at another logged event and it's _id field, I can see that the file that was downloaded was a DLL, with those MZ characters and the "This program cannot be run in DOS mode" phrase.
There are many more features and filter possibilities within Kibana. We have touched only a very small portion. With practice and experience working with Kibana, a professional can be adept at finding and identifying security breaches.
Summary 03:47-03:57 That's it for this demo. In this demo, I've walked you through a few of the features of Kibana.