Section 8.2 Security Orchestration, Automation, and Response (SOAR)
As you study this section, answer the following questions:
- What is a SOAR system?
- How is a playbook used in a SOAR system?
- What are the benefits of DevSecOps?
- What is single pane of glass orchestration?
In this section, you will learn to:
- Explain automation technologies
- Explain DevSecOps
- Explain workflow orchestration
The key terms for this section include:
Key Terms and Definitions
| Term | Definition |
|---|---|
| Security orchestration, automation, and response (SOAR) | A collection of technologies that allow the security team of an organization to collect various types of inputs that they can monitor. |
| Continuous integration | Automating the integration of changes that multiple contributors make to a development project. |
| Continuous delivery | Automatically deploying all the changes the coders make into a testing/production environment. |
| DevSecOps | An abbreviation for development, security, and operations. It is a philosophy that everyone in an organization should be responsible for the security of their systems. |
| Workflow orchestration | Planning out tasks in such a way that they are as efficient and effective as possible. |
| Scripts | Lists of commands that can be executed by a certain program. |
| Application programming interface (API) | A set of functions and procedures that allow two or more applications to communicate with each other. |
This section helps you prepare for the following certification exam objectives:
| Exam | Objective |
|---|---|
| CompTIA CySA+ CS0-003 | 1.2 Given a scenario, analyze indicators of potentially malicious activity
1.3 Given a scenario, use appropriate tools or techniques to determine malicious activity
1.5 Explain the importance of efficiency and process improvement in security operations
3.2 Given a scenario, perform incident response activities
|
8.2.1 Automation Overview
Click one of the buttons to take you to that part of the video.
Automation Overview 00:00-00:29 Think about the chores you do every day, like washing your clothes or doing the dishes. You could do these things by hand if you wanted, but it would take longer and probably be less effective than doing them with a machine. The process of having machines assist human labor is called automation, and a similar mindset is used in many kinds of computing. In this lesson, I'm going to cover a variety of facts about automation so you can have a better idea of how this works.
DevSecOps 00:29-01:19 The term Development, Security, and Operations is more commonly referred to as DevSecOps. This is the philosophy in which everyone in an organization is responsible for their system's security. This means that everyone should implement security decisions in the same way that development and operation decisions are made. If everyone in a company becomes better at security, the entire organization benefits, and customers have greater software assurance.
While this might take a while to set up initially, it eventually streamlines the process for everyone and leads to lower costs and development times. During the software development process, engineers constantly check each other's work and test the software for bugs and vulnerabilities. In this way, fewer vulnerabilities make their way into the final product, which means that we end up investing less time and money into fixing things post-release.
Workflow Orchestration 01:19-02:02 In a musical orchestra, there are many different instruments that are brought together to create something wonderful. Members follow what's written on their music sheets and the real-time directions given to them by the conductor. Just like a musical orchestra, workflows can be orchestrated to make them as efficient as possible so that they produce the best results.
A workflow refers to a collection of tasks that are performed in a logical sequence. Orchestration means that you plan these tasks in such a way that they're as efficient as possible.
For example, you can orchestrate different parts of your workflow, including the development, quality assurance, and security. Orchestration often incorporates the use of tools that automatically complete certain tasks in a sequence.
SOAR 02:02-02:55 To help with workflow orchestration, many companies are using automated cybersecurity solutions that are able to quickly identify and resolve potential attacks. One of these solutions is Security Orchestration, Automation, and Response, which is known by the acronym SOAR.
SOAR refers to a collection of software programs that allow an organization's security team to collect various inputs they can monitor. The point of SOAR is to replace tasks that are repetitive and done manually with automated workflows. These systems automatically flag security incidents and respond to them in a predetermined way.
This means that these incidents are caught earlier. This also frees up security analysts to spend their time and attention on only the most advanced security threats. For example, using a firewall application, SOAR can automatically detect a brute-force login attack and block the attacker's IP address.
Continuous Integration 02:55-03:49 Another orchestration technique we see is known as continuous integration, or CI. When you work on a software development team, you often have multiple people contributing to the same project. The practice of continuous integration means that you automate all integration changes made by these contributors back into a shared mainline.
In general, there's a central repository where all code changes are merged into a single file that's used to test the current build's effectiveness. Under this strategy, developers try to merge their changes back to the main branch as often as possible. The new changes are then automatically tested to make sure they don't break the application when they're integrated. This can greatly streamline the development process because developers don't need to manually discuss the changes they make with the rest of the team. This also cuts down on overhead costs and lets developers focus on the code and not on a complex web of communication.
Continuous Delivery and Deployment 03:49-04:32 Continuous delivery is like an extension of continuous integration. It automatically deploys all the changes coders make into a production environment. You have both an automated testing process and an automated release process that you can set to occur at whichever interval you feel is best.
Continuous deployment goes a step further than continuous delivery. Continuous deployment means that any change that goes through all the production pipeline stages is automatically released to customers. There's no human intervention in this process, and only if the change fails one of the tests along the way is it prevented from being pushed out. This has the potential to create an extremely streamlined process and quicker responses to customer feedback.
Summary 04:32-05:26 That's it for this lesson. In this lesson, we first discussed DevSecOps and explained how this philosophy adds security measures to every step of the development process. Next, we discussed workflow orchestration and showed how it can help the development process run more smoothly with all the different elements aligned in the most efficient way. A big part of this is automating whichever elements can possibly be automated.
We also introduced you to some tools to help in your orchestration, including SOARS, which lets the security team automatically collect data to help them identify security threats. And finally, we discussed how continuous integration lets you automate the integration of changes from multiple developers into a central staging area. Lastly, we showed that you can use continuous deployment to ensure changes are automatically deployed to a production environment and pushed out to end users.
8.2.2 SOAR Facts
Automation tools add the characteristic speed and efficiency of computers to incident response capabilities. Automated responses can be as simple as scheduled tasks designed to look for predefined attributes, such as log file entries, data content matches, or output from scripted tasks. These methods can be highly effective at detecting change and can include changes to the environment in response.
A more sophisticated implementation involves using security orchestration, automation, and response (SOAR) platforms, typically integrated with SIEM. A SOAR platform can leverage the work done by SIEM to process log data and identify events and then perform the next steps that a human analyst may typically require.
This lesson covers the following topics:
- SOAR
- Continuous integration
- Continuous delivery and deployment
SOAR
Security orchestration, automation, and response (SOAR) is designed to automate some of the routine tasks ordinarily performed by security personnel in response to a security incident. SOAR systems can quickly analyze large amounts of data. This allows the security team to detect and respond to threats quickly, even when network traffic is heavy. The following tools can be part of a SOAR solution:
- A security information and event management (SIEM) platform
- Firewalls
- Logs (user logins, network traffic, etc.)
- Intrusion detection tools
- A user behavior analytic (UEBA) system
- Cloud and SDN/SDV APIs
- Cyber Threat Intelligence (CTI) feeds
- Automated malware signature creation
The basis of SOAR is to scan security and threat intelligence data collected from multiple sources within the enterprise and then analyze it using various techniques defined via playbooks. A SOAR can also assist with provisioning tasks, such as creating and deleting user accounts, making shares available, or launching VMs from templates.
An incident response workflow is a classic example of a SOAR task defined within a playbook. A playbook contains a checklist of actions to perform in response to a specific event. A playbook should be made highly specific by including the query strings and signatures that will detect a particular type of incident. A playbook may account for compliance concerns, such as breach notification requirements, including when and to whom notification must be made.
When a playbook utilizes a high degree of automation from a SOAR system, it can be referred to as a runbook, though the terms are also widely used interchangeably. A runbook aims to automate as many of the playbook's stages as possible while incorporating clearly defined interaction points for human analysis. These interaction points should present contextual information and guidance needed for an analyst to make a quick, informed decision about the best way to proceed with incident mitigation.
- For example, a runbook may use integrations for cloud-based email platforms and anti-malware solutions. The runbook may take email attachments from user emails and submit them to a crowdsourced detection engine such as VirusTotal. If VirusTotal identifies the file as malicious, the SOAR can provide a new custom detection signature to the anti-malware software to locate and block any other instances of the malware.
SOAR platforms also typically provide a case management interface used by security analysts to manage and document their work. The case management feature operates similarly to a help desk ticketing solution. The case management feature allows analysts to search for specific events and filter/sort events by specific attributes, such as severity level.
Continuous Integration
Another orchestration technique is continuous integration (CI). The practice of continuous integration automates the integration of all changes made by contributors into a central repository. All code changes are merged into a single file. The central repository is used to test the current build.
Using this strategy, developers merge changes into the repository as quickly as possible. The new changes are automatically tested to ensure they do not break the application. This can streamline the development process because it eliminates the need for developers to discuss the changes in person with the rest of the team.
Continuous Delivery and Deployment
Continuous delivery is an extension of continuous integration. It automatically deploys into the production environment all changes coders make. There is an automated testing and release process that you can set to occur at a specified interval.
Continuous deployment goes one step further. Continuous deployment automatically releases to customers all the production pipeline stages. There is no human intervention in this process. If the change fails one of the tests in the process it is prevented from being pushed out. This can create an extremely streamlined process and quicker response to customer feedback.
8.2.3 DevSecOps Facts
Information technology and security operations are complex and continuously present new challenges to manage. Automation is an increasingly critical approach to protecting digital assets from malicious attackers and data breaches. Automation streamlines security operations, making them more efficient, consistent, reliable, and cost-effective.
This lesson covers the following topics:
- DevSecOps
- Streamlining operations
DevSecOps
DevSecOps is the philosophy that everyone in an organization is responsible for the security of the system. DevSecOps promotes that security is built-in and not a fence around apps. Everyone should implement security decisions within their portion of the project. If everyone in a company takes responsibility for security, the entire organization benefits, and there is greater customer software assurance.
While DevSecOps initially takes time to set up, it streamlines the process for everyone, leading to lower costs and development times. During the software development process, engineers constantly check each other's work and test the software for bugs and vulnerabilities. In this way, there are fewer vulnerabilities in the final product. The organization does fewer bug fixes and less damage control post-release.
Security automation is an important component of DevSecOps. It helps create shorter development cycles that experience fewer disruptions.
Streamlining Operations
Orchestrating threat intelligence data can help organizations streamline operations in several ways. By continuously monitoring threat intelligence feeds, organizations can automatically detect new threats and indicators of compromise (IOCs) in real-time and without manual intervention. This can improve incident response by reducing the time it takes to identify and respond to security incidents. Threat hunting can also be enhanced by orchestrating threat intelligence data, providing a more comprehensive view of the threat landscape, including emerging threats and attack trends. Additionally, aggregating threat intelligence data from multiple sources can help organizations make better decisions about their prioritization efforts, including where to allocate resources and which security controls to implement. Finally, sharing threat intelligence data across different teams and departments can improve collaboration, ensuring everyone can access the same information and work together more effectively to identify and respond to threats.
Security operations centers (SOCs) are integral to the success of an organization's information security program, as they are responsible for the detection, prevention, and response to cyber incidents across an organization's networks. Effective team coordination within a SOC is critical to the successful operation of the security operations center. The SOC team comprises network security engineers, security analysts, incident responders, and other IT professionals working together to detect and prevent cyber-attacks. The team must be able to quickly and effectively collaborate to identify and respond to potential threats. The team must also be able to work together to identify potential security incidents and develop plans for mitigating the threats. To ensure successful coordination, the SOC team needs to have strong communication skills and be able to effectively share information between team members. Additionally, the team should have established procedures for handling cyber incidents and an effective system for managing and responding to alerts. With a well-coordinated SOC team, an organization can be better prepared to detect, respond to, and prevent cyber threats.
Automation is becoming increasingly crucial in security operations centers (SOCs). Automation leverages automated scripts, specialized software tools, and data interfaces to help the security team manage activities more efficiently. Automation helps the SOC team reduce manual labor and increase accuracy and speed in detecting and responding to incidents. Automation can also reduce the costs associated with manual processes. To effectively manage automation in a SOC, the team must understand the tools used, the incidents they are dealing with, and the processes associated with responding to them. The team must also be able to work together to identify the most effective tools to use and develop procedures for responding to incidents as quickly as possible.
The following video gives an overview of implementing process improvement in security operations:
Video
Click one of the buttons to take you to that part of the video.
Implementing Process Improvement 00:00-08:00 James Stanger: You know, when it comes to security, process is so important and improving those processes is even more important. To tell us more about using automation, I've brought in Mitre's Jamie Williams. He is the Cyber Adversarial Emulation Engineer for Mitre. Did I get your title right?
Jamie Williams: You nailed it, thank you.
James Stanger: Alright.
Jamie Williams: Truly an Evangelist.
James Stanger: You know, yeah Chief Technology Evangelist may be a cool title; I think yours is far cooler. Tell us how you got started in security real quick and let's start talking about what you do to improve processes.
Jamie Williams: Okay, yeah. My security route was a little bit different. I wasn't building computers as a child, I was actually more of just kind of a curious, you know, I like math, I like engineering. So computer security has always been really fascinating to me in terms of, it's, once you get into that realm of understanding, you know, it's not just the computer doing, you know, what it needs to do, there's people trying to harm it. And there's, you know, this cat and mouse game and constant balance of trying to be more defended than your adversary.
So, just love that passion project of, you know, continuously learning and I think that's where automation comes to play in terms of your adversaries are always trying to get better. They're trying to find ways to get into your systems, to break your systems, to steal your data and they're constantly innovating. And I think as defenders, we need to at least try to keep up with that and it starts with automation, in terms of you know, we need to start looking at more the tip of the spear.
What's the best practices that we can do? But we can only really do that if we're doing the easy and the fundamental stuff well and I think that's where automation comes in as a really powerful solution in terms of, you know, kind of building that foundation so that we can start worrying about more of those next ideas and those bigger innovative kind of concepts.
James Stanger: What are the types of tasks that you emulate? For example, if you're working with a hospital, what are some of the types of things that you automate? 'Cause you're working with thousands of systems, aren't you?
Jamie Williams: Yeah. I think automation is one of those things, I think there's never really a case where it's not a good candidate to consider and I think that any, you know, something, especially when you think about automation, you start looking at things that need to scale in terms of if I'm collecting data, if I'm, you know, checking on a particular, you know, variable or a particular file or a piece of information, that's something that is a really good candidate.
But also, you know, any time we're doing like, processes or you know, we're doing something, you realize, we do it kind of the same way over and over again and we almost, feels like, there was like, I'm turning my mind off and I'm just doing the routine thing.
You know, I'm checking on this log, I'm looking for this particular, you know, variable over here [or burning] this command. Those are always those really good candidates where, hey, like, maybe we should try to automate this, take this off our plate so that we can worry about the next thing, 'cause that's exactly what our adversaries are doing, and so if we're not keeping up with that kind of progression, we're potentially falling behind the race.
So I think it's one of those, automation, I think is one of those kind of beautiful solutions where, you know, when you're doing the daily routine, you know, going through your processes, taking a step back and saying, you know, is there necessarily a shortcut or a way to fast forward this so that, you know, not only can you save a bunch of time on your side, but maybe make it more accessible for someone else?
You know, maybe someone new to the team or your hospital example, maybe we only have one person and they have ten different tasks to do, really great opportunity to maybe scale that capability.
James Stanger: And to do that, do you grab like, a team of developers to do that or sometimes there are specific tools you can use?
James Stanger: I think it starts with the business. It's understanding exactly, like you said, like what are we trying to do? Are we collecting data? Are we making sure passwords are safe? Are we deploying some type of system or software across the environment? And really understanding, you know, what is the objective?
And then to your point, the tools and the process and all the mechanics that really stitches that together kind of highlights itself, in terms of, you know, there's not just one, you know, great approach to automation, it's really understanding the use case, you know, what am I trying to achieve?
And then, you know, sometimes really the conclusion is, this isn't a good candidate for automation and there's a lot of, you know, uncertainty or maybe hiccups that could get in the way. But you really only can see that forest for the trees if you start to kind of map that out and really understand, you know, what are we actually trying to do here? 'Cause again, you don't wanna get in the way of the utility of whatever that process was actually supposed to accomplish.
James Stanger: So you'll sometimes work with IT folks or with people who aren't even IT folks to identify a process and then figure out from a security perspective what you can automate, what you can kind of, what boring you can eliminate, as it were, right?
Jamie Williams: Exactly. It's like you said, your goal with automation is to make everyone's lives easier. So again, like, thinking about the hospital example, you don't want a doctor in an operating room worrying about, you know, their passwords and trying to log into a system during that critical information or critical time versus, you know, making that, using automation and using technology to make that a seamless process where not only are they able to do their specialty and focus on their task, but they also, you have that security kind of wrapper where you've already ingrained, you know, those security foundations, principles that you're trying to instill into that process.
James Stanger: You know, to do certain things like orchestration and things like that, is that where you kind of identify if there's a certain condition, then there are certain tools that come into play there? Is that how that works?
Jamie Williams: Yeah, I think that's a good way to describe it and I think it's again, one of those cases where taking a step back, understanding the process and really those, I always like to think of those like, 'if then' scenarios, where you're orchestrating something, you know, if, it's almost a trigger to something. Or like, when do we wanna do this? When do we wanna take action? When do we not?
One of the conditions that might change the way we do this; if you're able to really get a really good picture of, you know, this is the process that we wanna do, whether it's manual or automated and if you're able to map that out very cleanly, you might have a really good opportunity, exactly as you said, trying to inject automation versus if there's a lot of confusion and chaos and you don't really understand what you're trying to do, you might just be creating a more, a bigger problem for yourself to deal with later.
You know, back to the hospital example, last thing you wanna do is deploy some automated solution and have all the doctors, nurses and staff yelling at you in this big chaotic mess that you created by trying to, you know, you were trying to do good but by way of technology, maybe created more of a problem.
James Stanger: Kind of brings a whole new meaning to the phrase, 'first do no harm,' right?
James Stanger: Exactly. [LAUGHS] Well done.
James Stanger: You know, Brian, at the end, sometimes you're asked to bring in a more visualization or a visibility to a situation - they'll call that a single pane of glass. To do that, are you using APIs often or webhooks and things like that to bring about that kind of automation?
Jamie Williams: Yeah, and I think the biggest use case exactly as you were saying, is more that defensive standpoint of, you know, as defenders we have to look across many systems. We're trying to look for bad. We sometimes know what bad looks like, sometimes we don't.
So there's a lot of automation kind of value in terms of, even in that very simple use case, you know, analyzing things without having to put it in front of a human every time. Bringing it all together, correlating it, being able to say, okay, well, maybe I see half bad over here; I'm not sure I see another half over here.
Together they're a whole, so I'm able to kind of maybe come to a better conclusion there. But I think exactly to your point, that's really only possible if you really understand the domain and understand what you're trying to automate, 'cause you're not gonna automate a process that you don't understand versus taking all that knowledge and all those lessons learned over time and baking that into something that, you know, bits and bytes and can kinda handle for you.
James Stanger: That's terrific, and so baking things in, you could bake in a threat feed, for example to get more enriched data and things like that, all sorts of things, right?
Jamie Williams: Exactly.
James Stanger: You know, Brian, thank you so much for your time, really appreciate learning more about, you know, bringing about better efficiency to our processes. Thanks, man.
Jamie Williams: Appreciate it.
8.2.4 Workflow Orchestration Facts
Orchestrating threat intelligence data is an essential strategy for staying ahead of adversaries. Data enrichment combines and analyzes data from disparate sources to better understand the threat landscape. This can involve combining different threat feeds to get a complete picture of the malicious actors, tools, and tactics that attackers use. It can also involve correlating data from multiple sources, such as network logs, endpoint data, and threat intelligence feeds, to identify and prioritize threats. By orchestrating threat intelligence data, organizations can better understand their threats and take preemptive action to protect their networks. With the right processes in place, organizations can effectively create a complete picture of the threat landscape and ensure they take the proper steps to stay ahead of cyber adversaries.
This lesson covers the following topic:
- Workflow orchestration
Workflow Orchestration
A workflow refers to a collection of tasks that are performed in a logical sequence. Orchestration means that you plan these tasks to be as efficient as possible. Orchestration often incorporates tools that automatically complete certain tasks in a sequence.
The following table describes several types of orchestration that organizations use.
| Orchestration Type | Description |
|---|---|
| Cloud orchestration | An approach that uses cloud tools to provision, start, or decommission servers, allocate storage, and enable apps to use cloud services. |
| Service orchestration | An approach that seeks to provide a complete solution for delivering a particular service. This includes gathering app requirements, developing the app, and deploying the app into production. |
| Release orchestration | A group of tools that works together to create deployment automation, pipeline management, and environment management. |
| Single pane of glass orchestration | Single pane of glass is a term used to describe a unified view of a computer network or system. It is a graphical user interface that allows network administrators to manage their entire network from one place. The user interface can include monitoring, configuration, and control of the network, its components, and related services. Single Pane of Glass Orchestration is a powerful way of managing security operations. It allows security teams to see, monitor, and control all their security systems and services in one place. By combining all security services into a "single pane of glass," security teams are better able to identify and respond to threats quickly and effectively. With this approach, security teams can automate workflows, allowing them to focus on responding to threats instead of managing multiple interfaces. It also provides real-time visibility into security incidents and events, simplifying the process of responding to and resolving them. Single Pane of Glass Orchestration is an invaluable tool for improving the efficiency of an organization's security operations. |
8.2.5 Automation Technologies
Click one of the buttons to take you to that part of the video.
Automation Technologies 00:00-00:42 Sometimes, having a good tool can make all the difference. For example, if your job is to gather lumber, there's a huge difference between trying to do it with an ax or doing it with a chainsaw. With an ax, you do most of the work yourself and it takes a lot longer. With a chainsaw, the machine does most of the work for you. Automation technologies are like that as well. They're digital tools that help people perform tasks with less human effort. They make a variety of tasks easier by leveraging the power of technology to do the heavy lifting so that human workers can focus on other important tasks. In this lesson, I'll talk about a variety of automation tools that can make our lives a lot easier.
Scripts 00:42-01:16 The first tool we'll discuss is scripts. Scripts are lists of commands that can be executed by a certain program. They're what automates software programs on our computers or creates interactive web pages online. In the field of digital security, we see scripts used to automate a whole variety of different tasks. They're usually lists of commands that can be read by special runtime environments, such as JavaScript. These commands are usually interpreted, rather than compiled like regular code.
Another automation technology that works a bit like scripting is called an API. Let's take a look.
Application Programming Interface 01:16-02:28 In our complicated world of modern technology, programmers need an easy way to access data without knowing all the nitty-gritty details about each site they're working with. To make integration like this possible, programmers use APIs. API is an acronym for application programming interface. These are basically just a way for programmers to interact separately with a certain portion of a site's code.
In other words, an API is the interface that defines the interactions between multiple software intermediaries. For example, an API is responsible for such things as defining the types of requests that can be made, how to make them, the formats to use, and so forth. An API is a bit like a restaurant menu. A menu gives you, the customer, the ability to order your food without needed to know where the food comes from or how it's cooked. Developers save a lot of time by using APIs and letting the program take care of the nuts and bolts behind the scenes, as can you if you learn how to utilize them. It's important to remember that an API isn't the server itself, but a bunch of code that helps a server accomplish useful tasks.
Let's move on and talk a bit about how automation technologies help companies with their security processes.
Malware Signatures 02:28-03:03 It's in an organization's best interest to spot malware before it's able to wreak havoc on their network. Security teams use malware detection software to look for malware signatures, which are the bits of code that give the malware away as malicious. Some systems simply look for known malware, while other more modern systems try to predict what code will do before it's run and identify malicious intent before it's too late. If people had to do this process by hand, it would take much, much longer. By automating the process, employees can worry about other things while the software works to keep them secure.
Threat Feeds 03:03-03:48 Another way that companies use automation to help with security is by using threat feeds. Threat intelligence is organized and prioritized information about past, current, and potential future threats to an organization's data.
This data can include URLs, domain names, IP addresses, and file names that've proven malicious in the past. A threat feed consists of a stream of threat intelligence information fed to the company through its automated systems. You can obtain this information from free indicator feeds, bulletins, paid feeds, and other strategic partnerships. For example, organization like CERT and SANS provide open-source feeds for free. This information can help your organization's security team make decisions about how to deal with current threats.
Security Content Automation Protocol 03:48-04:30 In addition to threat feeds, there are various security standards that companies can use to automate their security. One of these is known as Security Content Automation Protocol, or SCAP. SCAP is a framework that has a wide variety of uses and fits with numerous modern standards. For example, you can use SCAP's standards to help your organization automate its security processes so that there are systems in place to look for vulnerabilities, misconfigurations, and attacks. SCAP can also be used to help security teams scan computers and other devices based on a predetermined security baseline. This lets the organization know whether they're using the proper configurations to maximize system security.
Machine Learning 04:30-05:10 Finally, let's touch on the topic of machine learning, which is a large part of many companies' automation solutions.
Combing through massive amounts of data can be a daunting task, even for a large team of human minds. That's why machine learning is such an important tool. It automates the data analysis by looking for patterns. Machine learning uses algorithms to analyze different types of data as well, which helps analysts make better sense of everything.
For example, machine learning can display tailored search engine results and social media feeds, compile lists of trends in customer preferences, or even try to sniff out security threats. This could be a large group of numbers, text strings, or even pictures.
Summary 05:10-05:43 That's it for this lesson. In this lesson, we talked about how scripts are used to automate tasks. Next, we talked about APIs and how to integrate API technology to make our systems run smoother and more efficiently. We also talked about how threat feeds can help teams make sound decisions through data compilation. We discussed the benefits of SCAP, and also how machine learning helps analysts comb through vast amounts of data to look for patterns that can be leveraged for security purposes.
8.2.6 Automation Technologies Facts
Using the proper automation technologies can make a variety of tasks easier by leveraging the power of technology to do the heavy lifting so that human workers can focus on other important tasks.
This lesson covers the following topics:
- Scripts
- Application programming interfaces (APIs)
- Malware signatures
- Threat feeds
- Machine learning
Scripts
One way to automate tasks is by using scripts. Scripts are lists of commands that can be executed by a certain program. You can use scripts to automate processes on a computer or even to create web pages online. Key points are:
- Scripts are usually lists of commands. The commands are usually interpreted rather than compiled like code.
- Scripts require a runtime environment to execute.
- The runtime environment and scripts are platform-specific.
- Code is platform-independent.
- You use a scripting language to write a script. Scripting languages include:
- PHP
- Python
- JavaScript
- Each scripting language runs in a particular runtime environment. The actions a script can perform depend on that runtime environment.
- Programming languages usually run faster than scripting languages, even though they are much more code intensive.
- There are certain things that you cannot do with a scripting language. You are limited by what kind of commands the runtime environment supports.
The runtime environment carries out the commands and not the scripting language itself.
Application Programming Interfaces (APIs)
To interact with popular security services without human engagement, an application programming interface (API) is essential. In this context, an API is a set of functions and procedures that allow two or more applications to communicate with each other. An API defines the types of calls or requests that can be made, how to make them, the data formats that should be used, and the conventions to follow. It can also provide extension mechanisms so that software can extend its existing functionality.
It is important to remember that:
- An API is a specification for interaction with other software or resources.
- APIs free a developer from having to reinvent something that has already been created and is working well. Programmers do not have to write code for basic functions such as hardware controls (keyboard, monitor, etc.) and system controls (graphical interface, file storage, file copy, etc.).
- When you use an API on your website, the API interfaces directly with another site's web server, such as Google for authentication or Paypal for payment.
Malware Signatures
Security teams can use malware detection software to look for malware signatures (bits of code that identify the code as malicious). Other systems try to predict what the code will do before it is run and identify malicious intent.
Security personnel can use anti-virus software that checks the signature or hash from each file against a list of known malicious signatures. The depth at which files are analyzed can be adjusted. The more depth the program goes into, the more time it takes to run the scan.
To reduce processing time, you can change the settings to ignore certain types of files. These file types include:
- Non-executable files
- Files over a certain size
- Files with a legitimate checksum
You can use various methods to scan for malicious code. The following table describes these methods.
| Method | Description |
|---|---|
| String scanning method | A search for a sequence of bytes/strings that indicate a specific virus and are not likely to be found in other apps or programs. |
| Wildcards method | A search that allows certain bytes or byte ranges to be skipped. |
| Generic detection method | A search that uses a common string to detect known variants of a virus family. |
| Bookmarks method | A search that calculates the distance between the start of the virus's body and the detection string. |
| Skeleton detection method | A search that parses the statements in a virus and drops off all non-essential elements. |
Threat Feeds
Threat intelligence organizes and prioritizes information about past, current, and potential future threats to an organization's data. This data can include URLs, domain names, IP addresses, and file names proven malicious in the past. A threat feed consists of a stream of threat intelligence information fed to the company through its automated systems. This information can help an organization’s security team make decisions about how to deal with current threats and how to prevent future ones.
The threat feed looks at both external and internal threats. External threats are perpetrated by those outside of an organization. Internal threats are attacks from someone within an organization. This could be someone disgruntled with the organization or someone who has been bribed or forced by an external party to harm the organization.
Red flags that might show up in an organization's threat feed include:
- Traffic to known malicious websites, strange IP addresses, and suspicious geographic locations
- Changes to user permissions
- Spikes in the amount of data downloaded
- External requests for sensitive files
- Unusual traffic on network ports
- Unexpected file location changes
- Strange patterns in DNS requests
Machine Learning
Organizations create massive databases full of user information, including preferences and activity history. This information can provide a wealth of information for different parts of the organization, such as the marketing, research, and development teams.
The volume of information generated makes machine learning an important tool. Machine learning automates the process of analyzing huge amounts of data. The data can be a large group of numbers, strings of text, or even pictures. Machine learning uses algorithms to analyze the data and help analysts access the data they need.
Machine learning can be used for a variety of purposes, such as:
- Displaying tailored search engine results.
- Tailoring a social media feed.
- Compiling a list of customer trends and preferences.
- Looking for security threats.
- Recommending products based on purchase history.
- Supporting virtual personal assistants (such as Apple's Siri and Amazon's Alexa) that use this information to make personalized recommendations.
8.2.7 Security Content Automation Protocol Facts
Security Content Automation Protocol (SCAP) describes a suite of interoperable specifications designed to standardize the formatting and naming conventions used to identify and report on the presence of software flaws, such as misconfigurations and vulnerabilities.
This lesson covers the following topics:
- Security Content Automation Protocol (SCAP)
- SCAP languages
- SCAP identification schemes
Security Content Automation Protocol (SCAP)
A SCAP validation program is a program that runs a test to determine whether a system can employ SCAP standards. SCAP-validated applications often scan systems and compare them against open cybersecurity standards. SCAP reports back with a score indicating how well the system is doing. The score can be a starting point for organizations to discuss the organization's security seriously and make a plan for improving security.
SCAP Languages
The following table describes the different languages used by SCAP:
| SCAP Language | Description |
|---|---|
| Open Vulnerability and Assessment Language (OVAL) | OVAL is an XML schema maintained by MITRE for describing system security state and querying vulnerability reports and information. It helps describe three main aspects of an evaluated system, including:
Using OVAL provides a consistent and interoperable way to collect and assess information regardless of the security tools used. |
| Asset Reporting Format (ARF) | As the name suggests, ARF helps to correlate reporting formats to assess information independently from any specific application or vendor product for consistency and interoperability. |
| Extensible Configuration Checklist Description Format (XCCDF) | Written in XML, XCCDF provides a consistent and standardized way to define benchmark information as well as configuration and security checks to be performed during an assessment. |
SCAP Identification Schemes
The following table describes the identification schemes used by SCAP to identify vulnerabilities:
| Heading 1 | Heading 2 |
|---|---|
| Common Platform Enumeration (CPE) | CPE is a scheme for identifying hardware devices, operating systems, and applications developed by MITRE. It uses a syntax similar to Uniform Resource Identifiers (URI). |
| Common Vulnerabilities and Exposures (CVE) | CVE is a scheme developed by MITRE and adopted by NIST for identifying vulnerabilities. CVE is a list of records where each item contains a unique identifier used to describe publicly known vulnerabilities. Unique identifiers begin with CVE, followed by the year of identification and a unique number - CVE-YEAR-#####. |
| Common Configuration Enumeration (CCE) | CCE is a scheme developed by MITRE and adopted by NIST for provisioning secure configuration checks across multiple sources. CCE is similar to CVE, except it is focused on configuration issues that may result in a vulnerability. |