Cyber threat intelligence – a global overview

Many businesses and organizations aim for maximum digital presence to augment and optimize visibility (effectively reach the desired customers), as well as maximize it from the current digitalization age. For that, they are regularly exposed to cyber threats and attacks based on the underlying attack surface – the organization's size, architecture, applications, operating systems, and more.

Threat intelligence allows businesses to collect and process information in such a way as to mitigate cyberattacks. Hence, businesses and organizations have to protect themselves against threats, especially human threats. Cyber threat intelligence (CTI), as approached in this book, consists of intelligent information collection and processing to help organizations develop a proactive security infrastructure for effective decision making. When engaging in a CTI project, the main threats to consider are humans, referred to as adversaries or threat actors. Therefore, it is essential to understand and master adversaries' methodologies to conduct cyberattacks and uncover intrusions. Tactics, techniques, and procedures (TTPs) are used by threat actors. By doing so, organizations aim for cyber threats from the source rather than the surface. CTI works on evidence, and that evidence is the foundation of the knowledge required to build an effective cyber threat response unit for any organization.

Many organizations regard threat intelligence as a product that allows them to implement protective cyber fences. While this is true, note that threat intelligence hides an effective process behind the scenes to get to the finished package. As the intelligence team implements mechanisms to protect against existing and potential threats, adversaries change tactics and techniques. It becomes crucial for the intelligence team to implement measures that allow new threats to be analyzed and collected. Hence, the process becomes a cycle that is continually looked at to ensure that the organizations are not only reactive but proactive as well. The term threat intelligence life cycle is used to define the process required to implement an efficient cyber threat intelligence project in an organization. The following diagram shows this process:

Figure 1.1 – Threat intelligence life cycle

Threat intelligence is an ongoing process because adversaries update their methods, and so should organizations. The CTI product's feedback is used to enrich and generate new requirements for the next intelligence cycle.

Characteristics of a threat

Understanding what a threat is helps organizations avoid focusing on security alerts and cyber events that may not be a problem to the system. For example, a company running Linux servers discovers a .exe trojan in the system through the incident management tool. Although dangerous by nature, this trojan cannot compromise the company's structure. Therefore, it is not a threat. As a security intelligence analyst, it is vital to notify the system manager about the file's low priority level and its inability to infect the network. Secondly, government agencies are one of the highest adopters and owners of cyber projects. Governments have the tools and the knowledge necessary to attack each other. However, to avoid a cyberwar and ruin their friendship, the Canadian and American governments have no intention of attacking each other. Thus, they are not a threat to each other. If one party announces a spying tool's design, that does not mean that it wants to use it against another. Although there is the capability of spying, there might be no intent to do so. Therefore, one is not always a threat to another. Lastly, you can have the capability and the intent, but would need the opportunity to compromise a system.

Therefore, we can summarize a threat as everything or everyone with the capability, the intent, and the opportunity to attack and compromise a system, independent of the resource level. When the intelligence team performs threat analysis, any alert that does not meet these three conditions is not considered a threat. If any of these three elements is missing, the adversary is unlikely to be considered a threat.

Threat intelligence and data security challenges

Organizations face a lot of challenges when it comes to data protection and cybersecurity in general. Those challenges are located in all the functional levels of the organization. There are several challenges, but the most common ones include the following:

• The threat landscape: In most cases, cyberattacks are orchestrated by professionals and teams that have the necessary resources and training at their disposal. This includes state-sponsored attacks. However, with access to specific tools and training, private groups have developed sophisticated ways to conduct destructive cyberattacks. The landscape of threats is growing and changing as adversaries rely on new exploits and advanced social engineering techniques. McAfee Labs reported an average of 588 threats per minute (a 40% increase) in the third quarter of 2020, while Q3 to Q4 2020 saw more than a 100% increase in vulnerabilities and more than a 43% increase in malware.

Targeted attacks such as ransomware were the main concern for organizations in 2020, with more than a 40% increase by the end of the year (https://www.mcafee.com/enterprise/en-us/assets/reports/rp-quarterly-threats-apr-2021.pdf). Approximately 17,447 vulnerabilities (CVEs) were recorded in 2020, with more than 4,000 high-severity ones (https://www.darkreading.com/threat-intelligence/us-cert-reports-17447-vulnerabilities-recorded-in-2020/d/d-id/1339741). Thus, the threat landscape presents a dangerous parameter for organizations that have most of their resources, assets, services, and products on the internet. And understanding the threat landscape facilitates the risk mitigation process. Personal information is one of the most targeted components on the internet – Personally Identifiable Information (PII), payment card data, and HIPAA data, to name a few.

• Security alerts and data growth: Organizations are acquiring different security platforms and technologies to address security concerns and challenges – sandbox, firewalls, incident response, threat hunting, fraud detection, intrusion detection, network scanners, and more. According to an IBM study, an average IT company possesses 85 general security tools from at least 25 vendors. In most cases, those tools are not integrated across all teams. They have different security requirements. Each tool generates security alerts of different levels, and in most cases, security professionals rely on manual processes or external automation tools (with limited functionalities) to aggregate, clean, correlate, analyze, and interpret the data. The more tools an organization has, the more data is being collected, and the more exhausted and overwhelmed the security analysts become when having to mine the voluminous data. There is then a high chance of not using data effectively, thereby missing out on critical alerts. Having a high volume of alerts and data makes it difficult, if not impossible. for a human to handle correctly. This is known as visibility loss.
• Operational complexity: The core business components may involve several organizational departments that interact with different applications to reach their goals. The embrace of big data and the adoption of cloud technologies have facilitated the management of IT infrastructures. However, it has also opened doors to more attack points as cloud security is becoming a hot topic. This is because many third-party tools, resources, and suppliers (which also have their own vulnerabilities) are used to address the security problem. Third-party tools are somehow not transparent to the organization where they are installed because most of the processes happening in the backend are not exposed to the consumers. Therefore, they increase operational complexity, especially regarding ownership of each security aspect (such as incident management, intrusion detection, traffic filtering, and inspection). Policies and procedures must be set if organizations wish to have useful data security solutions. Organizations must find ways to regulate the authority of third-party and other external tools internally.
• New privacy regulations: New requirements are frequently put in place to address data security and privacy concerns worldwide. Regulations are used to enforce the law. However, as the number of regulations increases for different industries – medical, financial, transportation, retails, and so on – them overlapping becomes a challenge as organizations must comply with all policies. Should an organization fail to comply with regulations, penalties could be imposed independently of a breach's presence or absence. This is why it's important to have security solutions that are regulation-compliant.

Nevertheless, different regions and agencies have different security policies that need to be followed. A typical example is the European Union's General Data Protection Regulation (GDPR), which is used to protect EU citizens' privacy and personal information. The GDPR applies to the EU space, which means any organization (independent of its origin, EU or not EU) operating or rendering services in the EU region needs to comply with the GDPR. Tradecrafts and standards will be explained in Chapter 4, Cyber Threat Intelligence Tradecraft and Standards. Another example is the South African Protection of Personal Information (POPI) Act, which protects South African citizens' privacy and how their personal information is handled. Complying with such policies can be challenging, and organizations need to ensure compliance with regulations.

• Cybersecurity skills gap: As organizations grow, manual processes become a challenge, and the lack of a workforce manifests. According to the ISC2 2019 report (https://bit.ly/2Lvw7tr), approximately 65% of organizations have a shortage of cybersecurity professionals. Although the gap is being reduced over the years, the demand for cybersecurity professionals remains high. And that is a big concern. The job concerns relating to cybersecurity professionals, as reported by ISC2, are shown in the following diagram:

Figure 1.2 – ISC2 job concerns among cybersecurity professionals

Organizations spend more time dealing with security threats than training or equipping the team with the necessary knowledge. Adversaries keep on attacking and breaking through conventional security systems daily. This is why there is a great demand for cybersecurity professionals worldwide who are compliant with the industry standards and methods who are dependable, adaptable, and, most importantly, resilient. Organizations need to invest in empowering and training individuals in the field of cybersecurity and threat intelligence.

Importance and benefits of threat intelligence

Cyber threat intelligence (CTI) addresses the aforementioned challenges by collecting and processing data from multiple data sources and providing actionable, evidence-based results that support business decisions. Using a single platform (for correlation, aggregation, normalization, analysis, and distribution) or a centralized environment, CTI analyzes data and uncovers the essential patterns of threats – any piece of data that has the capability, the intent, and the opportunity to compromise a system.

CTI consolidates an organization's existing tools and platforms, integrates different data sources, and uses machine learning and automation techniques to define context regarding indicators of compromise (IoCs) and the TTPs of adversaries. Intelligence analysts and security professionals rely on IoCs to detect threat actors' activities. Therefore, the types of indicators that are selected are critical during intelligence execution. This is because they determine the pain it can cause adversaries or threat actors when IoCs access is denied. This is known as the pyramid of pain and provides correlations between indicator types and pain levels. This pyramid is shown in the following diagram:

Figure 1.3 – ISC2 job concerns among cybersecurity professionals

Hash algorithms provide unique ways to obfuscate information. Hash indicators can be used to detect unique threats (such as malware) and their variants since a change in information results in a complete change in hash. Therefore, it is easy for adversaries to change malware hash values, for example. IP addresses are one of the popular indicators used to detect threats. An analyst can spot malicious activities using IP addresses. However, they can be changed easily.

An adversary can use proxy and TOR services to modify the IP addresses constantly. Domain names are also prevalent indicators as they can be used to spot malicious domain names. However, changing domain names requires a bit of effort (registration, payment, and hosting). Because there are many free hosting domains, adversaries can simply change a domain.

Changing domains takes a while. Hence, it is not as easy as changing IP addresses. Network and host artifacts are also important indicator types. Once professional security changes the network and host information, adversaries are forced to review and reconstruct their attacks (most attack networks and hosts). Hence, changing the host and network artifacts annoys the adversary.

The next indicator type is tools, and they detect the kind of tools that adversaries use to orchestrate attacks. When the intelligence analyst can detect threat actors' tools and their artifacts, this means the adversaries in question have no other option than to change the tool or create a new one completely (this takes time and money for the adversaries). Hence, making changes to tools challenges the adversary enormously.

At the top of the pyramid is TTP. At this level, any detection from the analyst results in a complete reinvention from the adversaries because, at this level, the intelligence analysts operate on the behavior, not just the tool – the higher the operating level of intelligence, the more difficult it is for adversaries to compromise the system. More details on IoCs will be provided in Chapter 13, Threat Intelligence Metrics, Indicators of Compromise, and the Pyramid of Pain.

CTI helps organizations protect revenue and measure the efficiency of the entire security infrastructure. By integrating CTI in the business processes, organizations can create a positive return on investment in the short term. Data breaches can be costly in terms of financial implications, brand reputations, and business situations. Hence, CTI is an essential aspect of revenue protection and generation.

Threat intelligence is considered an intricate domain of exclusive analysts. However, threat intelligence analysts conduct CTI projects for others – to secure other people's infrastructures. Hence, it adds value to the functions of any organization. From small businesses to large corporations, governments, and threat actors, everyone is a benefactor of threat intelligence. CTI should not be considered a separate entity of the security components, but it should be a central element of every existing security function, as we will see in the coming chapters. The main reason for this is that the CTI project's output should be shareable and accessible across all the organization's security functions.

By now, every organization or individual should be able to do the following:

• Define threat intelligence and identify real threats by focusing on their characteristics.
• Enumerate and identify the challenges related to data security and threat intelligence.
• Understand the reason to integrate CTI as an essential business component.

Now that we have understood and mastered what CTI is all about, it is vital to understand and master the cyclic process of CTI and how business functions fit each step.

Planning, objectives, and direction

The planning step is the most critical step of a CTI project's integration. It is the main ingredient of the success or failure of a CTI project. If planning is not done properly and the objectives are not set reasonably, a threat intelligence project will likely fail. The planning and direction step can be segmented into two main objectives and three fundamental phases.

CTI main objectives

Any organization or individual who wants to implement threat intelligence must start by asking the right question: why do I want a CTI team? Planning a CTI program comes down to the objectives and goals of the CTI project. The answer to this question will define the purposes of the threat intelligence team. According to the SANS FOR578, a CTI team's primary function in an organization involves providing threat preventive measures, incident response, and strategic support:

• Preventive measures: Threat intelligence analysts who are part of a team can provide tremendous support to the security operations centers (SOCs). The SOC teams deal with frequent threat monitoring systems and are flagged continuously with alerts and issues. Because many processes are done manually in the legacy security system, it can be cumbersome for the SOC team to prioritize alerts or manage critical adversaries.

Because threat intelligence is based on a centralized approach, a CTI team adds more value to the organization's SOC by filtering and prioritizing alerts, expanding and enriching indicators of compromise (IoC), and extracting the correct information that's used to assess the system's efficacity.

• Incident response unit: In many organizations, the SOC team is separated from the incident response team. Threat intelligence can help the IR team respond to threats, consolidate the information, and share and benchmark threats against what is happening in other organizations. CTI is also about sharing information – the existence of security blogs, newspapers, and so on. By knowing what happened in the past in other organizations, threat analysts can improve the IR team's efficiency when dealing with known or unknown adversaries.
• Strategic support unit: At a strategic level, threat intelligence supports stakeholders' business decisions based on evidence and actionable facts or events. Strategic intelligence is the best way to keep an organization informed of the current and prospect threats landscape and their potential impact on the business. CTI also exposes the current resource situation of an organization to the stakeholders. For example, it can advise on the types of people that need to be acquired for the threat intelligence team or the best training or skills required to mitigate specific threats.

Another goal of the first process is to position the threat intelligence team within the organization, which will be detailed in the next chapter. Nevertheless, it is essential to know how the CTI team will work with other security functions such as SOC, incident response, malware analysis, and risk assessment. Threat intelligence has to work with all security functions to facilitate the unit's analysis process and information sharing.

The CTI team's objectives must be set in such a way that they match the organization's core business or values. And they must be set to reduce the time to respond or mitigate threats and minimize the negative impact on business operations while maximizing profit.

CTI planning and direction – key phases

When planning and setting a CTI team's direction, it is also crucial to look at its operational plan. There are three main operational planning phases in threat intelligence implementation: intelligence requirements collection, threat modeling, and intelligence framework selection. Including these three phases in the first step increases the chances to succeed in the threat intelligence implementation.

Each of these phases will be discussed as separate chapters in this book:

• Intelligence requirements collection: In this phase, the CTI team collects the requirements from each business function to create a database of requests and pain points that need to be addressed. This phase can be achieved through a set of single facts or activities. It is necessary to avoid open-ended questions as the CTI results need to be specific and evidence-based. The requirements need to be collected at each business level: strategic, operational, and tactical.
• Threat modeling: When planning for a CTI project or implementing an intelligence team, it is essential to evaluate all the assets that an adversary will target. Threat modeling involves identifying the organization's principal assets and performing a reconnaissance of the adversary. Using past information can help model threats using functional activities such as financial data, personal information, and intellectual property data.
• Intelligence framework selection: To effectively produce intelligence, threat analysts need to collect the data, process it, and deliver the output transparently. It is essential to project how data will be used to provide the desired answers. Intelligence framework selection is a critical parameter when producing intelligence. It gives insight into the different data sources (internal and external) and how the data is exploited to produce intelligence. An intelligence framework should fulfill a certain number of criteria, which will be detailed in Chapter 3, Cyber Threat Intelligence Frameworks. However, the main tip is to select a framework that provides an end-to-end view of the available data (external and internal).

Now let's take a look at the consumers of the results.

Determining the consumers of the results

During the planning phase, the threat analysts should also determine the consumers of the end products. Although CTI is beneficial to all, identifying the major players will help determine which area to focus on. For example, will the intelligence product be sent to the cybersecurity analysts (more technical and hands-on professionals), or will it be sent to the executives who focus on a global overview of the organization's security status to justify the investment in the project or the team?

The planning and direction of threat intelligence is summarized in the following diagram:

Figure 1.4 – Threat intelligence planning and direction summary

The CTI team and the organization security teams must use the layout shown in the preceding diagram to conduct the planning and direction phase. The output of this will drive the data collection phase. If we know the organization's security weaknesses, the assets to protect, and the possible threats to the security system, we will be able to acquire the correct intelligence data.