To understand your security architecture requirements, you must first ensure that you have a solid understanding of the IT environment that you are trying to protect. Before deploying any new security solution, there is a need to map out the solutions that are currently deployed and how they protect each area of the IT environment. The following list provides the major components of any modern IT environment:

• End user habits that are counter-productive to security endeavors
• Identity for the authentication and authorization of access to systems
• Networks to gain access to internal resources and the internet
• Storage and compute in the data center for internal applications and sensitive information
• End user devices and the applications they use to interact with data
• And in some environments, you can include Industrial Control Systems (ICS) and the Internet of Things (IoT)

When we start to look at the threats and vulnerabilities for these components, we quickly find ourselves deep in the alphabet soup of problems and solutions.

Figure 1.1 – The alphabet soup of cybersecurity

This is by no means an exhaustive list of the potential acronyms available. Understanding these acronyms is the first hurdle; matching them to the appropriate solutions and ensuring they are well deployed is another challenge altogether (a table of these acronyms can be found in the appendix of this book).

To assist with the discovery and mapping of current security solutions, we developed the cloud security reference framework. The following diagram is a section of this framework that provides the technical mapping components, and you can use this to carry out a mapping of your own environment:

Figure 1.2 – Technical mapping components; the cloud security reference framework

Each of these 12 components is described in the following list, along with some examples of the types of solutions to consider as they relate to integration with Microsoft Sentinel and the rest of your security architecture:

1. Security Operations Center: At a high level, this includes the following technologies and procedures: log management and Security Incident and Event Monitoring (SIEM), Security Orchestration and Automated Response (SOAR), vulnerability management, threat intelligence, incident response, and intrusion prevention/detection. This component is explored further in the Mapping the SOC architecture section later in this chapter.
2. Productivity Services: This component covers any solution currently in use to protect the business productivity services that your end users rely on for their day-to-day work. This may include email protection, SharePoint Online, OneDrive for Business, Box, Dropbox, Google apps, and Salesforce. Many more will appear in the future, and most of these should be managed through a Cloud Access Security Broker (CASB) solution.
3. Identity and Access Management: Identities are among the most important entities to track. Once an attacker gains access to your environment, their main priority is to find the most sensitive accounts and use them to exploit systems further. In fact, identity is usually one of the first footholds in your IT environment, usually through a successful phishing attack. A simple resolution is to implement multi-factor authentication, ensuring that even if a password is stolen (or guessed), the attacker would need multiple attempts to access the system.
4. Client Endpoint Management: This component covers a wide range of endpoints, from desktops and laptops to mobile devices and kiosk systems, all of which should be protected by specialized solutions such as Endpoint Detection and Response (EDR), Mobile Device Management (MDM), and Mobile Application Management (MAM) solutions to ensure protection from advanced and persistent threats against the operating systems and applications. This component also includes secure printing, managing peripherals, and any other device that an end user may interact with, such as the future of virtual reality/augmentation devices.
5. Cloud Access Security Broker (CASB): This component has been around for several years and is finally becoming a mainstay of modern cloud security infrastructure due to the increased adoption of cloud services. The CASB is run as a cloud solution that can ingest log data from Software as a Service (SaaS) applications and firewalls and will apply its own threat detection and prevention solutions. Information coming from the CASB will be consumed by the SIEM solution to add to the overall picture of what is happening across your diverse IT environment.
6. Perimeter Network: One of the most advanced components, when it comes to cybersecurity, must be the perimeter network. This used to be the first line of defense and for some companies still is the only line of defense. That is changing now, and we need to be aware of the multitude of options available; from external-facing advanced firewalls, web proxy servers, and application gateways to virtual private networking solutions and secure DNS, this component will also include protection services such as Distributed Denial of Service (DDoS), Web Application Firewall (WAF), and intrusion protection/detection services.
7. IoT and Industrial Control Systems: ICS are usually operated and maintained in isolation from the corporate environment, known as the Information Technology/Operational Technology (IT/OT) divide. These are highly bespoke systems that may have existed for decades and are not easily updated or replaced. The networks and devices may be highly sensitive to any latency or attempts to scan; instead, the recommended approach is passive monitoring of network traffic.

   The reference to IoT is different, yet similar; in these systems, there will be a lot of small devices that collect data and control critical business functions without working on the same network. Some of these devices can be smart to enable automation; others are single-use (vibration or temperature sensors). The volume and velocity of data that can be collected from these systems can be very high. If useful information can be gained from the data, then consider filtering the information before ingesting it into Microsoft Sentinel for analysis and short- or long-term retention.

8. Private Cloud Infrastructure: This may be hosted in local server rooms, a specially designed data center, or hosted with a third-party provider. The technologies involved in this component will include storage, networks, internal firewalls, and physical and virtual servers. The data center has been the mainstay of many companies for the last 2-3 decades, but most are now transforming into hybrid solutions, combining the best of cloud (public) and on-premises (private) solutions. The key consideration here is how much of the log data you can collect and transfer to the cloud for Microsoft Sentinel ingestion. We will cover data connectors more in Chapter 3, Managing and Collecting Data.

   Active Directory is a key solution that should also be included in this component. It will be extended to public cloud infrastructure (component 09) and addressed in the Privileged Access Management section (component 10). The best defense for Azure Active Directory is to deploy the Microsoft Defender for Identity solution, which Microsoft developed to specifically protect Active Directory domain controllers.

9. Public Cloud Infrastructure: These solutions are now a mainstay of most modern IT environments, beginning either as an expansion of existing on-premises virtualized server workloads, a disaster recovery solution, or an isolated environment created and maintained by the developers. A mature public cloud deployment will have many layers of governance and security embedded into the full life cycle of creation and operations. This component may include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) services; each public cloud service provider offers its own security protections that can be integrated into Microsoft Sentinel.
10. Privileged Access Management: This is a critical component, not to be overlooked, especially in terms of gaining access to the SOC platform and associated tools. The Privileged Access Management (PAM) capability ensures all system-level access is highly governed, removing permissions when not required, and making a record of every request for elevated access. Advanced solutions will ensure password rotation for service accounts, management of shared system accounts (including SaaS services such as Twitter and Facebook), and the rotation of passwords for the local administrator accounts on all computers and servers. For the SOC platform, consider implementing password vaults and session recording for evidence gathering.
11. Cloud Workload Protection Platform: This component may also be known as Cloud Security Posture Management (CSPM), depending on the view of the solution developed. This is a relatively new area for cloud security and is still maturing.

    Whatever they are labeled as, these solutions address the same problems: how do you know that your workloads are configured correctly across a hybrid environment and protect the resources within each of those environments? This component will also include any DevOps tools implemented to orchestrate the deployment and ongoing configuration management of solutions deployed to private and public cloud platforms. This solution should be capable of continuously scanning for, and potentially enforcing, configuration compliance with multiple regulatory and industry-standard frameworks.

12. Information Security: This component is critical to securing data at rest and in transit, regardless of the storage: endpoint, portable, or cloud storage. This component is important to cover secure collaboration, digital rights management, securing email (in conjunction with component 02, (Productivity Services), scanning for regulated data, and other sensitive information.

The cloud security reference framework is meant to be a guide to what services are needed to secure your cloud implementation. In the next section, we will look at the SOC in more detail.

As described earlier, the SOC platform includes a range of technologies to assist with the proactive and reactive procedures carried out by various teams. Each of these solutions should help the SOC analysts to perform their duties at the most efficient level to ensure a high degree of protection, detection, and remediation.

The core components of the SOC include log management and SIEM, SOAR, vulnerability management, threat intelligence, and incident response. All these components are addressed by the deployment of Microsoft Sentinel. Additional solutions will be required, and integrated, for other SOC platform capabilities such as intrusion prevention/detection, file integrity monitoring, and disaster recovery.

An SOC deployment using Microsoft Sentinel comprises the following components:

• Azure Monitor Log Analytics workspaces are created for data collection and analysis. These were originally created to ensure a cloud-scale log management solution for both cloud-based and physical data center-based workloads. Once the data is collected, a range of solutions can then be applied to analyze the data for health, performance, and security considerations. Some solutions were created by Microsoft, and others were created by partners.
• Microsoft Sentinel was developed to address the need for a cloud-native solution as an alternative to existing server-based SIEM solutions that have become a mainstay of security and compliance over the last decade. Microsoft Sentinel is built upon the existing services of Azure Monitor and Log Analytics. It is also integrated with other services such as Logic Apps and Azure Data Explorer.

  The popularity of cloud services provides some key advantages, including reduced storage costs, rapid scale compute, automated service maintenance, and continuous improvement as Microsoft creates new capabilities based on customer and partner feedback.

  One of the immediate benefits of deploying Microsoft Sentinel is rapid enablement without the need for costly investment in the supporting infrastructure, such as servers, storage, and complex licensing. The Microsoft Sentinel service is charged based on data consumption, per gigabyte per month. This allows the initial deployment to start small and grow as needed until full-scale deployment and maturity can be achieved.

  Ongoing maintenance is also simplified as there are no servers to maintain or licenses to renew. You will want to ensure regular optimization of the solution by reviewing the data ingestion and retention for relevance and suitability. This will keep costs reasonable and improve the quality of data used for threat hunting.

• Logic Apps provides integration with a vast array of enterprise solutions, ensuring workflows are connected across the multiple cloud platforms and to existing on-premises solutions. This is a core part of the integration and automation (SOAR) capabilities of the platform.

Logic Apps is a standards-based solution that provides a robust set of capabilities. You can also use third-party SOAR solutions if you have already invested in one of those platforms.

The SOC platform components are a starting point, but there may be several other services you will want to deploy in your SOC implementation. In the next section, we will look at an approach to mapping the SOC architecture's current state and requirements.

To implement a cohesive technical solution for your SOC platform, you will need to ensure that the following components are reviewed and thoroughly implemented. This is best done on a routine basis and incorporates regularly testing for the strength of each capability using penetration testing experts that will provide feedback and guidance to help improve any weaknesses.

Log management and data sources

The first component of an SOC platform is the gathering and storing of log data from a diverse range of systems and services across your IT environment. This is where you need careful planning to ensure that you are collecting and retaining the most appropriate data. Some key considerations we can borrow from other well-documented big data guidance are listed here:

• Variety: You need to ensure you have data feeds from multiple sources to gain visibility across the spectrum of hardware and software solutions across your organization.
• Volume: Too large a volume and you could face some hefty ingestion and storage fees; too small and you could miss some important events that may lead to preventing you from fully analyzing a breach.
• Velocity: Collecting real-time data is critical to reducing response times, but it is also important that the data is processed and analyzed in real time too.
• Value/veracity: The quality of data is important to understand the meaning; too much noise will hamper investigations.
• Validity: The accuracy and integrity must be verified to ensure that the right decisions can be made.
• Volatility: How long is the data useful for? Not all data needs to be retained long term; once analyzed, some data can be dropped quickly.
• Vulnerability: Some data is more sensitive than other data, and when collected and correlated together in one place, can become an extremely valuable data source to a would-be attacker.
• Visualization: Human interpretation of data requires some level of visualization. Understanding how you will show this information to the relevant audience is a key requirement for reporting.

Microsoft Sentinel provides a range of data connectors to ensure that all types of data can be ingested and analyzed. Securing Azure Monitor will be covered in Chapter 2, Azure Monitor – Introduction to Log Analytics, and connector details will be available in Chapter 3, Managing and Collecting Data.

Operations platforms

Traditionally, a SIEM was used to look at all log data and reason over it, looking for any potential threats across a diverse range of technologies. Today, there are multiple platforms available that carry out self-monitoring and alerting functionality, like the way a SIEM would work, except they are designed with a specific focus on a particular area of expertise. Each platform may carry out its own log collection and analysis, provide specific threat intelligence and vulnerability scanning, and make use of machine learning algorithms to detect changes in user and system behavior patterns. If they are advanced systems, they will also provide a level of automated response in reaction to the threats detected.

The following solutions each have a range of capabilities built in to collect and analyze logs, carry out immediate remediations, and report their findings to the SIEM solution for further investigation and cross-analysis:

• Identity and Access Management (IAM): The IAM solution may be made up of multiple solutions, combined to ensure the full life cycle management of identities from creation to destruction. The IAM system should include governance actions, such as approvals, attestation, and the automated cleanup of group membership and permissions management. IAM also covers the capability of implementing multi-factor authentication: a method of challenging the sign-in process to provide more than a simple combination of user ID and password. All actions carried out by administrators and user-driven activities should be recorded and reported to the SIEM for context and end user behavior analytics.

  Modern IAM solutions will also include built-in user behavior analytics to detect changes in baseline patterns, suspicious activities, and the potential of insider-threat risks. These systems should also be integrated with a CASB solution to provide session-based authentication controls, which is the ability to apply further restrictions if the intent changes or access to higher-sensitivity actions is required. Finally, every organization should implement privileged access management solutions to control access to sensitive systems and services.

• Endpoint Detection and Response (EDR): Going beyond anti-virus and anti-malware, a modern endpoint protection solution will include the ability to detect and respond to advanced threats as they occur. Detection will be based not only on signature-based known threats but also on patterns of behavior and integrated threat intelligence. Detection expands from a single machine to complete visibility across all endpoints in the organization, both on the network and roaming across the internet.

  Response capabilities will include the ability to isolate the machine from the network, to prevent the further spread of malicious activities, while retaining evidence for forensic analysis and providing remote access for investigators. The response may also trigger other actions across integrated systems, such as mailbox actions to remove threats that are executed via email or removing access to specific files on the network to prevent further execution of malicious code.

  Many companies have already invested in an EDR solution due to their effectiveness in reducing the risk of intrusion via advanced attacks. The trend now is to mature this implementation and focus on Extended Detection and Response (XDR) platforms: an XDR solution will include EDR, IAM, CASB, and several other solutions integrated to ensure complete attack chain detection and response capabilities.

• CASB: A CASB is now a critical component in any cloud-based security architecture. With the ability to ingest logs from network firewalls and proxy servers, as well as connecting to multiple cloud services via their APIs, the CASB has become the first point of collation for many user activities across the network, both on-premises and when directly connected to the internet. This also prevents the need to ingest these logs directly into the SIEM (saving on costs) unless there is a need to directly query these logs rather than pivoting from the SIEM to the CASB portal to carry out an investigation.

  A CASB will come with many connectors for deep integration into cloud services, as well as connection to the IAM system to help govern access to other cloud services (via Single Sign-On (SSO)), acting as a reverse proxy and enforcing session-based controls. The CASB will also provide many detection rule templates to deploy immediately, as well as offering the ability to define custom rules for an almost infinite set of use cases unique to your organization. The response capabilities of the CASB are dependent on your specific integrations with the relevant cloud services; these can include the ability to restrict or revoke access to cloud services, prevent the upload or download of documents, or hide specific documents from the view of others.

• Cloud Workload Protection Platform (CWPP): The CWPP may also be known as a Cloud Security Posture Management (CSPM) solution. Either of these will provide the unique capability of scanning and continuously monitoring systems to ensure that they meet compliance and governance requirements. This solution provides a centralized method for vulnerability scanning and for carrying out continuous audits across multiple cloud services (such as Amazon Web Services (AWS) and Microsoft Azure), while also centralizing policies and remediation actions. Resources within these services can be protected by implementing policies and technologies including Just In Time (JIT) access and Attack Surface Reduction (ASR).

  When these solutions are deployed, it is one less capability that we need the SIEM to provide; instead, it can take a feed from the service to understand the potential risk and provide an integration point for remediation actions.

• Next-Generation Firewall (NGFW): Firewalls have been the backbone of network security since the 1980s and remain a core component of the segmentation and isolation of internal networks, as well as acting as the front door for many internet-facing services. With NGFW, not only do you get all the benefits of previous firewall technologies, but now you can carry out deep packet inspection for the application layer security and integrated intrusion detection/prevention systems. The deployment of NGFW solutions will also assist with the detection and remediation of malware and advanced threats on the network, preventing the spread to more hosts and network-based systems.

As you can see from these examples, the need to deploy a SIEM to do all the work of centrally collecting and analyzing logs is in the past. With each of these advanced solutions deployed to manage their specific area of expertise, the focus of SIEM changes to look for common patterns across the solutions as well as monitoring those systems that are not covered by these individual solutions. With Microsoft Sentinel as the SIEM, it will also act as the SOAR, enabling a coordinated response to threats across each of these individual solutions, preventing the need to re-engineer them all each time there is a change in requirements for alerting, reporting, and responding.

Threat intelligence and threat hunting

Threat intelligence adds additional context to the log data collected. Knowing what to look for in the logs and how to identify serious events requires a combination of threat hunting skills and the ongoing intelligence feed from a range of experts that are deep in the field of cybercrime research. Much of this work is augmented by Artificial Intelligence (AI) platforms; however, a human touch is always required to add that gut-feeling element that many detectives and police officers will tell you they get from working their own investigations in law enforcement.

SOC mapping summary

The following diagram provides a summary of the multiple components that come together to help to make up the SOC architecture, with some additional thoughts when implementing each one:

Figure 1.3 – The SOC mapping summary

This solution works best when there is a rich source of log data streaming into the log management solution, tied in with data feeds coming from threat intel and vulnerability scans and databases. This information is used for discovery and threat hunting and may indicate any issues with configuration drift. The core solutions of the SOC operations include the SIEM, CASB, and EDR, among others, each with its own End User Behavior Analytics (EUBA) and SOAR capabilities. Integrating these solutions is a critical step in minimizing the noise and working toward improving the speed of response. The outcome should be the ability to report accurately on the current risk profile, compliance status, and clearly communicate in situations that require an immediate response and accurate data.

Microsoft Sentinel is designed to work with multiple security solutions, not just those that are developed by Microsoft.

At the most basic level, log collection and analysis are possible from any system that can transmit its logs via the Syslog collectors. More detailed logs are available from those that support CEF-encoded Syslog endpoints that share Windows event logs. The preferred method, however, is to have direct integration via APIs to enable two-way communication and help to manage the integrated solutions. More details relating to these options are included in Chapter 3, Managing and Collecting Data.

Common Event Format (CEF)

CEF is an industry-standard format applied to Syslog messages, used by most security vendors to ensure commonality between platforms. Microsoft Sentinel provides integrations to easily run analytics and queries across CEF data. For a full list of Microsoft Sentinel CEF source configurations, review the article at https://aka.ms/SentinelGrandlist.

Microsoft is continually developing integration options. At the time of writing, the list of integrated third-party solution providers includes the following:

Table 1.1 – Data connector list of companies

As you can see from this list, many of the top security vendors are available directly in the portal. Microsoft Sentinel provides the ability to connect to a range of security data sources with built-in connectors, ingest the log data, and display dashboards using pre-defined workbooks.

One of the key reasons you might be planning to deploy Microsoft Sentinel is to manage the security of your cloud platform deployments. Instead of sending logs from the cloud provider to an on-premises SIEM solution, you will likely want to keep that data off your local network, to save on bandwidth usage and storage costs.

Let's now look at how some of these platforms can be integrated with Microsoft Sentinel.

Integrating with Amazon Web Services (AWS)

AWS provides API access to most features across the platform, which enables Microsoft Sentinel to be a rich integration solution. The following list provides some of the common resources that should be integrated with Microsoft Sentinel if enabled in an AWS account(s):

• AWS CloudTrail logs provide insights into AWS user activities, including failed sign-in attempts, IP addresses, regions, user agents, and identity types, as well as potentially malicious user activities with assumed roles.
• AWS CloudTrail logs also provide network-related resource activities, including the creation, update, and deletion of security groups, network access control lists (ACLs) and routes, gateways, elastic load balancers, Virtual Private Cloud (VPC), subnets, and network interfaces.

Some resources deployed within an AWS account(s) can be configured to send logs directly to Microsoft Sentinel (such as Windows event logs). You may also deploy a log collector (Syslog, CEF, or Logstash) within an AWS account(s) to centralize the log collection, the same as you would for a private data center.

Integrating with Google Cloud Platform (GCP)

Google provides API access to most features of both GCP and the G Suite solution. G Suite Connector is currently in development. If you are managing either a G Suite or a GCP instance and want to use Microsoft Sentinel to secure them, you should consider the following options (until a fully supported connector is available):

• REST API—this feature is still in development; when released, it will allow you to create your own investigation queries.
• Deploy a CASB solution that can interact with GCP logs, control session access, and forward relevant information to Microsoft Sentinel.
• Deploy a log collector such as Syslog, CEF, or Logstash. Ensure that all deployed resources can forward their logs via the log collector to Microsoft Sentinel.

Integrating with Microsoft Azure

The Microsoft Azure platform provides direct integration with many Microsoft security solutions, and more are being added every month:

• Azure Active Directory, for collecting audit and sign-in logs to gather insights about app usage, Conditional Access policies, legacy authentication, self-service password reset usage, and the management of users, groups, roles, and apps.
• Azure Active Directory Identity Protection, which provides user and sign-in risk events and vulnerabilities, with the ability to remediate these risks immediately.
• Azure Activity, for insights into subscription-level events such as Azure Resource Manager, service health, write operations on resources, and the status of activities performed in Azure.
• Azure DDoS Protection, for the protection of web services that could be susceptible to attack through DDoS.
• Microsoft Defender, the integrated CWPP for security management across Azure, AWS, GCP, and hybrid deployments.
• Microsoft Defender for IoT, for insights into the IoT and OT networks with recommendations based on the severity of the risk.
• Azure Firewall, the managed, cloud-based network security service to protect Azure Virtual Networks.
• Microsoft Information Protection, to classify and optionally protect sensitive information.
• Azure Key Vault, for securely storing and accessing secrets including API keys, passwords, certificates, or cryptographic keys.
• Azure Kubernetes Service (AKS), an open source, fully managed container orchestration service to manage and deploy Docker containers.
• Azure SQL Database, a fully managed PaaS database engine. This connector lets you stream the audit and diagnostic logs into Microsoft Sentinel.
• An Azure storage account, a cloud solution for modern data storage scenarios.
• DNS, to improve investigations for clients that try to resolve malicious domain names, talkative DNS clients, and other DNS health-related events.
• Dynamics 365, for insights into admin, user, and support activities on this platform.
• Microsoft 365 Defender, a consolidation of multiple connectors (Endpoint, Identity, Office 365, and Microsoft Cloud App Security (MCAS)).
• Microsoft Defender for Cloud Apps, to gain visibility into connected cloud apps (SaaS), cloud services (IaaS and PaaS), and an analysis of firewall and proxy logs.
• Microsoft Defender for Endpoint, a security platform designed to prevent, detect, investigate, and respond to advanced threats across all client devices.
• Microsoft Defender for Identity, to gain visibility of the events and user analytics on Active Directory domain controllers.
• Microsoft Defender for Office 365, to provide insights into ongoing user activities, such as file downloads, access requests, changes to group events, and mailbox activity. This solution also protects advanced attacks in emails (such as phishing and whaling), Teams, SharePoint Online, and OneDrive for Business.
• Threat intelligence – TAXII, a service to ingest TAXII v2.0- and v2.1-compatible data sources to enable monitoring, alerting, and hunting using threat intelligence.
• Microsoft threat intelligence platforms, for integration with the Microsoft Graph Security API data sources: This connector is used to send threat indicators from Microsoft and third-party threat intelligence platforms.
• Windows Firewall, if enabled on your servers and clients (recommended).
• Azure WAF, to protect applications from common web vulnerabilities such as SQL injection and cross-site scripting.

Microsoft makes many of these log sources available to Microsoft Sentinel for no additional log storage charges, which could provide a significant cost saving when considering other SIEM tool options.

Other cloud platforms will provide similar capabilities, so review the options as part of your ongoing due diligence across your infrastructure and security landscape.

Whichever cloud platforms you choose to deploy, we encourage you to consider deploying suitable CWPP and CSPM solutions to provide additional protections against misconfiguration and compliance violations. These solutions can then forward events to Microsoft Sentinel for central reporting, alerting, and remediation.

In the next section, we will look at how you can integrate with private or on-premises infrastructure to ensure full coverage of your IT estate.

The primary method of integration with your private infrastructure (such as an on-premises data center) is the deployment of native connectors, where supported. The next logical step is to deploy the new Azure Monitor Agent (AMA) for those services that can support it. Otherwise, the remaining on-premises services can forward their logs' Syslog servers, which act as data collectors. While endpoints can be configured to send their data to Microsoft Sentinel directly, you will likely want to centralize the management of this data flow. The key consideration for this deployment is the management of log data volume; if you are generating a large volume of data for security analytics, you will need to transmit that data over your internet connections (or private connections such as ExpressRoute).

The Syslog data collectors can be configured to reduce the load by filtering the data, but a balance must be found between the volume and velocity of data collected to have sufficient available bandwidth to send the data to Microsoft Sentinel. Investment in increased bandwidth should be considered to ensure adequate capacity based on your specific needs.

A second method of integration involves investigation and automation to carry out actions required to understand and remediate any issues found. Automation may include the deployment of Azure Automation to run scripts, or through third-party solution integration, such as a SOAR platform, depending on the resources being managed.

Keep in mind that should your private infrastructure lose connectivity to the internet, your systems will not be able to communicate with Microsoft Sentinel during the outage. Investments in redundancy and fault tolerance should be considered.

In the next section, we will discuss the pricing options for Microsoft Sentinel.

There are several components to consider when pricing Microsoft Sentinel:

• A charge for ingesting data into Log Analytics
• A charge for running the data through Microsoft Sentinel
• Retention of data, past the initial 90-day default retention allowance
• Charges for running Logic Apps for Automation (optional)
• Charges for running your own machine learning models (optional)
• The cost of running any VMs for data collectors (optional)

The cost of Azure Monitor and Microsoft Sentinel is calculated by how much data is consumed, which is directly impacted by the connectors: which type of information you connect to and the volume of data each node generates. This may vary each day throughout the month as changes in activity occur across your infrastructure and cloud services. Some customers notice a change based on their customer sales fluctuations, or when they come under a DDoS attack.

The pricing is also influenced by how long the data is retained within Microsoft Sentinel. The default is 90 days but can be extended to up to 2 years. Most security operations require between 6 and 12 months of hot data retention. After the set retention period, use Azure Data Explorer (ADX) to retain data for as long as required (up to 99 years).

The initial pricing option is to use Pay as You Go (PAYG). With this option, you pay a fixed price per Gigabyte (GB) ingested, charged on a per-day basis. Microsoft has provided the option to commit to varying volume tiers and receive discounts in return based on larger volumes of data.

It is worth noting that Microsoft has made available some connectors that do not incur a data ingestion cost. The data from these connectors could account for 10-20% of your total data ingestion, which reduces your overall costs. Currently, the following data connectors are not charged for ingestion (generally the free ingestion is for alerts only; some connectors do provide the full data ingestion). The details are here: https://azure.microsoft.com/en-us/pricing/details/azure-sentinel/#faq.

• Azure Activity (activity logs for Azure operations)
• Azure Active Directory Identity Protection (for tenants with Azure Active Directory P2 licenses)
• Microsoft Information Protection
• Microsoft Defender
• Azure Security Center
• Microsoft 365 Defender
• Microsoft Defender for Cloud Apps
• Microsoft Defender for Endpoint
• Microsoft Defender for Office 365
• Microsoft Defender for Identity
• Office 365 audit logs (all Teams, Exchange admin, and SharePoint activity logs)

The pricing works by charging on a PAYG basis for each day, based on actual data consumption. There are capacity commitment tiers available to provide discount pricing when the volume of data ingested regularly reaches the reservation limits:

• 100 GB
• 200 GB
• 300 GB
• 400 GB
• 500 GB
• 1,000 GB (1 TB)
• 2,000 GB (2 TB)
• 5,000 GB (5 TB)

With capacity reservation, a fixed price is paid for the data each day at that tier, then charges are incurred at a PAYG price for each GB over that tier amount. The PAYG pricing is set to the same amount as the committed tier discount price. When you work out the calculations for the pricing tiers, it makes financial sense to increase to the next tier when you reach the point where the reservation is cheaper than paying PAYG pricing, which is between 50 and 80%.

For example, if you are ingesting an average of 130 GB per day, you will pay for the first 100 GB at a fixed price per GB, and then pay a PAYG price per GB for the additional 30 GB (example per day = $296). Now, if you increase your daily usage to 185 GB, you will save money by increasing your plan to the 200 GB option (example per day = $276) and paying for the extra capacity, instead of paying for the 100 GB (fixed) + 85 GB (PAYG) (total per day = $384.80).

When you look at the amount of data you are using, you may see a trend toward more data being consumed each month as you expand the solution to cover more of your security landscape. As you approach the next tier, you should consider changing the pricing model; you have the option to change once every 30 days.

The next area of cost management to consider is retention and long-term storage of the Microsoft Sentinel data. By default, the preceding pricing includes 90 days of retention. For some companies, this is enough to ensure visibility over the last 3 months of activity across their environment; for others, there will be a need to retain this data for longer, sometimes between 2 and 7 years depending on regulatory requirements in your country or industry. There are two primary ways of maintaining data long term, and both should be considered and chosen based on price and technical requirements:

• Azure Monitor: This is the native storage for Microsoft Sentinel and provides a default hot storage option of 90 days, which can be upgraded to store the hot data for up to 2 years.

  Pros: The data is available online and in Azure Monitor, enabling direct queries using KQL searches, and the data can be filtered to only retain essential information.

  Cons: This is likely the most expensive option per GB compared to the other options.

• Azure Data Explorer (ADX): This solution can maintain data indefinitely; pricing is based on a combination of the volume of data and the amount of compute required to carry out searching. Generally, this will be one-tenth of the cost of Microsoft Sentinel for long-term storage.

  Pros: The data is available online and in Azure, enabling direct queries using KQL searches. The data can be filtered to only retain essential information.

  Cons: This is a separate service and requires some initial configuration and integration effort for unsupported tables.

• Other storage options: Cloud-based or physical-based storage solutions can be used to store the data indefinitely, usually enabled by sending data via Event Hubs or Azure Storage.

  Pros: Cheaper options are available from a variety of partners.

  Cons: Additional charges will be made if data is sent outside of Azure, and the data cannot be queried by Microsoft Sentinel. Using this data requires another solution to be implemented to query the data when required.

Each of these components is highly variable across deployments, so you will need to carry out this research as part of your design. Also, research the latest region availability and ascertain whether Microsoft Sentinel is supported in the various government clouds, such as in China.

For the final section of this chapter, we are going to look at an important part of SOC development: scenario mapping. This process is carried out on a regular basis to ensure that tools and procedures are tuned for effective analysis and have the right data flow and that responses are well defined to ensure appropriate actions are taken upon detection of potential and actual threats. To make this an effective exercise, we recommend involving a range of different people with diverse skill sets and viewpoints, both technical and non-technical. You can also involve external consultants with specific skills and experience in threat hunting, defense, and attack techniques.

The following process is provided as a starting point. We encourage you to define your own approach to scenario mapping and improve it each time the exercise is carried out.

Step 1 – defining the new scenarios

In this first step, we articulate one scenario at a time; you may want to use a spreadsheet 
or other documentation methods to ensure information is gathered, reviewed, and updated as required:

• Impact analysis: This will be the summary of the complete analysis, based on the next components. You may want to provide a scoring system to ensure that the implementation of security controls is handled in priority order, based on the severity of the potential impact.
• Risk versus likelihood: While some scenarios would have a high risk of catastrophe if they were to occur, we must also balance that risk with the potential of them occurring. Risk calculations help to justify the budget and controls required to mitigate the risk but keep in mind that you are unlikely to achieve complete mitigation, and there is always a need to prioritize the resources you have, to implement the controls.
• Cost and value estimate: Estimate the value of the resource to your organization and the cost to protect it. This may be a monetary value or percentage of the IT security budget, or it could be some other definable metric such as time and effort. If the cost outweighs the value, you may need to find a more affordable way to protect the resource.
• Systems impacted: Create a list of the systems that are most likely to be targeted to get to the resources and information in one or many scenarios (primary systems) and a list of the other systems that could be used or impacted when attacking the primary systems (these are secondary systems). By understanding the potential attack vectors, we can make a map of the targets and ensure they are being monitored and protected.
• People impacted: For each scenario, list the relevant business groups, stakeholders, and support personnel that would be involved or impacted by a successful attack. Ensure that all business groups can contribute to this process and articulate their specific scenarios. Work with stakeholders and support personnel to ensure clear documentation for escalation and resolution.
• Customers impacted: For some scenarios, we must also consider the customer impact as regards the loss or compromising of their data or an outage caused to services provided to them. Make notes about the severity of the customer impact, and any mitigations that should be considered.

Step 2 – explaining the purpose

For each scenario, we recommend providing a high-level category to help group similar scenarios together. Some categories that may be used include the following:

• System health: This is the scenario focused on ensuring the operational health of a system or service required to keep the business running.
• Compliance: This is the consideration due to compliance requirements specific to your business, industry, or geographical region.
• Vulnerability: Is this a known system or process vulnerability that needs mitigation to protect systems or processes?
• Threat: This is any scenario that articulates a potential threat but may not have a specific vulnerability associated with it.
• Breach: These are scenarios that explore the impact of a successful breach.

Step 3 – the kill chain stage

The kill chain is a well-known construct that originated in the military and was later developed as a framework by Lockheed Martin (see here for more details: https://en.wikipedia.org/wiki/Kill_chain). Other frameworks are available, or you can develop your own.

Use the following list as headers to articulate the potential ways in which resources can become compromised in each scenario and at each stage of the kill chain:

• Reconnaissance
• Weaponization
• Delivery
• Exploitation
• Installation
• Command and control
• Actions on objectives

Step 4 – which solution will perform detection?

Review the information from earlier in this chapter to map which component of your security solutions architecture will be able to detect the threats for each scenario:

• SIEM
• CASB
• DLP
• IAM
• EDR
• NGFW
• WAF
• CWPP
• CSPM
• Many others

Step 5 – what actions will occur instantly?

As we aim to maximize the automation of detection and response, consider what actions should be carried out immediately, and then focus on enabling the automation of these actions.

Actions may include the following:

• Logging and alerting
• Notifying/warning the end user
• Blocking the action
• Offering alternative options/actions
• Triggering a workflow

Step 6 – severity and output

In this step, you should be able to assign a number to associate with the severity level, based on the impact analysis in the previous steps. For each severity level, define the appropriate output required:

• Level 0 – Logs and reports
• Level 1 – Dashboard notifications
• Level 2 – Generate events in the ticketing system
• Level 3 – Alerts sent to groups/individuals
• Level 4 – Automatic escalation to the senior management team (sirens and flashing lights are optional!)

Step 7 – what action should the analyst take?

Whereas the Step 5 – what actions will occur instantly? section was an automated action, this step is a definition of what the security analysts should do. For each scenario, define what actions should be taken to ensure an appropriate response, remediation, and recovery.

The following diagram is a simple reference chart that can be used during the scenario-mapping exercise:

Figure 1.4 – The scenario-mapping process

By following this seven-step process, your team can better prepare for any eventuality. By following a repeatable process, and improving that process each time, your team can share knowledge with each other, and carry out testing to ensure that protection and detection are efficient and effective as well as identifying new gaps in solutions that must be prioritized.

You should commit to taking time away from the computer and start to develop this type of tabletop exercise on a regular basis. Some organizations only do this once per year, while others will do it on a weekly basis or as needed based on the demands they see in their own systems and company culture.

In this chapter, we introduced Microsoft Sentinel and how it fits into the cloud security landscape. We explored some of the widely used acronyms for both problems and solutions and then provided a useful method of mapping these technical controls to the wide array of options available from many security platform providers today. We also looked at the future state of SOC architecture to ensure you can gain visibility and control across your entire infrastructure: physical, virtual, and cloud-hosted.

Finally, we looked at the potential cost of running Microsoft Sentinel as a core component of your security architecture and how to carry out the scenario-mapping exercise to ensure you are constantly reviewing the detections, the usefulness of the data, and your ability to detect and respond to current threats.

In the next chapter, we will take the first steps toward deploying Microsoft Sentinel by configuring an Azure Monitor Log Analytics workspace. Azure Monitor is the bedrock of Microsoft Sentinel for storing and searching log data. By understanding this data collection and analysis engine, you will gain a deeper understanding of the potential benefits of deploying Microsoft Sentinel in your environment.

1. What is the purpose of the cloud security reference framework?
2. What are the three main components when deploying an SOC based on Microsoft Sentinel?
3. What are some of the main operation platforms that integrate with a SIEM?
4. Can you name five of the third-party (non-Microsoft) solutions that can be connected to Microsoft Sentinel?
5. How many steps are involved in the scenario-mapping exercise?

You can refer to the following URLs for more information on topics covered in this chapter:

• Lessons learned from the Microsoft SOC – Part 1: Organization, at https://www.microsoft.com/security/blog/2019/02/21/lessons-learned-from-the-microsoft-soc-part-1-organization/
• Lessons learned from the Microsoft SOC – Part 2a: Organizing People, at https://www.microsoft.com/security/blog/2019/04/23/lessons-learned-microsoft-soc-part-2-organizing-people/
• Lessons learned from the Microsoft SOC – Part 2b: Career paths and readiness, at https://www.microsoft.com/security/blog/2019/06/06/lessons-learned-from-the-microsoft-soc-part-2b-career-paths-and-readiness/
• The Microsoft Security blog, at https://www.microsoft.com/security/blog