Cloud-native is autonomous. It empowers self-sufficient, full-stack teams to rapidly perform lean experiments and continuously deliver innovation with confidence. The operative word here is confidence. We leverage fully managed cloud services, such as function-as-a-service, cloud-native databases, and event streaming to decrease the risk of running these advanced technologies. However, at this rapid pace of change, we cannot completely eliminate the potential for human error. To remain stable despite the pace of change, cloud-native systems are composed of bounded, isolated, and autonomous services that are separated by bulkheads to minimize the blast radius when any given service experiences a failure. Each service is completely self-sufficient and stands on its own, even when related services are unavailable.

Following reactive principles, these autonomous services...

One of the major benefits of the Event Sourcing pattern is that it results in an audit trail of all the state changes within a system. This store of events can also be leveraged to replay events to repair broken services and seed new components. A cloud-native event stream, such as AWS Kinesis, only stores events for a short period of time, ranging from 24 hours to 7 days. An event stream can be thought of as a temporary or temporal event store that is used for normal, near real-time operation. In the Creating a micro event store recipe, we will discuss how to create specialized event stores that are dedicated to a single service. In this recipe, we will create a data lake in S3. A data lake is a perpetual event store that collects and stores all events in their raw format in perpetuity with complete fidelity and high durability to support auditing and replay...

Event sourcing is a key pattern for designing eventually consistent cloud-native systems. Upstream services produce events as their state changes, and downstream services consume these events and produce their own events as needed. This results in a chain of events whereby services collaborate to produce a business process that results in an eventual consistency solution. Each step in this chain must be implemented as an atomic unit of work. Cloud-native systems do not support distributed transactions, because they do not scale horizontally in a cost-effective manner. Therefore, each step must update one, and only one, system. If multiple systems must be updated, then each is updated in a series of steps. In this recipe, we leverage the event-first variant of the Event Sourcing pattern where the atomic unit of work...

In the Creating a data lake recipe, we will discuss how the Event Sourcing pattern provides the system with an audit trail of all the state-change events in the system. An event stream essentially provides a temporal event store that feeds downstream event processors in near real-time. The data lake provides a high durability, perpetual event store that is the official source of record. However, we have a need for a middle ground. Individual stream processors need the ability to source specific events that support their processing requirement. In this recipe, we will implement a micro event store in AWS DynamoDB that is owned by and tailored to the needs of a specific service.

...

In the previous recipe, Applying the event-first variant of the Event Sourcing pattern, we discussed how the Event Sourcing pattern allows us to design eventually consistent systems that are composed of a chain of atomic steps. Distributed transactions are not supported in cloud-native systems, because they do not scale effectively. Therefore, each step must update one, and only one, system. In this recipe, we will leverage the database-first variant of the Event Sourcing pattern, where the atomic unit of work is writing to a single cloud-native database. A cloud-native database provides a change data capture mechanism that allows further logic to be atomically triggered that publishes an appropriate domain event to the event stream for further downstream processing. In this recipe, we will demonstrate...

In the Applying the event-first variant of the Event Sourcing pattern recipe, we discussed how the Event Sourcing pattern allows us to design eventually consistent systems that are composed of a chain of atomic steps. Distributed transactions are not supported in cloud-native systems, because they do not scale effectively. Therefore, each step must update one, and only one, system. In this recipe, we leverage the database-first variant of the Event Sourcing pattern, where the atomic unit of work is writing to a single cloud-native database. A cloud-native database provides a change data capture mechanism that allows further logic to be atomically triggered that publishes an appropriate domain event to the event stream for further downstream processing. In the recipe, we demonstrate an offline...

The Command Query Responsibility Segregation (CQRS) pattern is critical for designing cloud-native systems that are composed of bounded, isolated, and autonomous services with appropriate bulkheads to limit the blast radius when a service experiences an outage. These bulkheads are implemented by creating materialized views in downstream services.

Upstream services are responsible for the commands that write data using the Event Sourcing pattern. Downstream services take responsibility for their own queries by creating materialized views that are specifically tailored to their needs. This replication of data increases scalability, reduces latency, and allows services to be completely autonomous and function even when upstream source services are unavailable. In this recipe, we will implement a materialized view in AWS DynamoDB.

In the Creating a materialized view in DynamoDB recipe, we discussed how the CQRS pattern allows us to design services that are bounded, isolated, and autonomous. This allows services to operate, even when their upstream dependencies are unavailable, because we have eliminated all synchronous inter-service communication in favor of replicating and caching the required data locally in dedicated materialized views. In this recipe, we will implement a materialized view in AWS S3.

1. Create the project from the following template:

$ sls create --template-url https://github.com/danteinc/js-cloud-native-cookbook/tree/master/ch2/materialized-view-s3 --path cncb-materialized-view-s3...

In the Creating a materialized view in DynamoDB recipe, we discussed how the CQRS pattern allows us to design services that are bounded, isolated, and autonomous. This allows services to operate, even when their upstream dependencies are unavailable, because we have eliminated all synchronous inter-service communication in favor of replicating and caching the required data locally in dedicated materialized views. In this recipe, we will implement a materialized view in AWS Elasticsearch.

1. Create the project from the following template:

$ sls create --template-url https://github.com/danteinc/js-cloud-native-cookbook/tree/master/ch2/materialized-view-es --path cncb...

In the Creating a materialized view in DynamoDB recipe, we discussed how the CQRS pattern allows us to design services that are bounded, isolated, and autonomous. This allows services to operate, even when their upstream dependencies are unavailable, because we have eliminated all synchronous inter-service communication in favor of replicating and caching the required data locally in materialized views. In this recipe, we will implement an offline-first materialized view in an AWS Cognito dataset.

1. Create the project from the following template:

$ sls create --template-url https://github.com/danteinc/js-cloud-native-cookbook/tree/master/ch2/materialized-view...

One of the advantages of the Event Sourcing and data lake patterns is that they allow us to replay events when necessary to repair broken services and seed new services, and even new versions of a service. In this recipe, we will implement a utility that reads selected events from the data lake and applies them to a specified Lambda function.

Before starting this recipe, you will need the data lake that was created in the Creating a data lake recipe in this chapter. The data lake should contain events that were generated by working through the other recipes in this chapter.

A data lake is a crucial design pattern for providing cloud-native systems with an audit trail of all the events in a system and for supporting the ability to replay events. In the Creating a data lake recipe, we implemented the S3 component of the data lake that provides high durability. However, a data lake is only useful if we can find the relevant data. In this recipe, we will index all the events in Elasticsearch so that we can search events for troubleshooting and business analytics.

1. Create the project from the following template:

$ sls create --template-url https://github.com/danteinc/js-cloud-native-cookbook/tree/master/ch2/data-lake-es --path cncb-data-lake-es

2. Navigate to...

Cloud-native systems are architectured to support the continuous evolution of the system. Upstream and downstream services are designed to be pluggable. New service implementations can be added without impacting related services. Furthermore, continuous deployment and delivery necessitate the ability to run multiple versions of a service side by side and synchronize data between the different versions. The old version is simply removed when the new version is complete and the feature is flipped on. In this recipe, we will enhance the database-first variant of the Event Sourcing pattern with the latching pattern to facilitate bi-directional synchronization without causing an infinite loop of events.