Event-driven architecture

EDA is an architectural pattern where capturing, processing, and storing events is the central theme. This allows a bunch of microservices to exchange and process information asynchronously. But before we dive into the details of the architecture, let's define what an event is.

Events

An event is the record of a significant occurrence or change that's been made to the state of a system. The source of the event could be a change in the hardware or software system. An event could also be a change to the content of a data item or a change in the state of a business transaction. Anything that happens in your business or IT infrastructure could be an event. Which events do we need to process and bring under EDA as an engineering and business choice? Events are immutable records and can be read and processed without the event needing to be modified. Events are usually ordered based on their creation time.

Some examples of events are as follows:

• Customer requests
• Change of balance in a bank account
• A food delivery order being placed
• A user being added to a server
• Sensor reading from a hardware or IoT device
• A security breach in a system

You can find examples of events all around your application and infrastructure. The trick is deciding on which are relevant and need processing. In the next section, we'll look at the structure of EDA.

Structure and components of an EDA

The value proposition of EDA comes from the fact that an event loses its processing value as it gets older. Event-driven systems can respond to such events as they are generated and take appropriate action to add a lot of business value. In an event-driven system, messages from various sources are ingested, then sent to interested parties (read microservices) for processing, and then persisted to disk for a defined period.

EDA fundamentally differs from the synchronous model that's followed by APIs and web stacks, where a response must be returned for every request synchronously. This could be compared to a customer support center using phone calls versus emails to respond to customer requests. While phone calls take a lot of time and need the support agent to be manually responding to the request, the same time can be spent asynchronously replying to a bunch of emails, often with the help of automation. The same principle applies to request-response versus event-driven models. But just like this example, EDA is not a silver bullet and can't be used on all occasions. The trick is in finding the right use case and building on it. Most critical systems and customer-facing services still have to rely on the synchronous request-response model.

The components of an event-driven model can be broadly classified into three types – event producer, event router (broker), and event consumer. The event producers are one or more microservices that produce interesting events and post them to the broker. The event broker is the central component of this architecture and enables loose coupling between producers and consumers. It is responsible for receiving the events, serializing or deserializing them, if necessary, notifying the consumers of the new event, and storing them. Certain brokers also filter the events based on conditions or rules. The consumers can then consume the interesting events at their pace:

Figure 1.5 – EDA

That sums up the EDA pattern. Now, let's look into the benefits of EDA.

Benefits of EDA

The following is not a comprehensive list of the benefits of EDA, but this should give you a fair idea of why this architecture pattern is important:

• Improved scalability and fault tolerance due to a producer or consumer failing doesn't impact the rest of the systems.
• Real-time data processing for better decisions and customer experience – businesses can respond in real time to changes in customer behavior and make decisions or share data that improves the quality of the service.
• Operational stability and agility.
• Cost efficiency compared to batch processing. With batch processing, large volumes of data had to be stored and processed in batches. This meant allocating a lot more storage and compute resources for a longer period. Once batch processing is over, the computing resource becomes idle. This doesn't happen in EDA as the events are processed as they arrive, and it distributes the compute and storage optimally.
• Better interoperability between independent services.
• High throughput and low latency.
• Easy to filter and transform events.
• The rate of production and consumption doesn't have to match.
• Works with small as well as complex applications.

Now that we have covered the use cases of EDA, let's look at some use cases where the EDA pattern can be implemented.

Use cases

EDA has a very varied set of use cases; some examples are as follows:

• Real-time monitoring and alerting based on the events in a software system
• Website activity tracking
• Real-time trend analysis and decision making
• Fraud detection
• Data replication between similar and different applications
• Integration with external vendors and services

While EDA becomes more and more important as the business logic and infrastructure becomes complicated, there are certain downsides we need to be aware of. We'll explore them in the next section.

Disadvantages

As we mentioned earlier, EDA is no silver bullet and doesn't work with all business use cases. Some of its notable disadvantages are as follows:

• The decoupled nature of events can also make it difficult to debug or trace back the issues with events.
• The reliability of the system depends on the reliability of the broker. Ideally, the broker should be either a cloud service or a self-hosted distributed system with a high degree of reliability.
• Consumer patterns can make it difficult to do efficient capacity planning. If many of the consumers are services that wake up only at a defined interval and process the events, this could create an imbalance in the capacity for that period.
• There is no single standard in implementing brokers – knowing the guarantees that are provided by the broker is important. Architectural choices such as whether it provides a strong guarantee of ordering or the promise of no duplicate events should be figured out early in the design, and the producers and consumers should be designed accordingly.

In the next section, we will discuss what our software choices are for EDA, both on-premises and in the cloud.

Brokers

There are open source brokers such as Kafka, Apache Pulsar, and Apache ActiveMQ that can implement some form of message broker. Since we are mostly talking in the context of the cloud in this book, the following are the most common cloud brokers:

• Amazon Simple Queue Service (SQS)
• Amazon Simple Notification Service (SNS)
• Amazon EventBridge
• Azure Service Bus queues
• Azure Service Bus topics
• Google Cloud Pub/Sub
• Google Cloud Pub/Sub Lite

EDA, as we've discovered, is fundamental to a lot of modern applications' architectures. Now, let's look at FaaS platforms in detail.