Learning the microservices architecture

Let's imagine that a client approaches you, wanting you to design a solution for them. They need some kind of banking application that has to guarantee data consistency within the whole system. Our client had been using an Oracle database until now and has also purchased support from their side. Without thinking too much, we decide to design a monolithic application based on a relational data model. You can see a simplified diagram of the system's design here:

There are four entities that are mapped into the tables in the database:

• The first of them, Customer, stores and retrieves the list of active clients
• Every customer could have one or more accounts, which are operated by the Account entity
• The Transfer entity is responsible for performing all transfers of funds between accounts within the system
• There is also the Product entity that is created to store information such as the deposits and credits assigned to the clients

Without going into further details, the application exposes the API that provides all the necessary operations for realizing actions on the designed database. Of course, the implementation is in compliance with the three-layer model. 

Consistency is not the most important requirement anymore; it is not even obligatory. The client expects a solution, but does not want the development to require the redeployment of the whole application. It should be scalable and should easily be able to extend new modules and functionalities. Additionally, the client does not put pressure on the developer to use Oracle or another relational database—not only that, but he would be happy to avoid using it. Are these sufficient reasons to decide on migrating to microservices? Let's just assume that they are. We divide our monolithic application into four independent microservices, each one of them with a dedicated database. In some cases, it can still be a relational database, while in others it can be a NoSQL database. Now, our system consists of many services that are independently built and run in our environment. Along with an increase in the number of microservices, there is a rising level of system complexity. We would like to hide that complexity from the external API client, which should not be aware that it talks to service X but not Y. The gateway is responsible for dynamically routing all requests to different endpoints. For example, the word dynamically means that it should be based on entries in the service discovery, which I'll talk about later in the section Understanding the need for service discovery.

Hiding invocations of specific services or dynamic routing is not the only function of an API gateway. Since it is the entry point to our system, it can be a great place to track important data, collect metrics of requests, and count other statistics. It can enrich requests or response headers in order to include some additional information that is usable by the applications inside the system. It should perform some security actions, such as authentication and authorization, and should be able to detect the requirements for each resource and reject requests that do not satisfy them. Here's a diagram that illustrates the sample system, consisting of four independent microservices, which is hidden from an external client behind an API gateway:

Understanding the need for service discovery

Let's imagine that we have already divided our monolithic application into smaller, independent services. From the outside, our system still looks the same as before, because its complexity is hidden behind the API gateway. Actually, there are not many microservices, but, there may well be many more. Additionally, each of them can interact with the others. That means that every microservice has to keep information about the others' network addresses. Maintaining such a configuration could be very troublesome, especially when it comes down to manually overwriting every configuration. And what if those addresses are changing dynamically after restart? The following diagram shows the calling routes between our example microservices:

Service discovery is the automatic detection of devices and services offered by these devices on a computer network. In the case of microservice-based architecture, this is the necessary mechanism. Every service after startup should register itself in one central place that is accessible by all other services. The registration key should be the name of a service or an identificator, which has to be unique within the whole system in order to enable others to find and call the service using that name. Every single key with the given name has some values assigned to it. In the most common cases, these attributes indicate the network location of the service. To be more accurate, they indicate one of the instances of the microservice because it can be multiplied as independent applications running on different machines or ports. Sometimes it is possible to send some additional information, but it depends on the concrete service discovery provider. However, the important thing here is that under the one key, more than one instance of the same service may be registered. In addition to registration, each service gets a full list of the other services registered on the particular discovery server. Not only that, every microservice must be aware of any changes in the registration list. This may be achieved by periodically renewing the configuration earlier collected from the remote server.

Some solutions combine the usage of service discovery with the server configuration feature. When it comes right down to it, both approaches are pretty similar. The configuration of the server lets you centralize the management of all configuration files in your system. Usually, such a configuration is then a server as a REST web service. Before startup, every microservice tries to connect to the server and get the parameters prepared especially for it. One of the approaches stores such a configuration in the version control system—for example, Git. Then the configuration server updates its Git working copy and serves all properties as a JSON. In another approach, we can use solutions that store key-value pairs and fulfill the role of providers during the service discovery procedure. The most popular tools for this are Consul and Zookeeper. The following diagram illustrates an architecture of a system that consists of some microservices with a database backend that are registered in one central service known as a discovery service:

Communication between services

In order to guarantee the system's reliability, we cannot allow a situation where each service would have only one instance running. We usually aim to have a minimum of two running instances in case one of them experiences a failure. Of course, there could be more, but we'll keep it low for performance reasons. Anyway, multiple instances of the same service make it necessary to use load balancing for incoming requests. Firstly, the load balancer is usually built into an API gateway. This load balancer should get the list of registered instances from the discovery server. If there is no reason not to, then we usually use a round-robin rule that balances incoming traffic 50/50 between all running instances. The same rule also applies to load balancers on the microservices side.

The following diagram illustrates the most important components that are involved in interservice communication between multiple instances of two sample microservices:

Most people, when they hear about microservices, consider it to consist of RESTful web services with JSON notation, but that's just one of the possibilities. We can use some other interaction styles, which, of course, apply not only to microservices-based architecture. The first categorization that should be performed is one-to-one or one-to-many communication. In one-to-one interaction, every incoming request is processed by exactly one service instance while, in one-to-many, it is processed by multiple service instances. But the most popular division criterion is whether the call is synchronous or asynchronous. Additionally, asynchronous communication can be divided into notifications. When a client sends a request to a service, but a reply is not expected, it can just perform a simple asynchronous call, which does not block a thread and replies asynchronously. 

Furthermore, it is worth mentioning reactive microservices. Now, from version 5, Spring also supports this type of programming. There are also libraries with Reactive support for interaction with NoSQL databases, such as MongoDB or Cassandra. The last well-known communication type is publish-subscribe. This is a one-to-many interaction type where a client publishes a message that is then consumed by all listening services. Typically, this model is realized using message brokers, such as Apache Kafka, RabbitMQ, and ActiveMQ. 

Failures and circuit breakers

We have discussed most of the important concepts related to the microservices architecture. Such mechanisms, such as service discovery, API gateways, and configuration servers, are useful elements that help us to create a reliable and efficient system. Even if you have considered many aspects of these while designing your system's architecture, you should always be prepared for failures. In many cases, the reasons for failures are totally beyond the control of the holder, such as network or database problems. Such errors can be particularly severe for microservice-based systems, where one input request is processed in many subsequent calls. The first good practice is to always use network timeouts when waiting for a response. If a single service has a performance problem, we should try to minimize the impact on the rest. It is better to send an error response than to wait on a reply for a long time, blocking other threads. 

An interesting solution for the network timeout problems might be the circuit breaker pattern. It is a concept closely related to the microservice approach. A circuit breaker is responsible for counting successful and failed requests. If the error rate exceeds an assumed threshold, it trips and causes all further attempts to fail immediately. After a specific period of time, the API client should get back to sending requests, and if they succeed, it closes the circuit breaker. If there are many instances of each service available and one of them works slower than others, the result is that it is overlooked during the load balancing process. The second often-used mechanism for dealing with partial network failures is fallback. This is a logic that has to be performed when a request fails. For example, a service can return cached data, a default value, or an empty list of results. Personally, I'm not a big fan of this solution. I would prefer to propagate error code to other systems than return cached data or default values.