This chapter hasn't got any special technical requirements, but now is a good time to install or update your Rust compiler. You can get this from Rust's official website: https://www.rust-lang.org/ . I recommend that you use the rustup tool, which you can download from https://rustup.rs/.

If you have previously installed the compiler, you need to update it to the latest version using the following command:

rustup update

You can get the examples for this book from the GitHub page: https://github.com/PacktPublishing/Hands-On-Microservices-with-Rust-2018/.

Modern users interact with microservices every day; not directly, but by using web applications. Microservices are a flexible software development technique that help to implement applications as a collection of independent services with weak relations.

In this section, we'll learn about why microservices are a good thing and why we need them. Microservices follow the REST architecture, which provides rules about using consistent HTTP methods. We will also look at how microservices can be deployed to the user, which is one of their main advantages.

Microservices are a modern software development approach that refers to the splitting of software into a suite of small services that are easier to develop, debug, deploy, and maintain. Microservices are tiny, independent servers that act as single business functions. For example, if you have an e-commerce suite that works as a monolith, you could split it into small servers that have limited responsibility and carry out the same tasks. One microservice could handle user authorization, the other could handle the users' shopping carts, and the remaining services could handle features such as search functionality, social-media integration, or recommendations.

Microservices can either interact with a database or be connected to other microservices. To interact with a database, microservices can use different protocols. These might include HTTP or REST, Thrift, ZMQ, AMQP for the messaging communication style, WebSockets for streaming data, and even the old-fashioned Simple Object Access Protocol (SOAP) to integrate them with the existing infrastructure. We will use HTTP and REST in this book, because this is the most flexible way to provide and interact with the web API. We'll explain this choice later.

Microservices have the following advantages over monolithic servers:

• You can use different programming languages
• The code base of a single server is smaller
• They have an independent DevOps process to build and deploy activities
• They can be scaled depending on their implementation
• If one microservice fails, the rest will continue to work
• They work well within containers
• Increased isolation between elements leads to better security
• They are suitable for projects involving the Internet of Things
• They are in line with the DevOps philosophy
• They can be outsourced
• They can be orchestrated after development
• They are reusable

There are, however, a few drawbacks of microservices. These include the following:

• Too many microservices overload the development process
• You have to design interaction protocols
• They can be expensive for small teams

A microservices architecture is a modern approach that can help you achieve the goal of having loosely coupling elements. This is where the servers are independent from one another, helping you to release and scale your application faster than a monolithic approach, in which you put all your eggs in one basket.

Since a microservice is a small but complete web server, you have to deploy it as a complete server. But since it has a narrow scope of features, it's also simpler to configure. Containers can help you pack your binaries into an image of the operating system with the necessary dependencies to simplify deployment.

This differs from the case with monoliths, in which you have a system administrator who installs and configures the server. Microservices need a new role to carry out this function—DevOps. DevOps is not just a job role, but a whole software engineering culture in which developers become system administrators and vice versa. DevOps engineers are responsible for packing and delivering the software to the end user or market. Unlike system administrators, DevOps engineers work with clouds and clusters and often don't touch any hardware except their own laptop.

DevOps uses a lot of automation and carries the application through various stages of the delivery process: building, testing, packaging, releasing, or deployment, and the monitoring of the working system. This helps to reduce the time it takes both to market a particular software and to release new versions of it. It's impossible to use a lot of automation for monolithic servers, because they are too complex and fragile. Even if you want to pack a monolith to a container, you have to deliver it as a large bundle and run the risk that any part of the application could fail. In this section, we'll have a brief look at containers and continuous integration. We will go into detail about these topics in Chapter 15, Packing Servers to Containers, and Chapter 16, DevOps of Rust Microservices – Continuous Integration and Delivery.

When we refer to containers, we almost always mean Docker containers (https://www.docker.com/). Docker is the most popular software tool for running programs in containers, which are isolated environments.

Containerization is a kind of virtualization where the scope of the application's resources is limited. This means the application works at its maximum performance level. This is different from full virtualization, where you have to run the full operating system with the corresponding overhead and run your application inside that isolated operating system.

Docker has become popular for a variety of reasons. One of these reasons is that it has a registry—the place where you can upload and download images of containers with applications. The public registry is Docker Hub (https://hub.docker.com/explore/), but you can have a private registry for a private or permissioned software.

Continuous Integration (CI) is the practice of keeping a master copy of the software and using tests and merging processes to expand the features of the application. The process of CI is integrated with the Source Code Management (SCM) process. When the source code is updated (for example, in Git), the CI tool checks it and starts the tests. If all tests pass, developers can merge the changes to the master branch.

CI doesn't guarantee that the application will work, because tests can be wrong, but it removes the need to run tests from developers on an automated system. This gives you the great benefit of being able to test all your upcoming changes together to detect conflicts between changes. Another advantage is that the CI system can pack your solution in a container, so the only thing that you have to do is deliver the container to a production cloud. The deployment of containers is also simple to automate. 

Around 10 years ago, developers used to use the Apache web server with a scripting programming language to create web applications, rendering the views on the server-side. This meant that there was no need to split applications into pieces and it was simpler to keep the code together. With the emergence of Single-Page Applications (SPAs), we only needed server-side rendering for special cases and applications were divided into two parts: frontend and backend. Another tendency was that servers changed processing method from synchronous (where every client interaction lives in a separate thread) to asynchronous (where one thread processes many clients simultaneously using non-blocking, input-output operations). This trend promotes the better performance of single server units, meaning they can serve thousands of clients. This means that we don't need special hardware, proprietary software, or a special toolchain or compiler to write a tiny server with great performance.

The invasion of microservices happened when scripting programming languages become popular. By this, we are not only referring to languages for server-side scripting, but general-purpose high-level programming languages such as Python or Ruby. The adoption of JavaScript for backend needs, which had previously always been asynchronous, was particularly influential.

If writing your own server wasn't hard enough, you could create a separate server for special cases and use them directly from the frontend application. This would not require rendering procedures on the server. This section has provided a short description of the evolution from monolithic servers to microservices. We are now going to examine how to break a monolithic server into small pieces.

If you already have a single server that includes all backend features, you have a monolithic service, even if you start two or more instances of this service. A monolithic service has a few disadvantages—it is impossible to scale vertically, it is impossible to update and deploy one feature without interrupting all the running instances, and if the server fails, it affects all features. Let's discuss these disadvantages a little further. This might help you to convince your manager to break your service down into microservices.

There are two common approaches to scaling an application:

• Horizontally: Where you start a new instance of application
• Vertically: Where you improve an independent application layer that has a bottleneck

The simplest way to scale a backend is to start another instance of the server. This will solve the issue, but in many cases it is a waste of hardware resources. For example, imagine you have a bottleneck in an application that collects or logs statistics. This might only use 15% of your CPU, because logging might include multiple IO operations but no intensive CPU operations. However, to scale this auxiliary function, you will have to pay for the whole instance.

If your backend works as a monolith, you can't update only a small part of it. Every time you add or change a feature, you have to stop, update, and start the service again, which causes interruptions.

When you have a microservice and you have find a bug, you can stop and update only this microservice without affecting the others. As I mentioned before, it can also be useful to split a product into separate development teams.

Another reason to avoid monoliths is that every server crash also crashes all of the features, which causes the application to stop working completely, even though not every feature is needed for it to work. If your application can't load new user interface themes, the error is not critical, as long as you don't work in the fashion or design industry, and your application should still be able to provide the vital functions to users. If you split your monolith into independent microservices, you will reduce the impact of crashes.

Let's look an example of an e-commerce monolith server that provides the following features:

• User registration
• Product catalog
• Shopping cart
• Payment integration
• E-mail notifications
• Statistics collecting

Old-fashioned servers developed years ago would include all of these features together. Even if you split it into separate application modules, they would still work on the same server. You can see an example structure of a monolithic service here:

In reality, the real server contains more modules than this, but we have separated them into logical groups based on the tasks they perform. This is a good starting point to breaking your monolith into multiple, loosely coupled microservices. In this example, we can break it further into the pieces represented in the following diagram:

As you can see, we use a balancer to route requests to microservices. You can actually connect to microservices directly from the frontend application.

Shown in the preceding diagram is the potential communication that occurs between services. For simple cases, you can use direct connections. If the interaction is more complex, you can use message queues. However, you should avoid using a shared state such as a central database and interacting through records, because this can cause a bottleneck for the whole application. We will discuss how to scale microservices in Chapter 12, Scalable Microservices Architecture. For now, we will explore REST API, which will be partially implemented in a few examples throughout this book. We will also discuss why Rust is a great choice for implementing microservices.

Let's define the APIs that we will use in our microservice infrastructure using the REST methodology. In this example, our microservices will have minimal APIs for demonstration purposes; real microservices might not be quite so "micro". Let's explore the REST specifications of the microservices of our application. We will start by looking at a microservice for user registration and go through every part of the application.

The first service is responsible for the registration of users. It has to contain methods to add, update, or delete users. We can cover all needs with the standard REST approach. We will use a combination of methods and paths to provide this user registration functionality:

• POST request to /user/ creates a new user and returns its id
• GET request to /user/id  returns information related to a user with id
• PUT request to /user/id  applies changes to a user with id
• DELETE request to /user/id  removes a user with id

This service can use the E-mail notifications microservice and call its methods to notify the user about registration.

The E-mail notifications microservice can be extremely simple and contains only a single method:

• The POST request to /send_email/  sends an email to any address

This server can also count the sent emails to prevent spam or check that the email exists in the user's database by requesting it from the User registration microservice. This is done to prevent malicious use.

The Product catalog microservice tracks the available products and needs only weak relations with other microservices, except for the Shopping cart. This microservice can contain the following methods:

• POST request to /product/ creates a new product and returns its id
• GET request to /product/id  returns information about the product with id
• PUT request to /product/id  updates information about the product with id
• DELETE request to /product/id  marks the product with id as deleted
• GET request to /products/ returns a list of all products (can be paginated by extra parameters)

The Shopping cart microservice is closely integrated with the User registration and Product catalog microservices. It holds pending purchases and prepares invoices. It contains the following methods:

• POST request to /user/uid/cart/, which puts a product in the cart and returns the id of item in the user's cart with the uid
• GET request to /user/uid/cart/id, which returns information about the item with id
• PUT request to /user/uid/cart/id, which updates information about the item with id (alters the quantity of items)
• GET request to /user/uid/cart/, which returns a list of all the items in the cart

As you can see, we don't add an extra "s" to the /cart/ URL and we use the same path for creating items and to get a list, because the first handler reacts to the POST method, the second processes requests with the GET method, and so on. We also use the user's ID in the path. We can implement the nested REST functions in two ways:

• Use session information to get the user's id. In this case, the paths contain a single object, such as /cart/id . We can keep the user's id in session cookies, but this is not reliable.
• We can add the id of a user to a path explicitly.

In our example, this microservice will be a third-party service, which contains the following methods:

• POST request to /invoices  creates a new invoice and returns its id
• POST request to /invoices/id/pay  pays for the invoice

This service collects usage statistics and logs a user's actions to later improve the application. This service exports API calls to collect the data and contains some internal APIs to read the data:

• POST request to /log  logs a user's actions (the id of a user is set in the body of the request)
  
• GET request to /log?from=?&to=?  works only from the internal network and returns the collected data for the period specified

This microservice doesn't conform clearly to the REST principles. It's useful for microservices that provide a full set of methods to add, modify, and remove the data, but for other services, it is excessively restrictive. You don't have follow a clear REST structure for all of your services, but it may be useful for some tools that expect it.

If you already have a working application, you might transform it into a set of microservices, but you have to keep the application running at the highest rate and prevent any interruptions.

To do this, you can create microservices step by step, starting from the least important task. In our example, it's better to start from email activities and logging. This practice helps you to create a DevOps process from scratch and join it with the maintenance process of your app.

If your application is a monolith server, you don't need to turn all modules into microservices, because you can use existing third-party services and shrink the bulk of the code that needs rewriting. These services can help with many things, including storage, payments, logging, and transactional notifications that tell you whether an event has been delivered or not.

I recommend that you create and maintain services that determine your competitive advantage yourself and then use third-party services for other tasks. This can significantly shrink your expenses and the time to market.

In any case, remember the product that you are delivering and don't waste time on unnecessary units of your application. The microservices approach helps you to achieve this simply, unlike the tiresome coding of monoliths, which requires you to deal with numerous secondary tasks. Hopefully, you are now fully aware of the reasons why microservices can be useful. In the next section, we will look at why Rust is a promising tool for creating microservices.

If you have chosen to read this book, you probably already know that Rust is an up-to-date, powerful, and reliable language. However, choosing it to implement microservices is not an obvious decision, because Rust is a system programming language that is often assigned to low-level software such as drivers or OS kernels. This is because you tend to have to write a lot of glue code or get into detailed algorithms with low-level concepts, such as pointers in system programming languages. This is not the case with Rust. As a Rust programmer, you've surely already seen how it can be used to create high-level abstractions with flexible language capabilities. In this section, we'll discuss the strengths of Rust: its strict and explicit nature, its high performance, and its great package system.

Up until recently, there hasn't been a well-established approach to using Rust for writing asynchronous network applications. Previously, developers tended to use two styles: either explicit control structures to handle asynchronous operations or implicit context switching. The explicit nature of Rust meant that the first approach outgrew the second. Implicit context switching is used in concurrent programming languages such as Go, but this model does not suit Rust for a variety of reasons. First of all, it has design limitations and it's hard or even impossible to share implicit contexts between threads. This is because the standard Rust library uses thread-local data for some functions and the program can't change the thread environment safely. Another reason is that an approach with context switching has overheads and therefore doesn't follow the zero-cost abstractions philosophy because you would have a background runtime. Some modern libraries such as actix provide a high-level approach similar to automatic context switching, but actually use explicit control structures for handling asynchronous operations.

Network programming in Rust has evolved over time. When Rust was released, developers could only use the standard library. This method was particularly verbose and not suitable for writing high-performance servers. This was because the standard library didn't contain any good asynchronous abstractions. Also, event hyper, a good crate for creating HTTP servers and clients, processed requests in separate threads and could therefore only have a certain number of simultaneous connections.

The mio crate was introduced to provide a clear asynchronous approach to make high-performance servers. It contained functions to interact with asynchronous features of the operating system, such as epoll or kqueue, but it was still verbose, which made it hard to write modular applications.

The next abstraction layer over mio was a futures and tokio pair of crates. The futures crate contained abstractions for implementing delayed operations (like the defers concept in Twisted, if you're familiar with Python). It also contained types for assembling stream processors, which are reactive and work like a finite state machine.

Using the futures crate was a powerful way to implement high-performance and high-accuracy network software. However, it was a middleware crate, which made it hard to solve everyday tasks. It was a good base for rewriting crates such as hyper, because these can use explicit asynchronous abstractions with full control.

The highest level of abstraction today are crates that use futures, tokio, and hyper crates, such as rocket or actix-web. Now, rocket includes high-level elements to construct a web server with the minimal amount of lines. actix-web works as a set of actors when your software is broken down into small entities that interact with one another. There are many other useful crates, but we will start with hyper as a basis for developing web servers from scratch. Using this crate, we will be between low-level crates, such as futures, and high-level crates, such as rocket. This will allow us to understand both in detail.

There are many languages suitable for creating microservices, but not every language has a reliable design to keep you from making mistakes. Most interpreted dynamic languages let you write flexible code that decides on the fly which field of the object to get and which function to call. You can often even override the rules of function calling by adding meta-information to objects. This is vital in meta-programming or in cases where your data drives the behavior of the runtime.

The dynamic approach, however, has significant drawbacks for the software, which requires reliability rather than flexibility. This is because any inaccuracy in the code causes the application to crash. The first time you try to use Rust, you may feel that it lacks flexibility. This is not true, however; the difference is in the approach you use to achieve flexibility. With Rust, all your rules must be strict. If you create enough abstractions to cover all of the cases your application might face, you will get the flexibility you want.

Rust rookies who come from the JavaScript or the Python world might notice that they have to declare every case of serialization/deserialization of data, whereas with dynamic languages, you can simply unpack any input data to the free-form object and explore the content later. You actually have to check all cases of inconsistency during runtime and try and work out what consequences could be caused if you change one field and remove another. With Rust, the compiler checks everything, including the type, the existence, and the corresponding format. The most important thing here is the type, because you can't compile a program that uses incompatible types. With other languages, this sometimes leads to strange compilation errors such as a case where you have two types for the same crate but the types are incompatible because they were declared in different versions of the same crate. Only Rust protects you from shooting yourself in the foot in this way. In fact, different versions can have different rules of serialization/deserialization for a type, even if both declarations have the same data layout. 

Rust is a system programming language. This means your code is compiled into native binary instructions for the processor and runs without unwanted overhead, unlike interpreters such as JavaScript or Python.

Rust also doesn't use a garbage collector and you can control all allocations of memory and the size of buffers to prevent overflow.

Another reason why Rust is so fast for microservices is that it has zero-cost abstractions, which means that most abstractions in the language weigh nothing. They turn into effective code during compilation without any runtime overhead. For network programming, this means that your code will be effective after compilation, that is, once you have added meaningful constructions in the source code.

Rust programs are compiled into a single binary without unwanted dependencies. It needs libc or another dynamic library if you want to use OpenSSL or similar irreplaceable dependencies, but all Rust crates are compiled statically into your code.

You may think that the compiled binaries are quite large to be used as microservices. The word microservice, however, refers to the narrow logic scope, rather than the size. Even so, statically linked programs remain tiny for modern computers.

What benefits does this give you? You will avoid having to worry about dependencies. Each Rust microservice uses its own set of dependencies compiled into a single binary. You can even keep microservices with obsolete features and dependencies besides new microservices. In addition, Rust, in contrast with the Go programming language, has strict rules for dependencies. This means that the project resists breaking, even if someone forces an update of the repository with the dependency you need.

How does Rust compare to Java? Java has microframeworks for building microservices, but you have to carry all dependencies with them. You can put these in a fat Java ARchive (JAR), which is a kind of compiled code distribution in Java, but you still need Java Virtual Machine (JVM). Don't forget, too, that Java will load every dependency with a class loader. Also, Java bytecode is interpreted and it takes quite a while for the Just-In-Time (JIT) compilation to finish to accelerate the code. With Rust, bootstrapping dependencies don't take a long time because they are attached to the code during compilation and your code will work with the highest speed from the start since it was already compiled into native code.

In this chapter, we have mastered the basics of microservices. Simply put, a microservice is a compact web server that handles specific tasks. For example, microservices can be responsible for user authentication or for email notifications. They make running units reusable. This means you don't need to recompile or restart units if they don't require any updates. This approach is simpler and more reliable in deployment and maintenance.

We have also discussed how to split a monolithic web server that contains all of its business logic in a single unit into smaller pieces and join them together through communication, in line with the ideology of loose coupling. To split a monolithic server, you should separate it into domains that are classified by what tasks the servers carry out.

In the last section of this chapter, we've looked at why Rust is a good choice for developing microservices. We touched on dependencies management, the performance of Rust, its explicit nature, and its toolchain. It's now time to dive deep into coding and write a minimal microservice with Rust.

In the next chapter we will start to writing microservices with Rust using hyper crate that provides all necessary features to write compact asynchronous HTTP server.

You have learned about the basics of microservices in this chapter, which will serve as a point for you to start writing microservices on Rust throughout this book. If you want to learn more about topics discussed in this chapter, please consult the following list:

• Microservices - a definition of this new architectural term, 2014, Martin Fowler, available at https://martinfowler.com/articles/microservices.html. This article introduces the concept of microservices.
• mio, available at https://github.com/carllerche/mio. This is a crate that is widely used by other crates for asynchronous operations in Rust. We won't use it directly, but it is useful to know how it works.
• Network Programming with Rust, 2018, Abhishek Chanda, available at https://www.packtpub.com/application-development/network-programming-rust. This book explains more about network addresses, protocols and sockets, and how to use them all with Rust.