Apache Kafka is a distributed messaging system that is becoming an industry favorite. A messaging system is commonly used to decouple application components and help scale out system architectures. Apache Kafka has the peculiarity of writing messages directly into the disk instead of memory. Initially, this might seem like a showstopper. However, this technique allows Apache Kafka to achieve extremely high message rates thanks to both sequential writes and the partitioning system. Kafka has demonstrated that it is capable of consuming and delivering millions of messages per second.
In this recipe, we will use Kafka 0.10.1. We will create two streams. The first one will publish to a Kafka topic. The second one will consume from it. Both the streams use the same topic, namely akka_streams_topic.Â
