Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Building Data Streaming Applications with Apache Kafka

You're reading from   Building Data Streaming Applications with Apache Kafka Design, develop and streamline applications using Apache Kafka, Storm, Heron and Spark

Arrow left icon
Product type Paperback
Published in Aug 2017
Publisher Packt
ISBN-13 9781787283985
Length 278 pages
Edition 1st Edition
Tools
Arrow right icon
Authors (2):
Arrow left icon
Chanchal Singh Chanchal Singh
Author Profile Icon Chanchal Singh
Chanchal Singh
Manish Kumar Manish Kumar
Author Profile Icon Manish Kumar
Manish Kumar
Arrow right icon
View More author details
Toc

Table of Contents (14) Chapters Close

Preface 1. Introduction to Messaging Systems FREE CHAPTER 2. Introducing Kafka the Distributed Messaging Platform 3. Deep Dive into Kafka Producers 4. Deep Dive into Kafka Consumers 5. Building Spark Streaming Applications with Kafka 6. Building Storm Applications with Kafka 7. Using Kafka with Confluent Platform 8. Building ETL Pipelines Using Kafka 9. Building Streaming Applications Using Kafka Streams 10. Kafka Cluster Deployment 11. Using Kafka in Big Data Applications 12. Securing Kafka 13. Streaming Application Design Considerations

Advance Queuing Messaging Protocol

As discussed in previous sections, there are different data transmission protocols using which messages can be transmitted among sender, receiver, and message queues. It is difficult to cover all such protocols in the scope of this book. However, it is important to understand how these data transmission protocols work and why it is an important design decision for your message-oriented application integration architecture. In the light of this, we will cover one example of such a protocol: Advance Message Queuing Protocol also known as AQMP.

AQMP is an open protocol for asynchronous message queuing that developed and matured over several years. AMQP provides richer sets of messaging functionalities that can be used to support very advanced messaging scenarios. As depicted in the following figure, there are three main components in any AQMP-based messaging system:

AQMP architecture

As the name suggests, producers sends messages to brokers that in turn deliver them to consumers. Every broker has a component called exchange that is responsible for routing the messages from producers to appropriate message queues.

An AQMP messaging system consists of three main components:

  • Publisher(s)
  • Consumer(s)
  • Broker/server(s)

Each component can be multiple in number and situated on independent hosts. Publishers and consumers communicate with each other through message queues bound to exchanges within the brokers. AQMP provides reliable, guaranteed, in-order message delivery. Message exchanges in an AQMP model can follow various methods. Let's look at each one of them:

  • Direct exchange: This is a key-based routing mechanism. In this, a message is delivered to the queue whose name is equal to the routing key of the message.
  • Fan-out exchange: A fan-out exchange routes messages to all of the queues that are bound to it and the routing key is ignored. If N queues are bound to a fan-out exchange, when a new message is published to that exchange, a copy of the message is delivered to all N queues. Fan-out exchanges are ideal for the broadcast routing of messages. In other words, the message is cloned and sent to all queues connected to this exchange.
  • Topic exchange: In topic exchange, the message can be routed to some of the connected queues using wildcards. The topic exchange type is often used to implement various publish/subscribe pattern variations. Topic exchanges are commonly used for the multicast routing of messages.
You have been reading a chapter from
Building Data Streaming Applications with Apache Kafka
Published in: Aug 2017
Publisher: Packt
ISBN-13: 9781787283985
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image