Summary

We walked you through some of the aspects of using Kafka in big data applications. By the end of this chapter, you should have clear understanding of how to use Kafka in big data Applications. Volume is one of the important aspects of any big data application. Therefore, we have a dedicated section for it in this chapter, because you are required to pay attention to granular details while managing high volumes in Kafka. Delivery semantics is another aspect you should keep in mind. Based on your choice of delivery semantics, your processing logic would differ. Additionally, we covered some of the best ways of handling failures without any data loss and some of the governance principles that can be applied while using Kafka in big data pipeline. We gave you an understanding of how to monitor Kafka and what some of the useful Kafka matrices are. You learned a good detail...