Writing a streaming program using Kafka Streams

In the previous recipe, we wrote a very basic Kafka producer and consumer. That might not be sufficient when you would like to work with your data. In fact, the consumer we wrote was not smart enough to filter out geolocations with invalid latitudes and longitudes. These are things you would want to do when you build a streaming application. In this recipe, we will be utilizing Kafka's Streams API to create a Kafka Streams application that will stream messages from a new topic called geolocationStreams, filter out bad geolocations, and forward the valid geolocations to the geolocations topic. This will make sure that we store only valid geolocations.

The following diagram depicts the flow of our application:

Getting ready

1. Before we write our Kafka Streams application, let's delete any existing geolocations in the data directory. Execute the following command in a new terminal shell:

   rm /opt/packt/geolocation/data/*
   

2. Now that you have cleaned...