Summary

In this chapter, we went through the Spark Streaming architecture and created a simple Spark Streaming application. This chapter also covered various MLlib packages that provide APIs to run machine learning applications over Spark.

Lastly, we covered the GraphX APIs, covering GraphFrames and graph algorithms implemented over Spark. In the next chapter, we will look at some optimization techniques to run our Spark code efficiently.