In this chapter, we discussed some of the optimizations provided by Spark. First, we discussed some hardware level optimizations, such as setting the number of cores, executors, and the amount of memory for your Spark applications. We then gave an overview of project Tungsten and its optimizations. Then, we covered application-level optimizations, such as choosing the right language, API and the file format for your applications. Finally, we covered optimizations provided by RDD and DataFrame APIs.
This is the end of this book. I thank you for staying with me till the end, and I hope you enjoyed the content. If you wish to explore Apache Spark in more detail, you may also consider buying Learning Apache Spark 2, by Packt.
