Summary

This concludes our Appendix where we covered some topics based on performance tuning, sizing up your executors, handling data skew, configuring security, setting up a Jupyter notebook with Spark and finally broadcast variables and accumulators.

There are many more topics still to be covered, but we hope that this book has given you an effective quick-start with Spark 2.0, and you can use it to explore Spark further. Of course, Spark is one of the fastest moving projects out there, so by the time the book is out there will surely be many new features. One of the best places to keep up-to-date on the latest changes is http://spark.apache.org/documentation.html, where you can see the list of releases and the latest news.