Apache Spark: Tips, Tricks, & Techniques: Discover Techniques to Test, Immune, and Parallelize Spark Jobs

Tomasz Lelek

Can$165.99

Video Nov 2018 2hrs 26mins 1st Edition

Tomasz Lelek

Can$165.99

Video Nov 2018 2hrs 26mins 1st Edition

What do you get with a video?

Download this video in MP4 format

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

View table of contents

Download Code

Key benefits

• Speed up your Spark jobs by reducing shuffles
• Leverage the Key/Value API in your big data processing to make your jobs work faster with lower network traffic
• Test Spark jobs using the unit, integration, and end-to-end techniques to make your data pipeline robust and bullet proof

Description

Apache Spark has been around for quite some time, but do you really know how to get the most out of Spark? This course aims at giving you new possibilities; you will explore many aspects of Spark, some you may have never heard of and some you never knew existed. In this course you'll learn to implement some practical and proven techniques to improve particular aspects of programming and administration in Apache Spark. You will explore 7 sections that will address different aspects of Spark via 5 specific techniques with clear instructions on how to carry out different Apache Spark tasks with hands-on experience. The techniques are demonstrated using practical examples and best practices. By the end of this course, you will have learned some exciting tips, best practices, and techniques with Apache Spark. You will be able to perform tasks and get the best data out of your databases much faster and with ease. All the code and supporting files for this course are available on Github at https://github.com/PacktPublishing/Apache-Spark-Tips-Tricks-Techniques

Who is this book for?

This course is for Apache Spark developers who want to learn some useful, interesting ways to further improve their Apache Spark skills for processing and analyzing a large amount of data in a much faster, unique, and easy way.

What you will learn

• Compose Spark jobs from actions and transformations
• Create highly concurrent Spark programs by leveraging immutability
• Ways to avoid the most expensive operation in the Spark API—Shuffle
• How to save data for further processing by picking the proper data format saved by Spark
• Parallelize keyed data; learn of how to use Spark's Key/Value API
• Re-design your jobs to use reduceByKey instead of groupBy
• Create robust processing pipelines by testing Apache Spark jobs
• Solve repeated problems by leveraging the GraphX API

What do you get with a video?