We can verify the setup by simply downloading the sample code from the Spark source tree and importing it into IntelliJ to make sure it runs.
Running a sample ML code from Spark
Getting ready
We will first run the logistic regression code from the samples to verify installation. In the next section, we proceed to write our own version of the same program and examine the output in order to understand how it works.
How to do it...
- Go to the source directory and pick one of the ML sample code files to run. We've selected the logistic regression example.
If you cannot find the source code in your directory, you can always download the Spark source, unzip, and then extract the examples directory accordingly.
- After selecting the example, select Edit Configurations..., as shown in the following screenshot:
- In the Configurations tab, define the following options:
- VM options: The choice shown allows you to run a standalone Spark cluster
- Program arguments: What we are supposed to pass into the program
- Run the logistic regression by going to Run 'LogisticRegressionExample', as shown in the following screenshot:
- Verify the exit code and make sure it is as shown in the following screenshot:
