At the moment supervised learning is the most common class of ML problems in the business domain. In Chapter 1, Predict the Class of a Flower from the Iris Dataset, we approached the Iris classification task by employing a powerful supervised learning classification algorithm called Random Forests, which at its core depends on a categorical response variable. In this chapter, besides the Random Forest approach, we also turn to yet another intriguing yet popular classification technique, called logistic regression. Both approaches present a unique solution to the prediction problem of breast cancer prognosis, while an iterative learning process is a common denominator. The logistic regression technique occupies center stage in this chapter, taking precedence over Random Forests. However, both learn from a test dataset containing...

The best way to get started is by understanding the bigger picture—gauging the magnitude of the work ahead of us. In this sense, we have identified two broad tasks:

• Setting up the prerequisite software.
• Developing two pipelines, starting with data collection and building a workflow sequence that could end with predictions. Those pipelines are as follows:
• A Random Forests pipeline
• A logistical regression pipeline

We will talk about setting up the prerequisite software in the next section.

First, please refer back to the Setting up the prerequisite software section in Chapter 1, Predict the Class of a Flower from the Iris Dataset, to review your existing infrastructure...

A good way to start this section off is to download the Skeleton SBT project archive file from the ModernScalaProjects_Code folder. Here is the structure of the Skeleton project:

Instructions to readers: Copy and paste the file into a folder of your choice before extracting it. Import this project into IntelliJ, drill down to the package "com.packt.modern.chapter", and rename it "com.packt.modern.chapter2". If you would rather choose a different name, choose something appropriate. The breast cancer pipeline project is already set up with build.sbt, plugins.sbt, and build.properties. You only need to make appropriate changes to the organization element in build.sbt. Once these changes are done, you are all set for development. For an explanation of dependency entries in build.sbt, please refer...

Before getting down to the implementation of a logistic regression pipeline, refer back to the earlier table in section Breast cancer dataset at a glance where nine breast cancer tissue sample characteristics (features) are listed, along with one class column. To recap, those characteristics or features are listed as follows for context:

• clump_thickness
• size_uniformity
• shape_uniformity
• marginal_adhesion
• epithelial_size
• bare_nucleoli

• bland_chromatin
• normal_nucleoli
• mitoses

Now, let's get down to a high-level formulation of the logistic regression approach in terms of what it is meant to achieve. The following diagram represents the elements of such a formulation at a high level:

The preceding diagram represents a high-level formulation of a logistic classifier pipeline that we are aware...

In this chapter, we learned how to implement a binary classification task using two approaches such as, an ML pipeline using the Random Forest algorithm and an secondly using the logistic regression method. 

Both pipelines combined several stages of data analysis into one workflow. In both pipelines, we calculated metrics to give us an estimate of how well our classifier performed. Early on in our data analysis task, we introduced a data preprocessing step to get rid of rows that were missing attribute values that were filled in by a placeholder, ?. With 16 rows of unavailable attribute values eliminated and 683 rows with attribute values still available, we constructed a new DataFrame.

In each pipeline, we also created training, training, and validation datasets, followed by a training phase where we fit the models on training data. As with every ML task...

We will now list a set of questions to test your knowledge of what you have learned so far:

• What do you understand by logistical regression? Why is it important?
• How does logistical regression differ from linear regression?
• Name one powerful feature of BinaryClassifier.
• What are the feature variables in relation to the breast cancer dataset?

The breast cancer dataset problem is a classification task that can be approached with other machine learning algorithms as well. Prominent among other techniques are Support Vector Machine (SVM), k-nearest neighbor, and decision trees. When you run the pipelines developed in this chapter, compare the time it took to build a model in each case and how many of the input rows of the dataset were classified correctly by each algorithm.

This concludes this chapter. The next chapter implements a new kind of pipeline, which is a stock...