Sequential API

Keras's Sequential API is straightforward to understand and implement. It lets us create a neural network linearly; that is, we can build a neural network layer-by-layer where we initialize a sequential model and then stack a series of hidden and output layers on it.

Getting ready

Before creating a neural network using the Sequential API, let's load the keras library into our environment and generate some dummy data:

library(keras)

Now, let's simulate some dummy data for this exercise:

x_data <- matrix(rnorm(1000*784), nrow = 1000, ncol = 784)
y_data <- matrix(rnorm(1000), nrow = 1000, ncol = 1)

We can check the dimension of the x and y data by executing the following commands:

dim(x_data)
dim(y_data)

The dimension of the x_data data is 1,000×784, whereas the dimension of the y_data data is 1,000×1.

How to do it...

Now, we can build our first sequential keras model and train it:

1. Let's start by defining a sequential model:

model_sequential <- keras_model_sequential()

2. We need to add layers to the model we defined in the preceding code block:

model_sequential %>% 
layer_dense(units = 16,batch_size = ,input_shape = c(784)) %>% 
layer_activation('relu') %>% 
layer_dense(units = 1)

3. After adding the layers to our model, we need to compile it:

model_sequential %>% compile(
 loss = "mse",
 optimizer = optimizer_sgd(),
 metrics = list("mean_absolute_error")
)

4. Now, let's visualize the summary of the model we created:

model_sequential %>% summary()

The summary of the model is as follows:

5. Now, let's train the model and store the training stats in a variable in order to plot the model's metrics:

history <- model_sequential %>% fit(
 x_data, 
 y_data,
 epochs = 30,
 batch_size = 128, 
 validation_split = 0.2
)

# Plotting model metrics
plot(history)

The preceding code generates the following plot:

The preceding plot shows the loss and mean absolute error for the training and validation data.

How it works...

In step 1, we initialized a sequential model by calling the keras_model_sequential() function. In the next step, we stacked hidden and output layers by using a series of layer functions. The layer_dense() function adds a densely-connected layer to the defined model. The first layer of the sequential model needs to know what input shape it should expect, so we passed a value to the input_shape argument of the first layer. In our case, the input shape was equal to the number of features in the dataset. When we add layers to the keras sequential model, the model object is modified in-place, and we do not need to assign the updated object back to the original. The keras object's behavior is unlike most R objects (R objects are typically immutable). For our model, we used the relu activation function. The layer_activation() function creates an activation layer that takes input from the preceding hidden layer and applies activation to the output of our previous hidden layer. We can also use different functions, such as leaky ReLU, softmax, and more (activation functions will be discussed in Implementing a single-layer neural network recipe). In the output layer of our model, no activation was applied.

We can also implement various activation functions for each layer by passing a value to the activation argument in the layer_dense() function instead of adding an activation layer explicitly. It applies the following operation:

output=activation(dot(input, kernel)+bias)

Here, the activation argument refers to the element-wise activation function that's passed, while the kernel is a weights matrix that's created by the layer. The bias is a bias vector that's produced by the layer.

To train a model, we need to configure the learning process. We did this in step 3 using the compile() function. In our training process, we applied a stochastic gradient descent optimizer to find the weights and biases that minimize our objective loss function; that is, the mean squared error. The metrics argument calculates the metric to be evaluated by the model during training and testing.

In step 4, we looked at the summary of the model; it showed us information about each layer, such as the shape of the output of each layer and the parameters of each layer.

In the last step, we trained our model for a fixed number of iterations on the dataset. Here, the epochs argument defines the number of iterations. The validation_split argument can take float values between 0 and 1. It specifies a fraction of the training data to be used as validation data. Finally, batch_size defines the number of samples that propagate through the network.

There's more...

Training a deep learning model is a time-consuming task. If training stops unexpectedly, we can lose a lot of our work. The keras library in R provides us with the functionality to save a model's progress during and after training. A saved model contains the weight values, the model's configuration, and the optimizer's configuration. If the training process is interrupted somehow, we can pick up training from there.

The following code block shows how we can save the model after training:

# Save model
model_sequential %>% save_model_hdf5("my_model.h5")

If we want to save the model after each iteration while training, we need to create a checkpoint object. To perform this task, we use the callback_model_checkpoint() function. The value of the filepath argument defines the name of the model that we want to save at the end of each iteration. For example, if filepath is {epoch:02d}-{val_loss:.2f}.hdf5, the model will be saved with the epoch number and the validation loss in the filename.

The following code block demonstrates how to save a model after each epoch:

checkpoint_dir <- "checkpoints"
dir.create(checkpoint_dir, showWarnings = FALSE)
filepath <- file.path(checkpoint_dir, "{epoch:02d}.hdf5")

# Create checkpoint callback
cp_callback <- callback_model_checkpoint(
 filepath = filepath,
 verbose = 1
)

# Fit model and save model after each check point 
model_sequential %>% fit(
 x_data, 
 y_data,
 epochs = 30,
 batch_size = 128, 
 validation_split = 0.2,
 callbacks = list(cp_callback)
)

By doing this, you've learned how to save models with the appropriate checkpoints and callbacks.

See also

• To find out more about writing custom layers in Keras, go to https://tensorflow.rstudio.com/keras/articles/custom_layers.html.