Activation Function

The activation function represented by equation (3.3) changes output values at a threshold and is called a "step function" or a "staircase function." Therefore, we can say, "a perceptron uses a step function as the activation function." In other words, a perceptron chooses a "step function" as the activation function from many candidate functions. When a perceptron uses a step function as the activation function, what happens if a function other than a step function is used as the activation function? Well, by changing the activation function from a step function to another function, we can move to the world of a neural network. The next section will introduce an activation function for a neural network.

Sigmoid Function

One of the activation functions often used in neural networks is the sigmoid function, represented by equation (3.6):

...