Practice of MLP question

Question :

In an MLP which has a single hidden layer with, let's say 10 neurons, and that the network is being trained for 1000 iterations. What is the total number of weights in this MLP?

 

 

Solution :

In an MLP with a single hidden layer and 10 neurons, the total number of weights is the sum of the weights connecting the input layer to the hidden layer and the weights connecting the hidden layer to the output layer.

The number of weights connecting the input layer to the hidden layer is equal to the number of input nodes (assuming no bias nodes) times the number of hidden nodes, plus the bias weights for the hidden layer. Assuming the input layer has n nodes, this would be (n+1) * 10 weights.

Similarly, the number of weights connecting the hidden layer to the output layer is equal to the number of hidden nodes times the number of output nodes (assuming no bias nodes), plus the bias weights for the output layer. Assuming there are m output nodes, this would be (10+1) * m weights.

Therefore, the total number of weights in this MLP would be:

 

(n+1) * 10 + (10+1) * m

 

Where n is the number of input nodes and m is the number of output nodes.

Note that the number of iterations during training does not affect the number of weights in the MLP.

 

 

 

Example :

The total number of weights in the MLP with a single hidden layer of 10 neurons, assuming no bias nodes, would be:

(n+1) * 10 + (10+1) * m

where n is the number of input nodes and m is the number of output nodes.

If we assume, for example, that the MLP has 5 input nodes and 2 output nodes, then the total number of weights would be:

(5+1) * 10 + (10+1) * 2 =

60 + 22 = 82

So, the MLP would have a total of 82 weights that need to be learned during training.