If you are familiar with Convolutional Neural Networks (CNNs) and know how these networks work, you understand that the important component of the process is reusable weights. While CNNs may seem unique from this perspective, they are not. The idea of reusing weights to enhance accuracy is critically important, and deep within it lies the power of intuition.

Let us try to understand this intuition with an example. You have developed a CNN that detects dogs in images. Take a step back and try to understand what key pieces of information”learning” are needed to detect a dog in an image. You need to learn about:

• Colors
• Contours (Lines, edges and corners)
• Small shapes

The final learning piece is to learn the combination of these features that correspond to a dog. Neural networks follow a similar approach, learning these features and that “learning” comes through weights.

Now let us assume that we use different weights to “learn” different parts of an image. In this case, each segment of weights has to independently learn what defines a line. This will happen only if there is not a process in place that allows a segment of weights in one part of the network, to leverage “learning” that another segment of weights in another part of the network have developed. This sounds like a drawback. Doesn’t it?

What intuitively comes across as a better approach is to reuse learnings, specifically reusable pieces of intelligence, in multiple parts of the network. That is what the idea of reusable weights is about. Hence one of the most important structural tips of deep learning is that if a network needs to leverage the same type of learning in multiple segments, the goal should be to use the same weights in those segments. This also makes the weights more “smart” by providing them more samples to learn and hence enhances generalization.

The fact is that some of the major developments in the field of deep learning have evolved from this idea. You can view deep learning methodologies like CNNs, RNNs and word embeddings and many more from this perspective. The gist is, when we know that the network needs to leverage the same idea in multiple places, we should force the network to leverage the same weights in those places.