Convolutional neural network (CNN) is a kind of feedforward neural network with deep structure, which includes convolution calculation and is one of the representative algorithms of deep learning.

The research of convolutional neural networks began in the 1980s and 1990s, and the earliest convolutional neural networks were delay networks and LeNet-5. After 2 1 century, with the introduction of deep learning theory and the improvement of numerical computing equipment, convolutional neural networks have developed rapidly and have been applied to computer vision, natural language processing and other fields.

nature

The connection between convolution layers in convolutional neural network is called sparse connection, that is, compared with the full connection in feedforward neural network, neurons in convolution layer are only connected to some of its adjacent layers, but not all neurons. Specifically, any pixel (neuron) in the L-layer feature map of convolutional neural network is just a linear combination of pixels in the receptive field defined by l- 1 layer convolution kernel.

The sparse connection of convolutional neural network has regularization effect, which improves the stability and generalization ability of network structure and avoids over-fitting. At the same time, sparse connection reduces the total number of weight parameters, which is beneficial to the rapid learning of neural network and reduces the memory overhead in calculation.