AI with Convolutional Neural Networks

In this article, we will discuss Convolutional Neural Networks. How they're different from regular neural networks and how they're used. Deep learning neural networks have empowered modern computing tasks. Providing breakthroughs in task processing that couldn't realistically be performed without deep learning. One such breakthrough has been convolutional neural networks – or CNNs – a special type of neural network. A convolutional layer is a neural network layer that isn't connected and includes weight-sharing units.

CNNs use many copies of an identical neuron or unit. The benefit being that a network can have many neurons and take on complex computational models while minimizing the parameters taken on by the neurons. Because they're identical neurons, they share the same behavior values. Convolutional neural networks have been very popular for their implementation in image and audio processing. A big challenge with typical neural networks is that they consider an entire input all at once.

The solution to this involves preprocessing, which is powerful but limited in its implementation, or convolutional input, which grabs small portions of an image and looks at just those. Convolution input refers to overlapping functions and input. Using a filter to obtain a portion of an image, feeding what's in the filter to a convolutional neural network neuron. Then the filter is moved, usually by a pixel. And the process is repeated until the entire image has been processed.

When discussing convolutional neural networks, we speak of pooling layers, which are interwoven with convolutional layers. They're processing layers used for down-sampling. If you've used an image editing programs like Photoshop, then you'll be familiar with down-sampling which is the process of taking an image file and shrinking its size. While that sounds pretty straightforward, a lot goes on in the background to shrink an image and make it look like the original at the end because down-sampling requires that information, pixels, be removed from an image and you can't just randomly choose a pixel of a specific color and remove it. The area around that pixel has to be analyzed and an understanding is made about what the area is trying to represent.

The pooling layer simplifies the input by reducing the number of parameters. It also reduces computations and overfitting, a statistical anomaly that occurs when data noise obfuscates any relationship that the data might normally indicate. The pooling layer is typically inserted between convolutional layers. Max pooling takes a maximum of a sample of fields to make new fields. For example, a 2x2 field of an image is analyzed and the number of pixels in that area is reduced by two on the width and height to make a new parameter and reduce a number of pixels in that area by 75%. There are some common uses for convolutional neural networks.

Things like images processing, game playing, character recognition and image tagging are ideal for CNNs. Artificial intelligence use convolutional networks to assess the current state of a game and make decisions based on that. Games like Go and some of the older Atari arcade games have been popular subjects for convolutional neural networks.

CNN Architecture

In the convolutional neural network, there are several layers beginning with the input layer and here there are four inputs. [x1, x2, x3, and x4.] Then there's the convolutional layer or layers, the rectified linear unit activation layer, pooling layers, or layers for down-sampling, and the fully connected layer or layers, [y1, y2, and y3.] then the SoftMax layer, which normalizes the data is next.

The convolutional layer extracts different features from each part of the image. The filter assigns weights to any area of an image. For example, a 5x5 filter would represent a 5 pixel by 5 pixel selection of the image. The feature map is a new image that's created when the filter is applied to the input. Stride occurs when the filter moves across the image. Think of an imaginary box that surrounds a 5X5 section and moves to the left by one pixel at a time when processing is complete – by one pixel when processing is complete on the 5X5 section. Stride size is how much to move, so a stride of 1 would move the filter by 1 pixel. Pooling combines variables into one. So, for example, taking the average value of 4 pixels. [Example CNN: LeNet-5]

LeNet-5 was one of the first convolutional neural networks. It was developed to recognize characters from 32x32 images. And the neural network has 7 layers – convolutional layer with 5x5 filters resulting in 6 28x28 feature maps, pooling layers, which down-sample images to a 14x14 feature map, 2 more convolutional layers, and 1 more pooling layer, 1 fully connected layer, and 1 output layer. [LeNet-5 Visualized] Here's a visualization of LeNet-5. The input is a 32x32 image, like the hand-drawn X here. A convolutional layer with 5x5 filters resulting in 6 28x28 feature maps. Pooling layers down-sample the image to feature maps, two convolutional layers, one fully connected layer, and one output layer. Then finally, the SoftMax layer normalizes the data and the output is the letter X.

Convolutional Neural Network Disadvantages

The weakness of CNNs lays in the amount of data you provide to them. If you provide them with less, expect the CNNs to perform poorly. CNNs have millions of parameters and with small dataset, you would run into an over-fitting problem because they needs massive amount of data to quench the thirst. So, if you provide enough data, CNNs is more strong and willing to give you better performance.

Overfitting is the most common problem of CNNs for example with image processing, when we change camera and illumination, most CNNs cannot perform well.

From practical perspective, ANN and CNN are easy to implement, but the code for CNN can be a bit longer than a NN due to additional max-pooling and the last layers consisting of fully-connected layers in case of classification.

Convolutional Neural Network Disadvantages

The usage of CNNs are motivated by the fact that they can capture / are able to learn relevant features from an image /video at different levels similar to a human brain. This is feature learning! Conventional neural networks cannot do this.

Another main feature of CNNs is weight sharing. CNN is more efficient in terms of memory and complexity. Imagine NNs and CNNs with billions of neurons, then CNNs would be less complex and saves memory compared to the NN. In terms of performance, CNNs outperform NNs on conventional image recognition tasks and many other tasks. Look at the Inception model, Resnet50 and many others for instance.

For a completely new task / problem CNNs are very good feature extractors. This means that you can extract useful attributes from an already trained CNN with its trained weights by feeding your data on each level and tune the CNN a bit for the specific task. Eg : Add a classifier after the last layer with labels specific to the task. This is also called pre-training and CNNs are very efficient in such tasks compared to NNs. Another advantage of this pre-training is we avoid training of CNN and save memory, time. The only thing you have to train is the classifier at the end for your labels.

Convolutional Neural Networks AI Jobs

These are some positions which can help you have an idea what recruiters are looking for in terms of experience, coding language, education, certificate ... etc.

Convolutional Neural Networks AI Videos

These are some videos related to Convolutional Neural Networks and artificial intelligence. Enjoy!

Convolutional Neural Networks Code Practice

Would you like to practice your coding skills? Here are some great resources:

More information: Convolutional Neural Networks offers a huge list of benefits to all. The usage of Convolutional Neural Networks is such that it cannot be limited to only one activity. Its growing popularity has allowed it to enter into some of the most popular and complex processes like Artificial Intelligence (AI), Machine Learning (ML), natural language processing, data science etc. We hope this page was helpful and provided you with some information about AI with Convolutional Neural Networks. Check out our main page for more components of artificial intelligence resources.