Training process, Weights and Back-propagation
The system learns to recognize every characteristic of our cat during the training process, gradually correcting the importance of the data that flows between the levels of the network.
This is possible because an attached “weight” is associated with each link between layers, the value of which can be increased or decreased to alter the importance of that link.
At the end of each training cycle, the system will examine whether the final output of the neural network approaches or moves away from the expected result.
For example, if the net is improving or worsening in identifying the cat breed.
To reduce the gap between the actual output and the desired output, the system will operate backwards through the neural network, altering the weights connected to all these links between the levels, as well as an associated value called a bias.
This process is called back-propagation.
Types of Neural networks
Neural networks are divided into many categories, with different strengths and weak points but also with different uses.
Let’s see some of them!
1/ Recurrent neural network
Commonly used for text-to-speech conversion. It is a type of artificial neural network in which the output of a particular layer is saved and sent back to the input, helping to predict the outcome of the layer. For example, the most likely next word in a speech.
Each node acts as a memory cell while computing and carrying out operations. Thanks to this, the neural network remembers the information it may need to use later.
2/ Feedforward neural network
One of the simplest networks. In fact, it can have just one layer. Unlike in more complex types, there is no backpropagation, and data moves in one direction only.
On the other hand, it is easy to maintain and able to deal with data that contains a lot of noise.
So, it’s used in technologies like face recognition and computer vision, where target classes are hard to classify.
Sequence to sequence model
It consists of two recurrent neural networks. An encoder processes the input and a decoder processes the output.
They can do so using the same or different parameters.
This model is particularly applicable where the length of the input data is not the same as the length of the output data.
For this reason, it is used in chatbots and question answering systems.
3/ Convolutional neural networks
A CNN contains one or more convolutional layers.
These layers can either be completely interconnected or pooled.
Before passing the result to the next layer, the convolutional layer uses a convolutional operation on the input that makes the network much deeper but with much fewer parameters.
Because of this, it’s probably the most well-suited for image recognition and in agriculture, where weather features are extracted from satellites to predict the growth of a field.
Neural networks and Supervised learning
Today, almost all the highly successful neural networks use the supervised training approach.
Among them, we can find FFNN, RNN, LSTM, FFNN, CNN, GAN, and U-Net.
You don’t have to remember all these acronyms. It’s just to prove my point and to impress you, ok?
The only frequently used neural network connected to unsupervised learning is Kohenon’s Self Organizing Map (KSOM), which is applied for clustering high-dimensional data.
In case you never heard about supervised or unsupervised learning, you can get a more precise idea by reading our previous article on the different categories of machine learning.
Supervised and Unsupervised learning
If, on the other hand, you are too lazy, or you have just broken all your fingers and cannot click on the link, we will summarize everything by saying that, in Supervised learning, we teach machines by example.
In this approach, we feed our computer using previously classified examples, knowing that there is a relationship between the input and the resulting output.
The machine will recognize the connection between them, and process a function that mathematically represents it.
In contrast, Unsupervised learning is used with data that hasn’t been previously classified by humans into different categories.
We use this system to explore the data and identify any internal structures or patterns.
The machine will identify groups, and find out for itself relationships or similarities in the data to split them into different categories and groups, known as clusters.
“Today, almost all the highly successful neural networks use the supervised training approach.”
My friends, that’s all for today.
I hope you enjoyed the article!
You can find other interesting stuff about Machine learning and AI in our blog.
I know that it’s interesting because i wrote it.