Deep Learning

What is deep learning?

Deep learning is one of the most interesting branches of Artificial Intelligence which has been applied to several areas such as Computer Vision, Speech Recognition, Natural Language Processing, Audio recognition, and etc. Deep learning is based on Machine Learning, Artificial Neural Network, and Represented Learning. Deep Learning, as a branch of Machine Learning, employs algorithms to process data and imitate the thinking process, or to develop abstractions. Deep Learning (DL) uses layers of algorithms to process data, understand human speech, and visually recognize objects. Information is passed through each layer, with the output of the previous layer providing input for the next layer. The first layer in a network is called the input layer, while the last is called an output layer. All the layers between the two are referred to as hidden layers. Each layer is typically a simple, uniform algorithm containing one kind of activation function.

Historical Trends in Deep Learning

The history of deep learning coma back to 1943, when Walter Pitts and Warren McCulloch designed a computer system based on the neural network of the human brain. They used a combination of an algorithm and mathematics they called threshold logic. The earliest efforts in developing Deep Learning algorithms came from Alexey Grigoryevich Ivakhnenko (developed the Group Method of Data Handling) and Valentin Grigorʹevich Lapa (author of Cybernetics and Forecasting Techniques) in 1965. They used models with polynomial (complicated equations) activation functions, that were then analyzed statistically. From each layer, the best statistically chosen features were then forwarded on to the next layer (a slow, manual process). During the 1970s the first AI winter kicked in, the result of promises that couldn’t be kept. The impact of this lack of funding limited both DL and AI research. Fortunately, there were individuals who carried on the research without funding.

How Deep Learning works?

As you may know, deep learning is based on a traditional neural network which has a limit number, 2 or 3, of the hidden layer. However deep learning contains many hidden layers. The deep learning model learns from large sets of labeled data and neural network architecture that learns features directly from data without needing manual feature extraction.

One of the popular forms of the deep neural network is convolutional neural networks (CNN or ConvNet) which are mostly suitable for 2D data, actually, CNN uses 2D conventional layers.

the above picture shows a network with many conventional layers. CNN learns to detect features of images using hundreds of hidden layers. Every layer detects a specific feature of a layer. For example, the first layer detects the edges of the picture and other layers detect other complex features. The complexity of the picture is going to rise in the next layers.

Difference between Machine Learning and Deep Learning

Deep learning is specialized in Machine Learning. In machine learning model learns from manually extracted features but deep learning that can be called end-to-end learning extracts features of data, for example, an image, and then categorize them. In simple words in deep learning, a network is given raw data and a task to perform, such as classification, and learns how to do this automatically.

Deep Learning applications

Deep learning application is going to rise in the future due to its ability to solve the problems and finding new optimal ways for doing tasks and fixing problems. some of its applications are:

• Self-driving cars
• Deep Dreaming
• Healthcare
• Natural Language Processing (NLP)
• Entertainment
• Photo Description
• Language Translations
• Virtual Assistance
• Fraud Detection
• Personalizations
• Visual Recognition
• Colorisation of Black and White images
• Detecting Developmental Delay in Children
• Automatic Handwriting Generation
• Demographic and Election Predictions
• Pixel Restoration
• Automatic Game Playing

Types of deep learning algorithms

1. Convolutional Neural Networks (CNNs)
2. Long Short Term Memory Networks (LSTMs)
3. Recurrent Neural Networks (RNNs)
4. Generative Adversarial Networks (GANs)
5. Radial Basis Function Networks (RBFNs)
6. Multilayer Perceptrons (MLPs)
7. Self Organizing Maps (SOMs)
8. Deep Belief Networks (DBNs)
9. Restricted Boltzmann Machines( RBMs)
10. Autoencoders