The Difference Between Machine Learning and Deep Learning

The Difference Between Machine Learning and Deep Learning

The Difference Between Machine Learning and Deep Learning

In this article we’re going to explore the two main ways of building an AI: machine learning and deep learning, explain them and compare them to one another.

But first, we need to try and understand what machine learning, deep learning and some other commonly used terms mean.

Machine Learning and Deep Learning Simplified

Generally speaking, Machine Learning and Deep Learning are two different ways to achieve Artificial Intelligence.

 

Machine Learning relies on the computer being fed information and assimilating it, “learning” in the process, while Deep Learning relies on the computer “simulating a brain” and figuring things out by itself.

Although they’re different, Deep Learning is at its core Machine Learning, only in a more complex way.

But which way is the best? What are the differences between the two and what are the similarities?

First, we need to try and understand what machine learning, deep learning and some other commonly used terms mean.

Then, we can see in what ways they’re similar to each other and how they differentiate themselves.

 

What is AI?

As most people know, AI stands for Artificial Intelligence. At a basic level it’s pretty simple to understand: a mechanical-type of intelligence, opposed to the natural intelligence humans and other animals display.

 

Although the concept can be traced way back to ancient Greece, to imaginings of non-natural beings with intelligence, the actual term “Artificial Intelligence” was first coined in 1956 by John McCarthy, when AI was founded as an actual academic discipline.

When we talk about AI we’re talking about intelligence applied to technology, or to computers if you will.

The term is usually mentioned whenever a machine performs a cognitive task that normally only a natural intelligence-possessing being would be able to perform, such as problem solving or recognizing patterns.

 

Several Types of AI

Of course AI comes in several different “flavors”.

AIs are usually optimized to perform a certain task — meaning that they are never as complex as a human mind.

Some AIs are really good at playing chess, but they’re unable to predict the weather.

Others are built to learn repeating patterns in images and replicate them and are unable to do anything else.

We can distinguish AIs in three different types: narrow,general, and super.

Narrow AI

Narrow AI is the only type of AI we can achieve so far. It’s the type of intelligence that’s only good for a certain task (or a few certain tasks).

It’s sometimes referred to as “weak AI”, but not due to actually being weak — they’re just not intelligent at a human, or near-human level.

Narrow, or weak AIs can still perform tasks that would take a regular human (or a team of humans) years to achieve, even if they’re not good for anything else.

General AI

General AI, sometimes referred to as “strong AI” is the AI type that’s closest to human intelligence.

So far it has been unachievable, even though it’s quite complicated to define what human intelligence actually entails. 

General AIs would relate to their environment the same way a human would.

They would be able to perform various, sometimes simultaneous, tasks, just like we do.

Even though computers are millions of times better than us at analyzing and processing raw data, they’ve never been able of thinking abstractly or coming up with original ideas.

Some scientists believe that general AI is “just around the corner”, even if they’ve been saying that for a number of years.

On the other end of the spectrum there are those who believe that we will never need such type of AI.

Super AI

If narrow AI is a weak AI, and general AI is a strong, human-level AI, then it’s quite obvious what a super AI Like the name indicates, a super AI would theoretically surpass human intelligence in ways we can’t imagine.

A super AI would be better than us at everything, from more academic and scientific efforts, all the way to creative and social endeavors.

Of course that if general AI is still an unattainable dream, super AIs won’t be coming any time soon.

Although it’s worth considering that there are those who believe that there will be a super short distance between general and super AI.

Our robotic friends will continuously improve themselves until they become our robotic overlords.

 

Machine Learning

Then, as a way of building artificial intelligence, we have machine learning. It’s possible to build AI’s without resorting to machine learning, but it’s simply not worth it.

 

To manually create AI would require millions upon millions of lines of highly complex code.

So, as an “easier” alternative, we can tell machines to learn the task themselves, without them being necessarily programmed to perform that particular task.

 

What is Machine Learning?

Machine learning, at a very simple level, is basically feeding data to a computer to train it. The machine — or, more specifically, the algorithm — improves itself based on the data it has been fed and it learns.

Most machine learning requires constant input by humans and being fed mostly-labeled data sets, although unsupervised algorithms exist as well.

 

How Does it Work and What is it Used For

It’s easy to understand machine learning if we think of a few examples.

Think of a software built to predict the risk of fire in a given area.

This software predicts the possibility of a fire happening in the area based on the data it has (stats from previous fires, correlations between temperature rise and fire, etc.) — the more data you feed the system, the more accurate the model becomes.

The program learns and adjusts its model according to the data you provide.

As it fails to predict a fire, or as it successfully predicts it, you then feed that data back into the system, allowing it to optimize itself and to become better at predicting fires.

There’s an example of machine learning people who use social networks usually encounter when they upload a new photo.

Facebook, for instance, tries to suggest people to tag on photos users have uploaded .

It may seem strangely accurate — how did it know it was Mark in the picture?

Well, after Mark has been tagged a few times on different pictures, the algorithm begins to understand what exactly is Mark by analyzing the pixels that define him in a given photo.

After a while, it’s able to “see” Mark whenever he appears on a photo and suggests his profile for you to tag.

What’s actually happening is that you are feeding the algorithm data sets labeled “Mark”, and whenever you upload a new photo the algorithm checks if the data you now provided bears any resemblance to Mark’s data.

You also encounter examples of machine learning outside of Facebook.

Whenever you have to solve a picture-based Captcha — those where you’re asked to click on every car, or every street sign – you’re effectively optimizing a system that recognizes the objects you’ve tagged.

 

What is Deep Learning?

Deep learning is at its core machine learning. However, it works — pardon the pun — at a deeper level than machine learning. Deep learning tries to mimic the human brain, by using Artificial Neural Networks (ANNs).

 

ANNs are built of “neurons” which form several “brain layers”. Each layer then tries to learn a specific feature.

As layers stack on top of each other, each with its different purpose, depth is achieved — hence the name.

 

How Does Deep Learning Work?

While machine learning mostly requires being fed labeled data, deep learning breaks unlabeled data into smaller chunks and hierarchizes it, trying to figure out which parts are more relevant to the task at hand.

For instance, let’s say we want to differentiate pictures of squares from pictures of triangles.

With machine learning we would need to tell the algorithm that squares have four sides while triangles just have three.

The algorithm would then look at the pictures provided, count the number of sides and sort them accordingly.

With a deep learning algorithm we wouldn’t need to tell the algorithm how many sides each geometrical figure has.

A deep learning algorithm would look at a square, break the square down into lines, count the number of lines, figure out how each line relates to each other and then do the same to the triangle, differentiating them.

It would move from the simpler task (breaking figures down into their basic components) to the more complicated ones (telling them apart).

After a while the algorithm would be able to instantly tell if new pictures were of triangles or squares.

Of course that for this to happen, deep learning algorithms need loads of data — way more than regular machine learning algorithms.

 

How do Machine Learning and Deep Learning Compare?

Now that we have a simple understanding of what these terms are and how they behave, lets try to understand how they compare to each other.

 

We’ve seen that deep learning seems to be better, but keep in mind that it requires much more data than simpler machine learning algorithms.

 

Performance vs Amount of Data

This is perhaps the biggest difference between the performance of machine learning and deep learning algorithms.

The smaller the amount of data is, the worse the deep learning algorithm performs.

As we said, deep learning algorithms need large amounts of data to be able to perform optimally.

Machine learning algorithms work better with lower amounts of data due to being handfed information.

Obviously, with larger amounts of data, deep learning algorithms outperform simple machine learning ones.

 

Hardware Requirements

While machine learning algorithms can work on lower-end machines, deep learning algorithms require complex and sophisticated hardware.

Due to the amount of matrix multiplication operations that deep learning algorithms perform, they require several GPUs, which are built for that very purpose.

So, using deep learning algorithms is much more expensive than using machine learning ones.

 

Is Labeling Data a Requirement for Deep Learning?

Deep Learning and Machine Learning need different types of data in order to work – Machine Learning, as we’ve seen, needs labeled data while Deep Learning doesn’t.

Feature engineering is basically labeling data in order to make it less complex so patterns become more evident and algorithms can work.

It’s a very difficult and complex process.

In machine learning data needs to be labeled by a human and hand-coded into the algorithm. The algorithm’s performance will vary depending on the labels provided.

Deep learning algorithms work differently. They don’t need to be handfed labels like machine learning algorithms do.

They try to learn lower-level features such as lines and work their way into progressively more complex features.

 

Machine Learning and Deep Learning Solve Problems Differently

Generally, to problem solve using a more common machine algorithm we break the problem down into smaller chunks to solve each part individually, combining them at the end.

Deep learning works differently by solving the entire problem at once.

Of course that for this to be a viable option you need to take into consideration the drawbacks of deep learning algorithms, such as the need for large amounts of data and the large power consumption.

 

How Long Does It Take To Train Each Algorithm?

Due to the complexity of deep learning algorithms, training them to perform certain tasks can take days or even weeks.

On the other hand, machine learning algorithms can be trained in a matter of hours (sometimes even less).

However, once trained, deep learning algorithms perform their tasks much faster than most machine learning algorithms when considering similar amounts of data.

As you increase the amount of data, deep learning algorithms exponentially outperform most machine learning ones.

 

Are Both Machine Learning and Deep Learning Outputs Easy to Interpert?

Understanding why a deep learning algorithm gave a certain result may require reverse-engineering the system to see which “neurons” were activated, which may be slightly too complicated for simpler tasks.

Machine learning algorithms, such as decision trees, work with a stricter set of rules, so it’s easier to understand the reasoning behind their output.

 

Where are Machine Learning and Deep Learning Being Used Right Now?

Machine learning has been around for a while and is widely used. Here are a few common use cases of machine learning:

 

Deep learning has also found a number of practical applications in recent days and is expected to revolutionize lots of industries.

Netflix recommendation system and Tesla or Google self-driving cars are examples of deep learning applications.

Where as before machine learning algorithm research was basically only done academically, these days industries are in the vanguard of machine learning research.

They see the potential — and don’t want to be left behind.

Siri, Alexa and even Microsoft’s Cortana also use deep learning to simplify speech recognition.

Health industries use it to locate cancerous cells and other indicators of serious diseases.

Deep learning is something that still has a lot of untapped potential.

These things have become so popular that even Google allowed us toy around with the subject by using Google DeepDream.

Thousands of people used Google’s neural network to create dream-like psychedelic images, even if they didn’t quite understand the concept of a neural network.

So…

 

What Does the Future Hold for Deep Learning?

It’s difficult to predict specific usages of deep learning. We know it will be used and it will be a staple in a number of technologies — there’s already a large number of jobs asking for “deep learning engineers”.

 

Whatever new technologies we develop in the next few years, chances are deep learning will be involved. Of course it is impossible to guess what those might be.

 

Final Remarks

We hope to have succeeded in giving you a overview of what artificial intelligence, machine learning, and deep learning are, how they correlate with each other and what differentiates them.

It’s obvious that these explanations have been very broad and simplistic — these are very specific sciences that take years of study to be able to fully grasp.

However, as you’ve seen, the general concepts are easily understandable and you can imagine the practical applications of these algorithms.

For now, we just have to wait for what the future holds.

We all want to see what exciting new technologies will come due to deep learning algorithms, and we probably will, very soon.

If you have any comments or any doubts regarding these matters let us know, we’ll try to help you.

 

An Introductory Guide to Machine Learning

An Introductory Guide to Machine Learning

An Introductory Guide to Machine Learning

It can seem from the mainstream media that every company is working on a machine learning project that promises to increase their revenue, eliminate competition, and provide revolutionary business insights.

With big promises like that, you might wonder what machine learning is and how it can help your business.

This guide will help to answer some of your questions.

 

We’ve put together an introductory guide to machine learning that briefly covers all significant areas of machine learning as a field.

We hope that you can pick the topics that apply to your area of work or interest, and do further research with more confidence.

This article will teach you a basic definition of machine learning, how a machine learning process actually works, and different types of problems this field can solve.

After that, we will look at a few common algorithms before jumping into discussing challenges and limitation of machine learning.

Happy learning! ?

1. The Definition of ML

The definition of machine learning is not set in stone. There are many different variations out there, and their nuances depend on whom you ask and what field they work in.

 

Before looking at the definitions, let’s place machine learning in the realm of other practices it’s closely related to: computer science, artificial intelligence, deep learning, and data science.

 

1.1 Machine learning vs. AI vs. deep learning

Traditional media often portraits deep learning as a synonym for AI. That is incorrect.

Machine learning is sometimes identified as one of the main techniques to achieve true AI, but it is just a subfield of AI, not its counterpart.

Deep learning is further a subfield of machine learning, which makes both areas subfields of AI.

What is missing from this venn diagram is computer science, which would encapsulate all three fields.

Data science is another term that you might have heard in relations to machine learning.

Data science is a bit more challenging to place, as it is an umbrella term that includes machine learning, as well as other disciplines of computer science that have little to do with AI.

1.2 Machine learning in one elegant sentence

When searching for a complete definition of machine learning, one has to turn to the leading authorities in the field.

In their course on AI, University of Helsinki defines machine learning as “systems that improve their performance in a given task with more and more experience or data”.

Andrew Ng, a Stanford University professor and a Coursera co-founder, has another definition. In his widely popular online course on machine learning, Andrew defines machine learning as “the science of getting computers to act without being explicitly programmed”.

That is, as you will learning in the guide, not entirely accurate, as a big part of the machine learning process requires much effort from data scientists.

Another prominent figure in the machine learning field is Tom Mitchell.

His definition of machine learning came out already in 1997. He explained machine learning with a problem, which read:

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E”.

What can you take from these definitions? First of all, machine learning revolves around computers progressively getting better at problem-solving.

Secondly, the problem-solving process should happen as autonomously as possible.

All this is achieved through the use of various algorithms that are designed to enable learning and improvement over time when exposed to new data. It is the algorithms that enable a computer to make data-driven decisions.

Let us now peek behind the curtain and see how machine learning processes work.

2. How Machine Learning Actually Works

A lot of information online about machine learning talks about what a machine learning algorithm can do, but not a lot about how it can do it.

 

To truly understand what machine learning is, and demystify the field, it can be useful to know how one goes about working with a machine learning project.

 

2.1 What machine learning process looks like

The flowchart below is a good representation of all the steps in a machine learning process. It might look complicated at first but fear not.

It is sufficient to know only a few basic steps to get an entry level understanding of machine learning.

A more detailed look at the machine learning process. Source: https://machinelearningmastery.com/4-steps-to-get-started-in-machine-learning/

Step 0: Decide on a specific business problem to solve

This may seem simple, even trivial, yet surprisingly many companies fail in their efforts to get value out of machine learning simply because they don’t ask the right questions.

The kind of business problem you want to solve with the help of machine learning determines everything in the steps that follow. From the type of algorithm you use, the data you gather, to the metrics you’ll use for performance evaluation of your model.

 

Step 1: Gather and prepare the data

Once you know what problem you need to solve, you can start collecting the data you need. For example, if your company wants to detect clients that are at risk of churn, you might need to collect data about their purchase activity, customer service interactions, basket size, etc.

Each of these factors is regarded as one feature of the data. The more complicated the problem at hand is, the more features a data set will need to have.

Data preparation process consist of actions like removing duplicates, formatting the data sets, randomizing the order of the data entries, and checking for data imbalances.

 

Step 2: Choosing a machine learning algorithm

An algorithm is a step-by-step guide that tells a computer how to solve a given task.The algorithms are heavily based on statistics and mathematical optimization.

There are many different algorithms that each fit different problems. Some are better suited to work with images, other with sounds or text files. Size and quality of your data will also play a deciding role. For example, a data set with a lot of features will require longer training time, because the algorithm needs more time to make sense of the data.

What algorithm you should choose also heavily depends on what you want to do with the results the algorithm will produce. How accurate does your result need to be?

How long time do you have to train the model? As training time and accuracy as closely related – one tends to go down with the other – answers to these questions will push you in the directions of different algorithms.

A good data science project will also use several types of algorithms to build robust models that help companies make decisions across a variety of business challenges.

 

Step 3: Training the algorithm

In this step, an algorithm becomes your model. By introducing the chosen algorithm to your training data set, it learns the patterns and correlations between the data, thereby establishing rules for future use.

As a rule of thumb, you should divide your data into a training set (about 70-80% of your data) and a validation set (remaining 30-20%). The training time will depend on the complexity of the algorithm and the amount of data you have.

 

Step 4: Evaluation of the model

The validation data set you set aside in the previous step will come in handy now. Because the model has never seen this data before, the performance of the model on this data set will be an indicationof how well the model will work in real life.

What metrics should you choose to evaluate the performance of your model on training data? The answer is: it depends. Dean Abbott put it best in his book “Applied predictive analytics”:

“If the purpose of the model is to provide highly accurate predictions or decisions to be used by the business, measures of accuracy will be used. If interpretation of the business is what is of most interest, accuracy measures will not be used; instead, subjective measures of what provides maximum insight may be most desirable.”

 

2.2 Who performs the machine learning process

All of the steps mentioned above are done by a group of data scientists.

Ideally, each step will be executed by different data scientists who are specialized in their respective areas.

If you have trouble wrapping your head around why you might need a whole team of data scientists instead of just one, think of a machine learning project as a menu at a restaurant.

A menu would typically include starters, a main, a dessert, and maybe some snacks. All of these dishes require different skill sets to prepare.

Baking is not the same as making a main, savoury dish. And even there, there are huge differences in how you prepare meat, seafood or vegetables.

You can be good at all of these dishes, but that would make you an outlier, not the norm.

In the same fashion, each step of the machine learning process calls for different skill sets.

3. Types of machine learning problems

Often, when you read articles about machine learning, you will stumble upon a question or a phrase that refers to “machine learning problems”.

 

These are not problems that machine learning as a field has to overcome or solve, but the different ways a machine learning algorithm can learn to perform a task.

There are three core types of machine learning: supervised, unsupervised and reinforced learning.

3.1 Supervised learning

In supervised learning, you begin with a labeled data set to show the algorithms what the correct output should look like.

The term supervised itself refers to the fact that data scientists need to tell the algorithm what they want it to predict The job of the algorithm is to learn the patterns in the data and when introduced to a data set different from the training set, make correct predictions on its own.

For that reason, supervised learning is sometimes referred to as predictive modeling. Supervised learning can be further divided into two groups: classification and regression.

Both can be applied to the same questions, but they will produce vastly different results.

For example, let’s take the problem of determining tomorrow’s weather. A classification algorithm will produce the answers “Hot” or “Cold”, while a regression algorithm will predict a value for the temperature that day.

So, what does this example tell us about these two concepts?

3.1.1 Classification problem

Classification is about predicting a discrete value, such as “yes/no”, “spam/not spam”, or “dog/muffin”. In other words, you are asking a model to group data entries together in two (or more) groups.

If you only have two groups to divide your data in to, you are working with a binary classification. If there are more than two, you have a multi-class classification.

3.1.2 Regression problem

Regression, on the other hand, is about predicting a continuous value. In other words, you aim to predict a number that ranges between – infinity and + infinity. In doing so, the regression algorithm also estimates the relationship between two or more variables.

To solve a regression problem, you need a set of data with predictor (explanatory) variables and a continuous response variable (outcome or target).

Once the underlying relationship (or lack thereof) is uncovered, it can be applied to new data sets in the future to make real-life predictions. Unlike the classification problem, there are many different regression types.

In its purest form, regression shows the relationship between one independent variable (X) and a dependent variable (Y), as in the formula below:

 

All regression models start off with this formula and get progressively more elaborate as we increase the number of independent variables, complexity to the data distribution, and look at different types of dependent variables.

Here is a short list of types of regression you will encounter as you learn more about machine learning:

  • Linear regression
  • Logistic regression
  • Polynomial Regression
  • Gradient Descent
  • Ridge Regression
  • Lasso Regression
  • ElasticNet Regression

3.3 Unsupervised learning

In unsupervised learning, you start with data sets without labels or description and ask the algorithm to find the structures in the given data.

Unsupervised learning is mainly used to find patterns, rules, and groups, which show meaningful insights and describe the data better to whoever needs to use it. In other words, you use unsupervised learning when you don’t know what the data can tell you.

Often, the algorithm might be able to teach you new things after it learns patterns in data.  Unsupervised learning can also be used to tackle more complex data sets that can’t simply be clustered into clear groups or patterns.

This is often referred to as “the cocktail party” problem, and in such cases, unsupervised algorithms are used to find structure in a chaotic environment. The name of this problem stems from the famous example that was used by Andrew Ng in his Stanford course on machine learning.

In his example, Andrew plays a recording of a man speaking while some music was playing. Two microphones were used to tape the recording:

The result of applying an unsupervised algorithm to these recording resulted in two new recording where the voice and the music were clearly separated.

As ordinary as it may sound to you, the fact that a simple algorithm was able to separate between different audio wavelength without instruction is incredible.

3.4 Reinforcement learning

Reinforcement learning can best be described by the saying “learning by doing”. This subfield of machine learning teaches a computer about its environment by allowing it to  perform actions and see the results.

The idea behind reinforcement learning is that a computer can learn from the environment by interacting with it and receiving rewards for performed actions.

This closely resembles how you learned as a child, where you will modify your future actions based on the incoming feedback from your current ones.

The goal of reinforcement learning is to maximize the expected cumulative reward. This is based on the Reward Hypothesis that states that all goals can be described by the maximization of the expected cumulative reward where long-term rewards get less weight than the short-term rewards, because the probability of getting the long-term results is lower.

There are two types of reinforcement learning tasks: episodic or continuous. An episodic task has a start and an end, for example, a game of Super Mario Bros. The computer receives feedback at the end of each episodic task, and adjust the behavior for the next task accordingly.

A continuous task runs until a human terminates it, and the feedback is constantly evaluated by the computer. A good example of a continuous task is a stock market trading algorithm.

3.5 Deep learning

It’s impossible not to mention deep learning when talking about machine learning. It’s a subfield of machine learning that is making some of the most interesting breakthroughs.

However, it is difficult to talk about deep learning in great details in this guide, as it is a subfield that deserves its own article.

What is deep learning? What sets deep learning algorithms apart from other machine learning algorithms are their capabilities.

Basic machine learning models can learn progressively, but they still need training and well-labeled data. If a machine learning model makes an inaccurate prediction, a data scientist needs to examine the problem and make adjustments accordingly.

A deep learning model, on the other hand, can train itself and determine on their own if the results are accurate or not.


Source: https://www.upwork.com/hiring/for-clients/log-analytics-deep-learning-machine-learning/

Deep learning is the closest you can come to the futuristic promised of AI, and some of them are already a reality.

Tesla is using deep learning to power their self-driving cars. Google is using deep learning both for pattern recognition in photos and to power one of their most used product – Google search engine.

4. Types of The Most Popular ML Algorithms

Now that you are familiar with the different types of machine learning, we can talk about the stars of the show - the algorithms.

 

We are not going to go into detail about each algorithm for several reasons. Firstly, there are too many to count.

Secondly, the intimate knowledge of each algorithm belongs to the higher level of machine learning education.

Furthermore, it’s more important to understandhow you choose the appropriate algorithm for your specific problem rather than knowing the names of multiple algorithms.

This was covered in section 3 of this guide. Finally, there is a wide-accepted theorem in machine learning called “No Free Lunch”. The theorem states that no one algorithm works best for every problem.

For example, you can’t say that neural networks are always better than decision trees or vice-versa. As we covered in section 3 of this guide, there are many factors that will affect the choice of algorithm for your particular problem.

Instead, you should try applying your data to many different algorithms.

Here is a quick rundown of machine learning algorithms that are used more frequently in solving real-world problems:

5. Challenges And Limitations

What the most pressing challenges of machine learning depend on whom you ask. A data engineer might have a different answer than a data analyst because they approach machine learning problems from different angles.

 

There are, however, a few universal issues that most members of the machine learning and data science fields seem to agree on.

These challenges are what stands between machine learning and its ultimate potential.

5.1 The need for data

In many ways, machine learning algorithms are worse than babies. They need to be taught everything from scratch, which in machine learning terms translates to the need for lots and lots of data.

The problem is the cost of collecting and processing all that data. The majority of machine learning algorithms that exist are supervised, which mean that the data need to be properly labeled and converted to a single format.

The latter is seldom the case in an average company. Most companies have their data in PDFs, Excel sheets, online database and even paper.

Formatting all of these documents into a uniform format takes time.

Labeling data can be costly too. Annotating and labeling of data needs to be done by hand, and the more intricate problems you work with, the high will be the cost.

For example, annotating pictures of different animals can be done by practically anyone. In other words, it would not be a high-paying job.

Reviewing and labeling MRI scans of tumors, on the other hand, will require a trained eye of a health practitioner who is familiar with tumors and how they show up on the scans.

Such a professional will require higher pay.

5.2 Black box AI

The black box AI applies largely to deep learning. Nonetheless, this challenges deserves a spot on this list as it touches upon one of the most fundamental things in human society: trust.

As of today, no one really knows how the most advanced deep learning algorithms arrive at the solutions to the problems they are asked to solve. Not even their creators.

Even the data scientists who build these models may struggle to identify the reason for any single action.

This may become a big problem in many industries as the companies who use deep learning algorithms to make decisions won’t be able to explain the reasoning behind them.

If a mortgage application is turned down or an individual is selected for additional screening at airport security, the bank and airport must be able to explain why.

The inner workings of deep learning algorithms must be made more understandableto their users and creators for several reasons.

Firstly, to keep the tech accountable.

Secondly, to predict when failures might occur.

Lastly, without the why, people won’t be able to trust the technology to make decisions that are in their best interest.

5.3 The hype

The field of machine learning, AI in particular, is in a state of overhype at the moment.

While it makes it easier for new and experimental projects to get funding, the hype also introduced some problems.

Firstly, we have the expectations. People’s expectations of what machine learning can do far exceed what is possible today. Gary Marcus, a professor of psychology and neural science, and a former director of Uber AI Labs, addressed the negative effects of machine learning hype in his widely-discussed paper. He says

“One of the biggest risks in the current overhyping of AI is another AI winter. [..] When high-profile figures […] promise a degree of imminent automation that is out of step with reality, there is fresh risk for seriously dashed expectations. Executives investing massively in AI may turn out to be disappointed. Already, some major projects have been largely abandoned, like Facebook’s M project.”

Another area where the hype is having a negative impact is the access and the cost of acquiring talent.

With the role of data scientist voted “the sexiest job of the 21st century”, many people without the necessary skill sets have transitioned into the field.

This has made it harder to find people with the technical ability to understand and implement it machine learning algorithms in a proper way.

Those who can, on the other hand, have become high in demand. The average salary of a data scientist is have grown with 36% in the last 5 years, ranging between $4,393 – $30,729.

The cost and difficulty of finding the right people for the job put a lot of machine learning projects on hold, thereby creating a backlog of exciting machine learning discoveries.

Peter Kudlacek is a CEO at Apro Software . He has been in software development business for the last 15 years. He succesfully built several IT companies.

Pin It on Pinterest