What Everyone Should Know About Machine Learning

In recent months, the author has had the opportunity to discuss artificial intelligence, particularly machine learning, with many decision-makers. Their investors have been asking executives about their machine learning (ML) strategies and where they have implemented ML. This article explores how this technical topic has suddenly become a discussion point in company boardrooms.

Computers should solve tasks for humans. The traditional approach is to "program" the required process—in other words, we teach the computer a suitable problem-solving algorithm. The algorithm is a detailed description of a process, similar to a recipe. Algorithms can effectively describe many tasks. For example, in elementary school, we all learned algorithms for adding numbers. When it comes to executing algorithms quickly and perfectly, computers far outperform humans.

However, this approach has its limitations. How do we identify a photo of a cat? This seemingly simple task is difficult to construct as an algorithm. Let's think about it. Even simple instructions such as "has four legs" or "has two eyes" have their drawbacks because these features might be hidden, or the photo might show only part of the cat. Then we encounter the next task of recognizing legs or eyes, which is as difficult as recognizing the cat.

This is precisely where the power of machine learning comes in. Instead of having to develop an algorithm to solve the problem, computers use examples to learn their own algorithms. We train computers using samples. Using our cat example, this might mean training the system with a large number of photos depicting correspondingly labeled cats (supervised learning). In this way, the algorithm develops and matures, ultimately enabling the recognition of cats on unfamiliar images.

In this case, computers typically do not learn classical programs like model parameters, such as edge weights in a network. This principle can be compared to the learning process in our brains, where connections between nerve cells (neurons) are adapted. Like the brain, unlike classical programs, such networks with edge weights are almost impossible for humans to interpret.

A special artificial neural network learning method has proven particularly successful in this case. Deep learning is a specialty of machine learning, which in turn is a sub-discipline of AI—an important branch of computer science research. As early as 2012, a research team at Google successfully trained a network of 16,000 computers to recognize cats (and other object categories) from images using 10 million YouTube video segments, using the method of deep learning.

Many practical problems belong more to the category of "recognizing cats" than "adding numbers" and therefore cannot be adequately solved with human-written algorithms. This is often a matter of recognizing patterns in certain data—for example, recognizing objects in images, recognizing text from language, or attempting to detect fraud in transaction data.

For a simple example, let's look at predictive maintenance. Imagine many sensors sending data streams, and occasionally a machine fails. Therefore, the challenge is to understand the patterns in the data stream that ultimately lead to failure. Once these patterns are understood, they can be identified during normal operation to anticipate and prevent potential failures.

Although the principles of machine learning are not new, it is currently gaining popularity. There are three main reasons: first, the availability of large amounts of data required for applications and training ("big data"). Second, we now have enormous computing power, especially in cloud computing. Third, a series of open-source projects have made algorithms accessible to everyone.

Machine learning does not replace traditional programming but complements it. It provides tools that enable us to additionally solve major problems that have been difficult or even impossible to master. Altogether, these provide new opportunities, and existing systems are increasingly adapting to machine learning capabilities.

Repetitive operations following patterns are a typical example. Imagine a computer program that accesses one hundred functions through a series of complex menus, but you can only actively use some of them daily. By observing the steps you usually take, the computer can learn to predict your next action, thereby improving efficiency. Or, the distribution and transformation of data (for example, for ETL jobs that populate data warehouses); where the computer "learns" repetitive data and objects, many steps can be automated and accelerated.

There are more examples in almost every field: learning materials suited to individual students (especially "Massive Open Online Courses" or MOOCs), early diagnosis of diseases, correct online marketing target groups, customer churn, automatic identification of data quality issues, or matching user profiles through daily services.

With its advanced tools, Spark (used in combination with Hadoop) has become the leading big data framework in machine learning. Talend is also adopting this approach, but adding a layer by modeling the work (including deployment in training and production). Modeling reduces complexity while resulting in a certain degree of independence from the underlying technologies, which continue to change rapidly and are therefore accessible only to a few experts.

Only a few experts need to truly understand the finest algorithmic details in the machine learning field. On the other hand, understanding the concepts of ML is beneficial for everyone. ML is essentially learning patterns from examples and being able to apply them to new data sets. Ultimately, this expands the category of problems that can be solved with machines and therefore automated: particularly through decision-making processes. This is exactly what computers learn; they make decisions about new data based on knowledge accumulated from training data. On one hand, through automated decision-making, we can take advantage of this—regardless of our business or circle. On the other hand, we ourselves represent a constant data source that other people's machines will analyze to optimize their own businesses.

In summary, computers can now not only follow explicit instructions but also learn through examples. Depending on the challenge, one program may be more suitable than the other. However, the two programs can be combined in infinitely many ways, ultimately bringing more opportunities for automation.