AI - Traditional Machine Learning – Numbers and probability prediction
Updated: Feb 2
For an introduction to key terminology on AI and Deep Learning, please refer to this article.
What it is and the value it drives
Organizations are continuously trying to predict numbers, e.g., demand in sales forecasts in order to make capacity adjustments, the price that consumers are willing to pay, or discounts that win customer conversions. On the cost side, businesses often try to predict the price of commodities or inputs, or the right amount to spend on marketing with a positive return. Predicting these numbers today is often a mixture of data analysis, experience, “tribal” knowledge, and crowdsourcing. Some companies do better than others. Governments also require numerical predictions – for example concerning the number of people that may need specific forms of government support or infrastructure.
Traditional ML is a powerful tool to predict numbers. It takes tabular transaction data, uses it to train a model, and outputs a numerical prediction. There are two types of numerical predictions that matter.
The first involves predicting an actual number, for example the price of a product, or how many items of a product should be produced. Companies can use ML models to predict any number of profit and loss drivers, or other KPIs. These models largely leverage linear regression and can also handle a large set of data inputs that are beyond the scope of traditional data analysis.
The second type of prediction involves predicting Yes (1) or No (0) and is done in practice by predicting the probability of either outcome. This has nearly boundless applicability, as businesses can ask an infinite variety of questions such as: “Will this type of customer complete a purchase, or click on an ad?”. Binary predictions are often known as classification problems and are powered by simple statistical formulas such as logistic regression. Many complex ML and DL techniques ultimately boil down to a probability prediction of Yes or No.
The value of this technology for companies lies in increasing the predictive accuracy of various drivers of the business and in the process driving a quantum leap in performance.
Where it is today
Some Japanese corporations are already reaping the benefits of better numerical predictions using traditional ML. Japan's iconic retailer UNIQLO worked with Google to develop Ariake, a system to predict demand. Better demand forecasts result in better production planning which reduces unwanted or unsold inventory. The system uses a combination of customer preferences, sales data, and other input features to predict what the actual demand may be, with some degree of confidence.
Another great application in Japan has been the development of an ML model to predict energy consumption. Because energy consumption drives the operating costs of power plants, precision in forecasting the number of kilowatts or megawatts that need to be produced given expected demand can make an enormous difference. The system was developed by Weathernews and uses the latest weather forecasts and energy consumption data, in order to make predictions of electrical power demand. The system was developed with the cooperation of Sumitomo Corporation, a leading power producer and supplier (PPS), and Summit Energy Corporation which contributed historical power consumption data and experience in supply-and-demand planning.
A final example is Sony's work with Minnano Taxi to support taxi drivers and streamline their work. The project involved the development of a demand prediction service to inform drivers of when, where, and about how many passengers are expected, within finely subdivided areas around Tokyo. The prediction system was built by applying ML to probe data from participating taxi companies, which have status data for nearly 10,000 vehicles.
Today the traditional linear, logistic, or multinomial regression algorithms to make these predictions are widely available, including as a service on major cloud platforms. The way forward will depend on users’ imagination and the will to build and innovate.
How the technology will continue to evolve
Companies looking to get started in driving predictions can do so today. As they continue to move up the learning curve, this technology will get easier to deploy in three ways:
Firstly, more use cases will be implemented and shared on how to apply this technology. There is almost no limit to the types of questions that can be posed, and some of the examples show that businesses need help in predicting various drivers.
Secondly, more context data will be used as inputs to these models. This can include data on weather, sentiment, traffic or other variables that can inform models of external events, so that models are more attuned to how external forces can affect predictions.
And third, more use will be made of AutoML models – ready-made or pre-trained models that can be purchased “off-the-shelf” and that only require feeding with data inputs. For example, retailers looking to improve forecasting can get started on AWS Forecast, which has retail forecasting and demand forecasting. Customers looking to answer predictive questions such as "which customers will buy" can also do so with a few instructions on Google BigQuery.
The key applications
As with the aforementioned examples, there are two broad applications. The first is predicting profit and loss drivers, so as to make better decisions. ML allows for a quantum leap beyond traditional 'data science' which relies on slicing and dicing data for insights. The exhibit shows several examples of the types of predictions ML can answer.
The second major application is predicting operational drivers. These vary by industry sector, as all industries are looking for different outcomes. The Healthcare-Pharma industry for example may be looking to predict the probability that an R&D drug research effort is successful, while an automotive manufacturer may be looking to predict the best market to launch a new vehicle in.