About a week ago, Matt Turk published his 2016 Big Data Landscape and asked “is Big Data still a thing?”. If you follow the industry pundits, you can get the clear impression that focus and attention has shifted away from Big Data technology, which is now being perceived of being “so 2014”, towards artificial intelligence, machine intelligence and predictive applications.
In fact, this is a false dichotomy, and just as things get technically boring enough to no longer warrant the attention of the Silicon Valley in crowd, the most interesting practical applications open up.
There are two aspects to the question that help us evaluate the importance and validity of the question of Big Data vs. Artificial Intelligence. On the one hand, we need to understand what the practical implications for technology users would be, given Big Data wasn’t “a thing” anymore. On the other hand, what would be the effect for AI or predictive applications?
In reality, technology practitioners and users take a slightly different perspective on technology than Silicon Valley investors or tech reporters. When your work is solving day-to-day problems for your business, your employer or your customers, what’s next is not so much the issue as it is how technology can help you today. In this regard, Big Data technology is performing exceptionally well. In the last few years, we have seen how our customers, retailers across all verticals, have been adopting Big Data technology and have been investing in data storage, data aggregation, data cleansing, data integration, and data consolidation. These investments in basic technology have been and are paying off. Ironically, many of these payoffs are enabled by predictive applications and artificial intelligence.
Looking at the value generated by these investments in Big Data, it comes clear that there is little value in data alone, but that data is the enabler for higher-value outcomes like analysis, insight and decisions. In this hierarchy, decisions are at the highest level and represent the most condensed form of value out of data. But even data-driven business decisions are worthless if not put into action.
And this is where artificial intelligence and predictive applications come to play. Through Predictive Applications, data can be turned into decisions and these decisions can be put into action as part of an automated process. In the past, many of the most complex and interesting applications of AI in business have been held back by either a lack of available training data or a lack of computing power.
The investments made into Big Data technology act as an enabler to practical artificial intelligence in two ways: first, the vast amounts of data collected, now serve as usable training sets for machine learning algorithms that would be useless without this kind of data. For example, in Blue Yonder’s customer targeting product, we rely on customer files and CRM databases that not only record past marketing activity, but actual customer response. Only with both sources of data combined, we are in the position to make an effective decision on who should get a catalog or coupon, and who should not.
The second way how Big Data technology is enabling predictive applications is that the increasing availability of computing power needed for Big Data, and the resulting infrastructure, especially in the cloud, are agnostic to the use case. This means, the same hardware that has yesterday been used for simple number crunching can now be put to use for complex optimization problems, for instance in price optimization.
In conclusion: the shift in attention of startups, investors and blogs from Big Data to artificial intelligence does not mean that Big Data is passé, but that practical applications on top of the collected data and the provided infrastructure have now become possible. It also means that there is lot of reason to get excited about the business impact these predictive applications are having at companies in all verticals.
At Blue Yonder, we are especially excited about the possibilities we can offer to the retail industry. Wherever retailers are making large amounts of decisions, be it in pricing, in the supply chain or in marketing, data availability has been outpacing the evolution of processes and best practices. As a result, retailers that are investing in predictive applications and automated decision have the highest potential to capitalize on this opportunity and get a competitive edge.