Problems of using data driven solutions

The report “Problems in implementing and using data solutions” was made by Evgeniy Limarenko, CIO Inventive Retail Group , at an offline industry meeting of the Global SIO portal. This format of communication between top IT managers has proven its effectiveness over many years, and the meeting of retail IT managers was no exception. Limarenko’s speech was a brief but vivid summary of many projects in the field of AI, RPA, and neural networks. He identified the main problems of such implementations and outlined possible ways to solve them.

Key thesis: any work with data solutions is an evolutionary process in the company. It is impossible without establishing basic hygienic order in the data. The shit in – shit out principle works like a charm. If there is no base, including infrastructure, then there can be no talk of any predictive, for example, analytics.

Evgeny Limarenko distinguishes three types of data analytics: descriptive, predictive, prescriptive. Descriptive analytics are reporting systems. It shows what is happening here and now. Predictive analytics is used to model situations in the future and answer the “what if” question. Prescriptive analytics provide clues about what to do when certain events occur.

Fig. 1. Problems and solutions of descriptive analytics.

Data quality problems and, in the broadest sense, the lack of a culture of working with them are characteristic of all types of analytics. At the same time, Evgeniy emphasizes, “if you don’t have a basic infrastructure where you correctly collect data, store it and store it, including backup, nothing will work.” Infrastructural problems, including the creation of a full-fledged data warehouse, organizational issues of data management, must be resolved before attempting to move on to building mathematical models and forecasts.

The speaker considers working with users to be critically important. We need to teach business and ourselves to formulate questions that are sought to be answered in the data, and to find answers to them. If the user does not understand how the forecast is built, it will not work, he is sure. This, according to E. Limarenko, also includes users’ misunderstanding of their role and distrust of calculations. When introducing such products, the user’s role changes, and this must be explained to him. He has no task of checking the system, no task of counting “for it.”

Fig. 2. Problems and solutions of predictive analytics.

When a business does not understand what it is doing, does not understand its role, there is no trust in what is happening. If the data is dirty and unstructured, and there are no catalogs of solutions for prescriptive systems, then there is no point in expecting positive results. Whenever some new attributes appear, the system must be retrained. Models must also be constantly updated along with business processes. To do this, analysts must at least be aware that processes are changing.

The problem with mathematical models is that they do not last forever. There are no perfectly tuned models once and for all; they degrade, the speaker emphasizes. Therefore, we need a team that monitors them. Business processes are improved, new data appears, and users strive to improve quality and accuracy. And this is achieved by the emergence of new models, new data and new rules. This is constant, continuous work. There is also a nuance: professional analysts from the CDO office are always trying to “build spaceships”; they are bored with working on trivial tasks. This must not only be kept in mind, but also such a problem must be systematically solved.

Fig. 3. Problems and solutions of prescriptive analytics.

The most important thing, Evgeniy Limarenko is sure, is that “we must explain to the user that his task is not to fight the system, not to check its calculations, but to look for new factors affecting the accuracy of the forecast, and configure the system in the best way through a better formulation of the problem , achieving improved accuracy and quality.”

The talk concludes with a quote from entrepreneur and consultant Michael Gerber: “Let systems run the business and people run the systems. People come and go, but systems remain.”