Data science and experimentation are typically seen as two sides of the same business coin. While this is true to a certain extent, it’s a simplistic view that overlooks a number of important nuances in the relationship between the two. The most significant of these is the fact that experimentation within the realm of data analytics can be harnessed in two quite distinct ways depending on what business outcomes are desired.
Of course, when the primary desired business outcome is a greater sense of understanding and stability in a highly uncertain operating environment, such as the one created by Covid-19, the value of both of these main forms of data science experimentation is highlighted.
The first level of experimentation takes place within the realm of data analytics itself and is applied by a data scientist in order to find answers to specific questions or problems. This could be referred to as the ‘exploration’ variety of data analytics experimentation. Then there is a second form of experimentation, which is also undertaken by the data scientist, but is not aimed at answering a specific question. Rather it is a form of experimentation that could be referred to as ‘prospecting’, and it requires that the data scientist be given at least a measure of freedom or empowerment in order to ‘dig around in the data’ and see if anything of value emerges. Of course, as with any type of prospecting, there’s a good chance that this type of experimentation doesn’t deliver any results that are of massive significance to the business. But at worst, it might help to identify actions that should be avoided; and at best, it could uncover the next massive vein of business gold that could be mined by the organisation for many years to come.
The obvious problem with this second form of ‘prospecting’ experimentation is that the potential for a significant return on investment is limited. Which is why most companies are understandably unsure about throwing money at it. After all, in a challenging environment, where success at cost cutting has become a key performance indicator, it’s already difficult to justify the often-high salaries paid to data scientists due to pure supply and demand dynamics. And adding a layer of experimentation to their job descriptions, can make it even more difficult to fully quantify the bottom-line value, if any, that they add. So, it’s understandable that some businesses are reticent to make such an investment that offers no real guarantee of generating significant returns.
In contrast, most companies are quite comfortable with their data scientists spending a good amount of their work time using experiments to explore solutions to clearly defined problems. Especially since this type of work has clarity on expected outputs and places a measure of accountability on the data scientist for delivering the best solution in the end. However, the mind of your average data scientist is wired for creativity, discovery and prospecting experimentation. It is what they are taught to do for many years while acquiring their university degrees. So, when an organisation is unwilling to give them the freedom, at least for some of their working day, to do such prospecting, there’s a very good chance that they will quickly move on to an employer that will. What’s more, by preventing data scientists from using their science and skills to do these types of general experiments, a business could very well be missing out on identifying the next big idea or opportunity that could propel it to massive success, or at least, significant competitive advantage.
So, as is the case with so many sound business strategies, balance is key. While data scientists undoubtedly need to earn their keep by finding solutions to identified business problems, they also must be given the room to conduct experiments that may, or may not, achieve outcomes that are of any real value to the business. Achieving such balance is usually a product of a clear understanding of the fact that, within any business environment, the sought-after culture of innovation is, in reality, a combination of disruption and optimisation. As such, the data analytics function has to exist and operate very close to the business and have a clear view of its strategy and desired outcomes. At the same time, the analysis of data must be given permission to be disruptive. The ratio of such a disruption focus in relation to business-needs driven outcomes will differ from business to business, and is usually a function of various factors, particularly the size, budget and maturity of the organisation.
At the same time, the potential value of any level of ‘prospecting experimentation’ will always be directly proportional to the extent to which the data analytics function is given a voice in the business. So, even if you are a forward-thinking company with the budget to allow your data scientists to spend as much as 50% of their work time doing such blue sky experimentation if you are not willing to give them the opportunity to discuss their findings at an executive level, and really invest in understanding those findings, there’s every chance that you’ll be wasting their time and your money.
But if a business is willing to invest in the infrastructure, allow its data analysts the time and freedom to experiment, and bridge the often wide communication gap that exists between data science and business strategy, the potential exists that it could reap significant returns, and even become an industry game-changer. And even if it doesn’t, empowering data scientists to do what they do best, will almost certainly open the door to enhanced strategic thinking, which may well translate to diversified income streams across the organisation’s full value chain. Which, today, is a vital ingredient for business sustainability and certainly worth the investment.
Covid-19 has reinforced the importance and value of both of these types of data science experimentation. The pandemic represents a situation in which businesses are required to find solutions to problems they have never encountered before, and for which little to no historical data existed. In many cases, even identifying the specific problems that need solving is challenging, let alone coming up with solutions to them. In this type of unknown scenario, prospecting and exploration really are your only viable options; and this has meant that organisations which had already recognised the importance of giving their data scientists the freedom to experiment went into the Covid-19 crisis with a relatively healthy advantage, and could well emerge from the crisis with something of a head start on competition in the post-Covid-19 world.
By Dr Yudhvir Seetharam, Head of Analytics, Insights and Research: FNB Business