The Post-Algorithmic Era Has Arrived
By Bill Franks, Dec 14, 2017
Last week, IIA hosted our annual Predictions and Priorities webinar, as well as the associated research brief. When we sat down to determine what we should focus on this year, Tom Davenport and I both immediately raised a trend that we’ve recently been discussing with organizations. After reconciling our semantics, we realized that we were both excited about the same base trend. I want to reiterate it here as I think it is a critical trend to understand and adapt to. Namely, “the post-algorithmic era has arrived”.
Does This Mean Algorithms Are a Thing of The Past?
Of course not! In fact, algorithms are being embedded into more and more business processes every day. If anything, analytics processes built on top of algorithms are going to continue to rise in importance for the foreseeable future. If that’s the case, then what does “post-algorithmic era” mean?
The defining factor of the post-algorithmic era is that having access to algorithms, and knowledge of how to execute them, is no longer a differentiator. Not long ago, a large part of the value proposition for an analytics professional was:
Knowing how to use the then-few (and complex) software tools that contained algorithms, and
Knowing from experience which specific algorithms might work best in any given situation
It took a high degree of skill to use the analytics software packages. At the same time, processing was so expensive that it was necessary to limit the options explored. So, analytics professionals with the two listed skills differentiated themselves as highly valuable.
What is The New Reality?
The reality today is that algorithms are ubiquitous and widely available. Algorithms are not only available in commercial software, but many algorithms (and the software packages that contain them) are open source and free. Furthermore, it is possible to simply rent algorithms within a simple, user-friendly interface for little cost on the cloud. This is a far cry from the days where major investments in hardware and software were a necessary precursor to even thinking about building meaningful analytics processes. Simply knowing how to access algorithms is not a differentiator today.
Similarly, processing cost has decreased substantially such that today we can literally try every possible algorithm that is appropriate for a given situation and let the results tell us which is best. This has long been a common approach in machine learning. In the past, analytics professionals added value by being able to tightly narrow and target the set of algorithms to try. Today, it is so inexpensive to try all the options that even if you’re confident you’re right about the direction to go, it still makes sense to validate through testing. In practice, knowing what works best (and when) isn’t a differentiator anymore either.
How Do You Adapt?
The post-algorithmic era will be defined by analytics professionals who focus on innovative uses of algorithms to solve a wider range of problems as opposed to the historical focus on coding and manually testing algorithms. We can now let cheap processing test all the viable options for a given problem while we consider what to do with the results.
It is critical to note that analytics professionals will still have to ensure a problem is well defined, that the data is properly prepared, and that the portfolio of algorithms being tested is theoretically sound. The fact that the testing of the options will be automated does not negate these base needs.
2018 is the time to begin implementing tools to do the automated testing of algorithms for you. Many tools, both open source and commercial, are available today to enable this. Or, you could build your own. The key is that your organization must move to this higher level of automation to remain competitive in the analytics space.
Evolving The Value Proposition of Analytics Professionals
Simultaneously, the value proposition of analytics professionals will continue to evolve. With time freed from basic coding and testing, analytics professionals can redirect that time and energy to solving more problems and operationalizing results. Operationalizing is a critical step that has only recently begun to get the attention it deserves. One implication of this is that business acumen, already an important skill for analytics professionals, will be even more important in the future than it is today.
Analytics has helped to disrupt software, data architectures, and business processes over the past few decades. It is now time for us to disrupt how we in the analytics community do things. This means changing some long standing habits and updating how we define our value propositions.
Originally published by the International Institute for Analytics
About the author
Bill Franks is IIA’s Chief Analytics Officer, where he provides perspective on trends in the analytics and big data space and helps clients understand how IIA can support their efforts and improve analytics performance. His focus is on translating complex analytics into terms that business users can understand and working with organizations to implement their analytics effectively. His work has spanned many industries for companies ranging from Fortune 100 companies to small non-profits.
Franks is the author of the book Taming The Big Data Tidal Wave (John Wiley & Sons, Inc., April, 2012). In the book, he applies his two decades of experience working with clients on large-scale analytics initiatives to outline what it takes to succeed in today’s world of big data and analytics. Franks’ second book The Analytics Revolution (John Wiley & Sons, Inc., September, 2014) lays out how to move beyond using analytics to find important insights in data (both big and small) and into operationalizing those insights at scale to truly impact a business. He is an active speaker who has presented at dozens of events in recent years. His blog, Analytics Matters, addresses the transformation required to make analytics a core component of business decisions.
Franks earned a Bachelor’s degree in Applied Statistics from Virginia Tech and a Master’s degree in Applied Statistics from North Carolina State University. More information is available at www.bill-franks.com.
A look at the year ahead in the world of analytics
Accelerate your organization’s journey to analytics maturity
Get the data sheet to learn how the Research & Advisory Network advances analytics capabilities and improves performance.