President at Elephant Scale, responsible for AI and cybersecurity training.
getty
A CEO friend asked me, “Will AI ever be able to offer an ROI for enterprises?” I thought the answer through and wrote it down. Then, I realized that this information could be beneficial and valuable to many people, so here’s my answer, based on my experience in the artificial intelligence (AI) and cybersecurity training industry.
Let’s begin by defining where we stand and ask the same question about IT projects in general: “Will IT ever be able to offer an ROI for enterprises?” Here, the statistics are known to be about a 30% success rate. So, if AI, with a current success rate of about 15%, can reach the IT project success rate, we can consider the mission accomplished.
What are the best practices for achieving success with AI? First, the quality of the AI answers should be “good enough.” The measure of “good enough” is called the “error rate.” For example, IBM’s translation in the 2010s was based on rules extracted from translations of Canadian parliament speeches. This rule-based approach was fragile and one had to add a manual check. So, an AI project based on the IBM technology of the 2010 timeframe would not deliver an ROI. Today, the accuracy of Google Translate reaches the 94% range. This quality is good enough, and people find it satisfactory.
In this instance, by “people,” I mean lawyers who are using it to translate the documents in their cases. Why am I choosing lawyers for my use case? Lawyers have a low tolerance for mistakes. So, when an AI algorithm offers its services, they demand precision from it.
Second, the results must not be hard to use. Many unsuccessful AI projects have an issue with problematic results. In these cases, AI may be sold as a time-saver but it’s not intuitive and straightforward. In our lawyer scenario, the usual solution is to force its use through sales or ask the judge to require its use. It works for a while but fails in the long run. One possible way to promise a good ROI is to use technologies of today that offer human-like quality, such as BERT models for natural language processing and self-supervised training instead of instructor-led training.
Third, and just as important as the previous suggestions, the AI project must include the discipline of machine learning engineering or MLOps. For many readers, MLOps may be a new word. It derives from DevOps, a trend to combine developers and operations. MLOps is based on designing the AI systems end-to-end, from model scoping to data needs planning to modeling strategies and, finally, to the deployment environment. It then applies best practices and progressive delivery techniques to maintain and monitor a continuously operating production system. Today, many AI scientists can run their AI in their lab environment but are often uninformed about putting it into a production pipeline. This is why MLOps is crucial to success.
Today, we see examples of an ROI on AI being possible, especially in companies that apply the above three principles. Let’s look at some examples: In 2019, it was estimated that Amazon’s Alexa would generate $19 billion by the end of this year. Google’s BERT is improving search quality 10% of the time. People no longer need to conduct searches on Google using specially selected keywords. Instead, they just ask the question they need to be answered. This helps Google solidify its search market share, which is estimated at 92.47%.
And the ROI doesn’t have to be measured in straight dollars. Instead, it can help the company achieve another part of its business strategy. For example, mining companies in Australia have improved job safety and productivity by 20%. And lest you say that AI is only for large companies, I will address the AI strategies for small- to mid-size companies in my next article.
In summary, if your AI technology meets these criteria, it should offer an ROI for the enterprise. Education in the above three principles may well be a prerequisite for successful AI projects.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
