AI ROI questions to ask and the hidden costs of AI

How to estimate and measure the ROI of AI projects

Organizations are quickly learning that artificial intelligence projects are different from traditional software projects. For the latter, resourcing needs are usually fixed and success conditions and outcomes are easily measured. For AI projects, however, those aspects are rarely the case.

Managers of these AI projects should be able to answer the following:

• What business problem am I trying to solve? Make sure that you're targeting a real business problem for your organization and then be able to show how your project will help solve this problem. Start small, think big, iterate often. This mantra will help you work on small projects that can have immediate impact on ROI, while working towards the bigger goal.
• Is this providing my company a competitive edge? Make sure that the time and resources you're investing in the product will result in a competitive edge for your company. Otherwise you're just wasting time, capital and resources that could have been better spent elsewhere. Make sure that there is some way to measure what competitive edge this project is bringing.
• What effect on costs does this project have? The AI application may be saving the company time and resources but, if the project costs twice as much in human capital, compute power and other resources, then it may not be producing the ROI expected. Make sure to always evaluate the entire picture and be able to show what savings or other cost benefits the project is providing.
• What is the "speed to value"? This phrase has become a bit of a buzzword, but projects should be delivering and providing value early and often, especially when taking an agile approach. Managers should be purposeful about goals and make sure that features are being delivered in a timely manner.

The hidden costs of AI

For anyone who has worked with artificial intelligence, they know that there can be hidden costs of implementing these kinds of projects. These hidden costs can take many different forms and need to be planned for or they can cause significant delays. Below are some things of which project managers need to be aware.

• Obtaining training data -- Having a large amount of data doesn't automatically equate to smooth sailing for AI training. One of the biggest challenges to the success of an AI project is a lack of useful training data. Many projects fail due to lack of sufficient training data and obtaining needed data from third-party sources can be expensive. Clarity in respect to what kind of data is required and how it should be accessed is necessary.
• Cost of data preparation and labeling -- Beyond acquiring the right data is the need for solid data preparation. For supervised learning approaches this data also needs to be accurately labeled. Unfortunately, this can be a time-intensive and expensive process. Make sure that you have gathered the correct data for the project, you have access to that data and you get the data prepared, cleaned and labeled before you start training your AI model.
• Licenses and tools -- While there are many open source tools on the market, users need to be savvy and have a certain level of knowledge if they want to use these tools. As a result, there are many tools that help speed up AI projects but require data literacy. However, you need to watch out for license agreements and the actual cost of using the tool as costs can add up quickly.
• Wrong team in place -- Having the wrong team in place can also provide lots of hidden costs for your AI project. Data scientists are very expensive and hard to come by, so it is important to utilize these employees in the right way. Make sure that the data science team you put in place has the right balance of skills and are given the right responsibilities.

Is ROI the most important thing in AI projects?

At the end of the day, companies are looking to implement AI to improve their business. ROI can mean different things to different companies. Companies may implement a recognition system to help increase production. For example, automotive manufacturer BMW uses AI through machines that help sort the millions of parts received from their suppliers. Rather than having humans sort through these parts, AI has helped BMW drastically speed up this process. The measurable ROI is that AI helped to increase the number of manufactured cars.

AI has helped e-commerce sites such as Yelp experiment with their user interface. Artificial intelligence has allowed the company to experiment with button placement, scrolling options, A/B testing of headlines and other important features that influence online conversions. The measurable ROI for Yelp could be improved user satisfaction or longer time on the site, which may increase ad revenue.

While not all projects may need to have a direct impact on a company's bottom line, there needs to be something to show for the investment. After all, without some sort of measurable ROI solving a real business problem that provides more in benefits than the overall costs, you must question why you're doing that project in the first place.