Demonstrating strong Return on Investment (RoI) for any IT project is always important, but for AI related projects it can be critical to the long-term deployment of AI across the your organisation, so getting this right is both a major challenge and a great opportunity with additional complexities that must be considered to ensure the full benefits of any AI project are realised and factored into the RoI metrics.

One of the challenges some clients come to us with is that of defining the return on investment for AI projects and therefore being able to define a strong business case for funding of the project.

One can take a simple view that an AI project is like any other IT project, the fact that the project is going to leverage some form of AI or Machine Learning techniques as part of the implementation can be considered, at a high-level, a detail on the project that should not affect the ROI metrics.

Unfortunately, when you get into the lower-level details, the fact the project involves an AI delivery can require additional factors to be considered.

Below we share some of the key factors to be considered when looking at creating a strong business case to support AI related projects together with highlighting other benefits that often come from running initial AI projects that can then help with the delivery of future deliveries.

Key Factors for Return on Investment

The first factor will be more of a concern if this is the first AI project the company is pursuing and has limited internal expertise. AI projects have an element of experimentation to determine the correct technique that successfully solves the task it is given. This can take time to ascertain the best method and provides a small amount of uncertainty on the projects timeline. This can be managed by putting a sensible timebox around the work, allowing the project the freedom to experience to find the best solution, but with some constraints that allow the focus needed to deliver a working solution.

This time-boxing also allows the possibility that the project may fail. This should not be seen as a failure, but providing valuable insight on the data available and the positioning of the problem, which in turn allows for further investigation to better understand what is possible or what needs to be done (in terms of additional data collection) to make the project work. We acknowledge that this can be very difficult to accept, especially when this might be your very first project. The organisation needs to take a portfolio view for AI projects, rather than just focus in on the individual performance, with this portfolio view, we can also see how even projects that don’t complete can still add value to the organisation and deliver return on investment in different ways.

For any company, they need to see the delivery of AI as a firm wide undertaking, and the benefit of individual AI projects need to be seen at the portfolio level. Widening the skill base of the organisation by doing an AI project has significant ROI implications from a reskilling and staff engagement perspective. This opportunity to train staff with AI and Machine Learning technologies will be a significant benefit to the organisation in the long term, as more and more applications and systems look to benefit from data science and analytics techniques that leverage predictive capabilities. The success of early projects will create a positive acceptance and enthusiasm for AI related projects, giving the organisation confidence and acceptance of AI applied within the organisation that again as multiple positive benefits that must be factored into the ROI.

A company doing AI projects for the first time, will bring out hidden talent within the organisation that wish to get involved because they have some previous knowledge and experience. This allows the company an opportunity to realign its resources and maximise its hidden talent, creating an internal team of people with both a passion for and knowledge of machine learning. Another very positive ROI of these early AI projects.

The initial AI projects will also provide an environment for evaluating some of the different AI platforms, being able to assess them against the firms existing technology landscape. This will be valuable work that future projects will be able to benefit from.

An additional ROI for initial AI projects would be to partner with AI platform / vendor, and to co-create an initial PoC or PoV together, allowing both side to better understand the needs of the other organisation. This is a perfect way to explore a partner relationship, allowing potential integration and onboarding to be done as part of an initial concept or innovation workstream enabling both parties to have better understanding of the requirements and features the other offers. This can be valuable insight and help determine the right AI platforms to align to the organisations current technical landscape. This will also make transparent the pricing model used by the vendor which can inform the longer-term business case.

Other Challenges to Consider

The main challenges are around the data, if you do not understand these and plan for them there is a risk that the RoI will be impacted.

There are several considerations here. The data used during an initial proof-of-concept or proof-of-value will be limited and potentially not have all of the possible variations in the data that production volumes would. Therefore, building a model with a reduced dataset will take less time and provide a perspective on the viability of the method (related to the first point above), but the model will need to be retrained with a production size dataset and this might lead to the need for a different method / algorithm to be used, which extends the experimentation phase of the project. The data may be incomplete, with data quality issues or even missing data that is needed for the success of the model. This will have knock-on implications that add pressure onto the project to solve these issues before the solution can be delivered. The third consideration with the data, is that over time, the production data will change, requiring the model to be monitored in terms of its performance and potentially needed to be regularly retained on the most recent data.

Take a Portfolio Perspective

While we acknowledge that making the ROI transparent to business owners and sponsoring stakeholders can be challenging, there are many additional sources of value that should feed into and feature as part of the wider ROI discussions, even if qualifying them all maybe difficult, they need to be exposed as added value as longer term benefits to the organisation, some of which can still add value even if the individual AI project is, for whatever reason, unable to deliver successfully.

Ultimately, as highlighted in several areas of this article, the best way to view delivery of AI work is as a portfolio of projects rather than just on a case by case basis. This will allow many of the wider benefits to be considered as part of the overall return on investment, allow the freedom for some early AI projects to fail fast while still giving valuable insights into the reasons why and helping to train and educate your staff into the tools, frameworks and workflow processed needed for a scaled professional data science team.