There’s a widespread consensus that we are in the middle of a new phase of evolution in digital technology – a phase driven by Artificial Intelligence (AI).

75% of businesses expect AI to transform their organisation within three years; 61% believe it will completely transform their industry in the same time period. According to PwC, 52% of companies have accelerated their AI adoption plans in response to the COVID-19 pandemic.

The perceived benefits of AI are manifold and apply across the full spectrum of industries and business functions. AI leads to smarter decision making and better use of data. It bridges the divide between data-led intelligence and action, opening the door to sophisticated, ‘active’ automation. This can be used both to take on the burden of repetitive, labour-intensive but low-value tasks from people, and to perform complex tasks with a higher degree of accuracy and reliability than human beings are capable of.

All in all, it is estimated that AI could add $13 trillion to the global economy by 2030, or a 1.2% annual boost to GDP.

So where’s all the AI?

Yet the road to an AI-enabled future is not entirely without its bumps and obstacles. We can get a sense of this from the fact that, while businesses are betting on AI for the future, current use remains surprisingly low. Nine out of 10 of the world’s largest organisations have investments in AI, but just 15% actively use it at present.

So what’s holding people back? One issue that stands out is that AI development remains expensive and continues to have a high failure rate.

According to figures from Gartner, only around half of AI projects ever make it from prototype to production, while an eye-watering 85% of projects fail to deliver on their business objectives.

This is not exactly unusual in tech investment. From the dot.com bubble to the first cloud boom, we regularly see much-hyped technologies attract a frenzy of investments in projects that ultimately don’t go anywhere. Before the market in a new technology matures, there is a lot of jumping on the bandwagon rather than thinking strategically about how it can best be used.

But the high failure rate in AI projects does suggest another, more fundamental issue. And that’s that AI development is complex and technically challenging. Plug-and-play AI tools that integrate seamlessly with your existing IT assets and deliver all the benefits of AI out of the box do exist. But they tend to be at the more ‘lightweight’ end of the AI spectrum – tools like chatbots, sales and marketing automation etc.

 Once you get into the realm of Deep Neural Networks, self-driving vehicles, autonomous medical robotics and the like, AI becomes a whole different beast. The more complicated the task, the more precise you need the AI system to be (e.g. to drive a car safely or perform complex surgery, rather than just target an advert at someone), the more demanding the task of programming the required algorithms.

Not only that, the most powerful AI platforms simply cannot exist as off-the-shelf pieces of code. They need an entire infrastructure to support them – a cloud back end with an appropriate amount of processing and network capacity, APIs to allow them to work within your broader IT infrastructure. Many of these things have to be custom built on a project by project basis.

Can AI solve its own development challenges?

All of this adds cost to the AI development process. And as well as sheer complexity, there are other factors that contribute to the high failure rate, such as a shortage of developers with specialist AI programming skills and a patchy approach to models and standardisation.

If you use the analogy of building a car, many AI projects can feel like trying to build the entire thing from scratch from raw materials. What makes production more cost effective, efficient and allows you to scale is having a set of ready-made components available that you simply have to bolt together.

As is so often the case with technology, AI may well end up being the answer to the development challenges it creates. We’re starting to see Machine Learning tools applied to some of the more demanding aspects of AI development.

For example, Galileo is a platform that monitors the development process to highlight potential issues in how the AI application will actually work. It focuses in particular on data modelling, aiming to make the complex and laborious task of data ‘training’ more streamlined and efficient.

It’s not quite AI creating AI, or some dystopian future vision of machines self-replicating. But it does highlight how, if more businesses are to benefit from AI in the near future, AI’s powers perhaps need to be turned inwards to address the issue of cost and complexity in development.