Smart Chips: How AI is Shaping the Future of Processing

Smart Chips: How AI is Shaping the Future of Processing

Dec 16, 2020

It’s probably not something you have ever considered, but is instantly recognisable the moment it is pointed out to you. Isn’t it strange how most conversations about Artificial Intelligence (AI) – most articles written about it, for that matter – are nearly always focused on software, with only occasional thought given to the hardware that has to run these wonderful futuristic programmes?

Ok, so the main exception to this premise is robotics, which represents a very specialised (and fascinating) branch of AI in its own right. But autonomous thinking robots aside, most talk about AI is focused on cutting edge developments like Machine Learning, Natural Language Processing, Computer Vision and all the rest. These are all the innovations that make the very concept of AI possible, and are all examples of software – beautifully complex algorithms rendered in a programming language.

We might say that it is software that delivers all the marvels that make AI such a fascinating subject – intelligent automation that refines responses as it ‘learns’ from patterns in data, quasi-sentient machines that use and understand human language, computer systems with hitherto unimaginable analytical and predictive powers.

But behind the scenes, less conspicuous because it is focused more on the means rather than the ends of AI, there is another quiet revolution going on that has every chance of transforming the very nature of computing as we currently understand it. Largely missing from mainstream conversations about AI, this conceptual breakthrough is all about hardware, the physical foundations of computing that are increasingly hidden deeper and deeper from sight.

We’re talking about the AI chip.

The need for speed

Let’s start with the basics – what is an AI chip, how do they differ from conventional computer chips, and why are they necessary?

Top hop around those three important questions a little, the reason why specialised AI chips have emerged is because AI is seriously compute-heavy. This shouldn’t be a surprise to anyone. All of those astonishing things AI software is capable of, from accurately reading intentions in natural speech and replying appropriately to identifying, remembering and adjusting to tiny nuances in input don’t happen by magic. They happen because AI depends on gargantuan feats of data processing performed at incredible speeds.

Once you start getting into the really heavyweight AI solutions, such as neural networks and deep learning, you soon reach a point where conventional computing architecture is just not fast and efficient enough to deliver the processing required. This applies to conventional computer chips – there comes a point where they start to be an impediment to what AI programmes could achieve. Or, to be more precise, running AI solutions using conventional chips starts to become prohibitively expensive.

That, in a nutshell, explains the need for AI chips. But how are they different? This is where things start to get interesting for the potential impact on computing in general. Until recently, the truth is that AI chips haven’t been all that different to conventional chips. Yes, they have been faster and more efficient, but they have generally been based on the same basic architecture as high powered GPUs – the chips that give serious gamers the flawless graphic performance they want from their PC. What AI chip makers have done is take the GPU concept and push it to its limits, focusing the processing power available on very precise applications.

Hard wired for cognition

But the real breakthrough has come recently with the emergence of chips that are capable of running what might be described as the twin pillars of any self-learning AI system – the ‘training’ of the system, which involves running huge volumes of data through it so it can ‘learn’ the patterns, and the application side, which sees the system use what it has learned to make inferences that guide output actions. This dual ability is known as parallelization, or the simultaneous execution of different processes in parallel. Because the training element of Machine Learning in particular is so resource-intensive, to date AI systems have relied on dedicated chips performing the training and inference roles separately.

Consolidating both into a single chip therefore brings considerable efficiency benefits – so much so that the introduction of parallelization in chip architecture is potentially one solution for an issue that has long troubled computer scientists. This is the fact that, as demand for processing power and speed (driven significantly in recent years by AI) continues to accelerate, the evolution of CPUs has gradually plateaued, largely because, Moore’s Law notwithstanding, it gets increasingly challenging to pack more and more transistors into the same (or smaller) physical space, nevermind for lower costs. By designing chips specifically to perform in parallel the linear algebraic functions (known as tensor mathematics) which predominate in Machine Learning algorithms, it is believed that chip makers are on the threshold of a radical breakthrough in the efficiency of transistor utilisation. In other words, the latest AI chips could be about to smash what has for many years looked like a glass ceiling in the physical capabilities of computer chips.

The other cause for excitement about chips that process in parallel is that a distributed version of parallelization is throught to be at the heart of how human cognition works. For example, cognitive science describes memory as a function of a parallel form of interaction between neurons in our brain which simultaneously shares and creates resources. Parallelization is therefore of great interest in the field of artificial neural networks and deep learning.

Deep learning, and especially the ‘training’ side of it, is acknowledged to be especially demanding on processing resources, mainly because all the nodes (which often stretch into the billions) have to be trained simultaneously. The efficiency gains offered by parallelized AI chips could therefore make the scaling of neural networks possible to an extent that has simply not previously been practicable or viable. Freed from the limits of processing resources, some commentators believe neural networks would become as commonplace as the internet is today. Fitted with AI chips, every device would be able to contribute to a ‘hive mind’ approach to deep learning at an astronomical scale. With such a breakthrough in processing capabilities,  a futuristic concept like a globalised smart net weaving AI into every fibre of our daily lives certainly looks a practical step closer to becoming possible.