By Giuseppe Desoli, PhD

Company Fellow · ST’s Chief Innovation Officer and General Manager and Group Vice President, System Research and Applications Group

“For seventy years, we have built machines that move data to where computing happens. Edge AI will only scale if we turn that around, with computing happening inside memory and intelligence built into the sensor. That shift is the roadmap.”

AI has been a cloud story. Why does it have to move to the edge?
Because the economics have changed. For years, the hardest part of AI was training models. Training is expensive, but you do it once in a data center and use the result many times.

What is coming now is different. We are seeing systems that need to reason all the time, and physical systems that need to sense and react all the time. In both cases, the main cost is no longer training. It is inference, meaning the moment when the AI actually makes a decision. And inference happens where the data is created.

Once you see that, the case for edge AI becomes very practical. A machine that must react in a millisecond cannot wait for data to travel to the cloud and back. A device inside someone’s home should not need to send video elsewhere. And if you have hundreds of millions of sensors, it is simply too expensive for each one to rely on remote computing.

So latency, privacy, and cost all point in the same direction. Intelligence needs to sit close to the data, close to the action, and close to the real world.

You describe the key trend as “from storage to computing”. What does that mean in practice?
In a neural network, moving data often costs much more energy than doing the math itself. So the real design question is not just how powerful the computing unit is. It is how far the data has to travel.

You can think of the shift in four steps.

First, there is the classic von Neumann model, where the processor and the memory are separate and data moves between them.

Second, there is near-memory computing, where memory and processing are brought onto the same chip. That reduces data movement and improves bandwidth.

Third, there is SRAM-based in-memory computing. In that case, the key math operation happens inside the memory array itself.

Fourth, there is non-volatile in-memory computing. Here, the same dense memory array both stores the model parameters and performs the computation. The weights do not move at all.

That is the real change. Memory stops being just a place to keep data. It becomes part of the computer itself. That idea is at the center of our architectural choices.

In-memory computing test vehicles matter because simulation alone is not enough. An architecture only becomes real when it has faced the issues of silicon, such as variability, drift, manufacturing yield, and compiler support.

Where does ST actually stand on in-memory computing today?
We are already doing this in silicon, and we have been for several years.

At ISSCC 2023, we presented a fully digital in-memory computing accelerator built in 18 nm FD-SOI. It reached 40 to 310 TOPS/W at precision levels up to 4 bits.

What matters most is not just the number. It is what sits behind it.

First, the design uses multiple tiles. That means the efficiency is not limited to a single lab block. It can scale into a larger, more practical accelerator.

Second, it is digital. That means it gives exact, repeatable results and can move through standard design, test, and product development flows. That is what makes it relevant for real products.

This work did not appear overnight. It builds on years of progress. It started with earlier silicon we presented in 2017, then moved through the platform and software work needed to make the architecture programmable. That path led to the Neural-ART family of NPU accelerators, including the first generation in the STM32N6, the first STM32 with built-in hardware AI acceleration.

So for us, this is not separate research on one side and product planning on the other. It has been one continuous program for nearly ten years.

Physical AI, embodied AI, robotics — a different problem, or the same one?
It is closely related, but the challenge becomes more demanding because timing matters so much.

A physical AI system has to keep sensing, understanding, deciding, and acting in real time. To do that well in a changing, unpredictable environment, it needs fast local inference. It cannot rely too heavily on sending data away and waiting for a response.

That changes what edge computing has to deliver. These systems need to understand context on the device, combine data from different sensors, protect privacy, and support instant, reflex-like responses when needed.

This is one reason robotics is changing. Older systems often relied on carefully written rules for a specific task. That works well in a controlled setting. It works much less well in the real world, where situations change constantly.

What is replacing it is a more unified approach. One model can connect perception, reasoning, and control more directly. That makes it easier for machines to handle new objects and unfamiliar tasks.

The models themselves are also evolving. They are moving toward ways of generating smoother, more continuous motion. That is good news for real machines, because motors and actuators need smooth, frequent updates to work well.

And the gap between what these models need and what edge devices can provide is getting smaller. That means this is starting to look like a realistic design target, not a distant idea.

This is exactly where memory-focused design matters. A system like this has to keep track of a changing world while making decisions in real time. So context awareness is not something you add later. It has to be built in from the start through the way memory and computing work together.

How does an exploratory architecture become something a customer can buy?
Mostly through the tools.

New hardware is not a product if developers cannot actually use it. That is why every generation of accelerator we build also comes with compiler and quantization work that fits into a familiar development flow.

That includes the ST Edge AI tools and the STM32 AI ecosystem, where a trained network comes from a standard framework and is then turned into code for the target device.

A lot of our published work sits exactly in that less visible but essential layer: mapping networks onto hardware tiles and exploring compiler optimizations layer by layer.

Part of this work also happens in open collaboration. We have long-standing partnerships with universities and with IBM Research Zurich, and now a joint corporate lab with the National University of Singapore.

Memory-centric computing is too broad and important for any one company to explore alone. Many of the questions that will matter most ten years from now are still being debated today in research conferences.