AMD recently launched its second generation of PC processors with integrated neural processing units, or NPUs. And Intel’s new Core Ultra processors based on Meteor Lake architecture also have built-in NPUs for hardware-accelerated AI. So we’re seeing a growing number of PC makers position their latest systems as “AI PCs.”
But ARM-based chip makers have been been using NPUs for years, and Qualcomm says its upcoming Snapdragon X Elite chips for Windows PCs will run circles around the latest Intel and AMD processors when it comes to AI features, with the NPU alone able to deliver up to 45 TOPS of AI performance. But Qualcomm isn’t just focused on bringing AI improvements to PCs. The company is also continuing to focus on smartphones and tablets.
To put that 45 TOPS number in perspective, Intel says its Core Ultra 7 165H processor delivers up to 34 TOPS of AI performance, while AMD’s Ryzen 9 8945HS can hit 39 TOPS. But you only get that level of on-device AI performance by leveraging the Intel AI Boost NPU and the chip’s CPU and GPU cores.
Qualcomm says by delegating AI-related tasks to the NPU, its Snapdragon X Elite processor can not only offer faster performance, but also significantly more efficient performance, leading to longer battery life and leaving a computer’s other resources free for other tasks.
For example, Qualcomm says a laptop with a Snapdragon X Elite processor running GIMP with the Stable Diffusion plugin was able ot generate an image in 7.25 seconds that took 22.26 seconds to complete on a similar PC with an Intel Core Ultra 7 155H processor.
At this point, many of the phones and PCs that device makers are positioning as having enhanced AI features are actually still relying on the cloud for some features. For example Google’s Magic Eraser feature that lets you remove unwanted objects from photos? It won’t work without an internet connection.
Qualcomm says by delivering higher-performance NPUs on mobile devices like smartphones and laptops, more of that work can be done on-device. And that could have benefits for user privacy and security. (your data isn’t sent to a remote server) as well as cost (you might not need a subscription to a cloud-based service to perform the same things).
Whether we will actually see an explosion of AI apps that are designed to run totally on your device remains to be seen though. Sure, the NPUs in mobile chips are only going to get better over time, but a lot of companies seem to be building their businesses on the idea of getting users to sign up for subscriptions to cloud-based services. There’s probably not as much money in software that you buy once and user forever (unless there are a lot of in-app purchases or upgrades, I guess).
Qualcomm is hoping to help encourage developers though – the company is launching a new Qualcomm AI Hub as a central location for developers to find AI models that are optimized for deployment across Snapdragon and Qualcomm platforms. There are more than 75 optimized AI models available, and Qualcomm says these optimized models deliver 4X faster inferencing speeds than non-optimized models.
Meanwhile, not everything Qualcomm is also playing up AI features in some of the other products the company is announcing at MWC 2024 this week.
Case in point? The new Snapdragon X80 5G modem-RF system supports 5G-Advanced networks and NB-NTN satellite communications. But it also has “a dedicated tensor accelerator” that the chip maker says optimizes throughput, coverage, latency, power efficiency, and mmWave beam management, among other things.
And the new Qualcomm FastConnect 7900 mobile wireless system apparently also uses AI to “adapt to specific use cases and environments” to optimize performance, speed, and efficiency when using WiFi 7, Bluetooth, and Ultra Wideband wireless features.
Finally, Qualcomm has also hinted at an upcoming initiative to bring 5G connectivity to budget smartphones that sell for $99 and less. There aren’t many details about what this involves yet, but the company did release a graphic describing it as a 2-antenna 5G standalone solution that could bring up to 5X faster peak speeds when compared with 4G LTE.
Until Qualcomm makes a killer app for their NPU, it is a difficult sell to phone makers and developers. Face-unlock is probably the best NPU feature I have seen. Second place is probably Apple’s Portraits (auto-generated photo arrangements). I would happily ditch both of those features for $100-off my next phone.
First must-haves will be DL next gen video and audio codecs. LM and image processing models evolve too fast and pretty big, while codecs are well standardized. Audio is already there for a few years and has some adoption, and video is coming soon.
Companies might still use it but still sell their software as a service requiring a subscription. There are plenty of apps which do that (and isn’t this totally adobe’s current bushes model?). Yes, is theoretically harder to justify but they still do it and it behooves them too, they don’t have to spend as much on cloud computing resources then. And then they can sell it to you as a feature, increased security, save on bandwidth/data costs and possibly faster results