Artificial intelligence is poised to become even more personalized as it directly integrates into consumer devices such as laptops and mobiles. On-device AI will bring about a revolutionary era of personalization, tailoring content to us in advanced and intimate ways, without the need to connect to a server or the cloud. This isn’t just about detecting faces in photos or unlocking phones – it’s a game-changer in personalization.
It will enable a fantastic user experience, whether generating fresh wallpapers on demand, taking meeting minutes, organising events, or automating video editing and photo enhancements. For instance, consider Google’s Pixel 8 Best Take photo feature for adjusting someone’s expression in a picture. Imagine a volleyball player about to serve from beyond the net. As the player throws the ball into the air and prepares to volley it across, his facial muscles contract, and his mouth tightens. The Best Take Photo feature will capture that facial expression with clarity.
In addition, AI will bring several benefits to the device, including low latency, improved security, and flexibility.
Let’s explore the additional potential of on-device AI applications for consumers and businesses.
On-device AI is for all
Although on-device AI is still in the early stages of development, we have seen enhancements in AI applications for both consumers and businesses. These include real-time language translation, automation inferencing, and immersive visual, audio, and gaming environments. Leading phone manufacturers are already adding AI features, such as circling an image to search. Another helpful feature is identifying fraudulent phone calls or warning users when they are about to make a mobile payment to an unverified service provider or business.
Designed with user-friendliness, on-device AI ensures users enjoy an enhanced digital experience at every level. Video meetings, conference calls, presentations, and interactive communication are all improved, bringing us one step closer to a fully immersive digital environment. The integrated AI will adapt to the user’s preferences, while AI noise reduction technology will remove background noise from audio files, images, and videos. This will provide users with distraction-free audio, phone calls, Facetime, Microsoft Teams meetings, etc.
The AI video rendering feature will significantly impact creative industries. It will empower filmmakers, video editors, professional gamers, graphic designers, and artists to improve productivity. The combination of vibrant graphics and exceptional audio will create high demand in the gaming world. By harnessing the power of on-device AI, users can optimise workflows and eliminate the labour and time spent on manual, repetitive tasks such as data entry, document processing, scheduling, invoicing, and similar business operations.
On-device AI offers several useful applications, but it’s important to note to have this capability on your device, you need a combination of specialised AI chips, processing units, and language models.
Specialised AI chips
Microprocessor manufacturers, including Intel, Qualcomm, AMD, and Nvidia, have significantly shifted their focus towards producing more dedicated SoCs and NPUs. This shift responds to the increasing demand for AI capabilities in consumer and enterprise devices.
AI chips are designed to be more energy-efficient than conventional chips. Some AI chips incorporate techniques like low-precision arithmetic, enabling them to perform computations with fewer transistors and, thus, less energy. Using AI chips could also help AI devices run more efficiently. For example, if you want your cellphone to collect and process your data without sending it to a cloud server, the AI chips powering that cellphone must be optimised for energy efficiency so they don’t drain the battery.
Another type of AI chip, known as an NPU, is specifically designed to handle AI tasks. The need for NPUs arises from the increasing complexity and volume of AI tasks that need to be executed on consumer and enterprise devices.
NPUs – a faster and better processor
NPUs are specifically designed for AI tasks. These tasks involve neural networks, which are like artificial brains capable of learning from data and performing remarkable tasks.
An NPU, with its superior speed and efficiency in running neural networks, can significantly reduce the workload on the CPU and GPU. By handling tasks such as background blurring in video calls or object detection in video or photo editing, NPUs free up the CPU and GPU to focus on other operations, thereby optimizing the overall performance of your device.
The CPU, GPU, and NPU are all crucial components of a computer’s overall operation. They are designed to handle different rendering and computing tasks, ensuring that no processor becomes overwhelmed with its workload. This prevention of overload is crucial as it directly impacts a computer’s performance.
Advantage of SLM
Small Language Models (SLMs) are AI computational models capable of responding to natural human language prompts. These models are trained as probabilistic machine learning models, which means they predict a probability distribution of words suitable for generating a sequence of phrases, aiming to emulate human intelligence.
SLM (for example, Microsoft Phi) is similar to a large language model (LLM) but has fewer training data and parameters. It essentially performs the same function as an LLM, which is to understand and generate language, but it is smaller and less complex. That makes it suitable for consumer devices.
The main advantage of SLMs is that they are designed for a specific use case with a focused scope. This means they require less data to train and fewer computing resources, making them more efficient than LLMs.
SLMs have the advantage of enhanced security due to their smaller compute footprints. This allows them to run locally on workstations or on-prem servers, increasing flexibility and scalability. Furthermore, it reduces the risk of data exposure during training and shortens the development lifecycle.
On-device AI seems promising, but time will tell if it ushers in the next big wave in computing or if it is just another new technology.
