Like most technology enthusiasts, the launch of Samsung’s latest Galaxy AI phones yesterday must have caught your attention. In case you missed the news, here is the gist: Samsung has joined forces with Google to bring AI to the 2024 Galaxy range of smartphones. Google’s Gemini Pro and Imagen 2 will be deployed on the Galaxy S24 series and will aid in capabilities like generating new text, voice, and image features. Samsung will be the first GCP partner to deploy Gemini Pro and Imagen 2 on Vertex AI via the cloud to devices. This should not be surprising since this partnership goes way back and has been a strategic one.

Just a month ago, I wrote an article pertaining to a research paper published by Apple engineers on running LLM models on the edge. Looks like this was already in the works for this specific product. While most capabilities need a cloud connection (read internet connection), limited capabilities can run on the phone, with the NPU being used for on-device AI. The world of technology moves fast ! While my postulation in the article linked above was that within 2-3 years, we will see decent-size models being run on phones, it looks like the journey has already started.

As highlighted in my article, “LLMs on the Edge,” there is no doubt that the capability to run LLMs on edge can revolutionize the gamut of solutions and capabilities that can be built. However, the focus should not be only on the hardware aspect. Another focus area should be designing and developing compact (sub-10 billion parameters) models that deliver decent capabilities. This blog from Qualcomm effectively captures this aspect and shares some interesting data. The figure below highlights the types of sub-10 billion LLMs that can perform effectively on edge devices like your phone.

Figure: LLMs that can run off-cloud

Source: Qualcomm Blog

This is where creative algorithm development can make a strategic impact. If you can experiment with designing sub-10 billion parameter algorithms even within areas like text-to-image and voice understanding, you can significantly alter the race to bring powerful LLMs to mobile phones and create true iPhones.

And yes, I am swapping my iPhone for the Galaxy AI S24. The delta between the trade-in value and S24 price will be covered by offloading another victim of tech advance- my Sony ZV-1 camera, that has been made defunct by my DJI Osmo Pocket 3. The pace of technology !