Raspberry Pi 5 Desktop Mini PC: Ollama GUI

Machine Learning is the practice of using algorithms to parse data, learn insights from that data, and then make a determination or prediction. The machine is ‘trained’ using huge amounts of data.

Deep Learning is a subset of Machine Learning that uses multi-layers artificial neural networks to deliver state-of-the-art accuracy in tasks such as object detection, speech recognition, language translation and others. Think of Machine Learning as cutting-edge, and Deep Learning as the cutting-edge of the cutting-edge.

Both Machine Learning and Deep Learning are changing the world. Deep Learning is trending.

Large Language Models (LLMs) represent a breakthrough in artificial intelligence, employing neural network techniques with extensive parameters for advanced language processing.

Did you know that even a Raspberry Pi 5 is able to run LLMs locally? Even better, the developer of Pi-Apps has produced an optimized install script which makes it Raspberry Pi 5 friendly.

Besides installing the Ollama command-line tool, the developer says it runs it with a low priority, pre-downloads 2 LLMs (Codellama:7b and Phi3:mini). Further the script also installs a simple Python-based GUI.

Installation

If you haven’t already done so, you first need to install Pi-Apps. Install Pi-Apps from the terminal. Issue the command:

$ wget -qO- https://raw.githubusercontent.com/Botspot/pi-apps/master/install | bash

Now we can install the software. In Pi-Apps go to Tools, select Ollama GUI and click the Install button.

The two LLMs aren’t that big downloads.

In Operation

Launch the program from the Education menu entry.

We’ve chosen one of the two LLMs and asked a question.

Observations

• The software lets you run LLMs similar to ChatGPT locally on your Pi without internet access. From a privacy perspective that might be important to you. And if your internet access is intermittent, the software still runs.
• The two LLMs that the script downloads work with the 8GB or 16GB Raspberry Pi 5 models. The models will not work with the 2GB or 4GB models, but you can still download models that will work. On balance it’s definitely better to use the 16GB model as you run larger models (e.g. deepseek-r1:14b which has a memory footprint of 10GB) and still have sufficient RAM to have other programs open at the same time.
• The accuracy of the included models leaves a lot to be deserved, but there’s no way of increasing RAM on the Pi above 16GB.
• The writing speed is lethargic on the Pi 5 to say the least but that’s not surprising considering the hardware specification of the Pi.
• The GUI is very basic but functional.
• It’s very easy to add new LLMs. Just paste the name of the model in the download box in Ollama GUI’s Settings. I’ve tried a fair few of the other models.

Summary

While there are obvious limitations using a Raspberry Pi 5 to run LLMs, the fact it runs as well as it does is impressive in itself. I didn’t notice the script running the program with a lower priority.

This article was written on a Raspberry Pi 5 (16GB RAM) provided by SunFounder, a company focused on STEAM education with products like Open-source robots, Arduino & Raspberry Pi Kits, display screens, and other smart devices.

All articles in this series: