This is a blog looking at using the AMD Ryzen AI 9 HX 370’s NPU in Linux. The AMD Ryzen AI 9 HX 370 is a mobile processor with 12 cores and 24 threads. This processor features the Radeon 890M integrated graphics solution, and a new generation XDNA 2 NPU. Int8 TOPS is rated at up to 80 TOPS combining CPU cores, GPU cores, and NPU.
For this article, I’m going to explore Gaia. What’s Gaia? It’s software designed for the quick setup and execution of generative AI applications on local PC hardware. It enables fast and efficient execution of LLM-based applications using a hybrid hardware approach that combines the AMD NPU and Integrated Graphics Processing Unit (iGPU).
The feature set of Gaia looks very interesting including the ability to run direct LLM queries, access interactive conversations with history, 3D content creation, model evaluation, and speech-to-text conversation. And this is AMD software that’s designed to run under Linux with both GUI and CLI support.
Here’s my opinionated thoughts.
AMD’s installation instructions for Gaia under Linux are sufficient but warrant further explanation. First, AMD can only be described as being a bit lazy when describing the installation procedure. They outline that you need to 1) clone the GitHub repository, 2) install GAIA CLI via pip, and then 3) install the Lemonade server.
There’s no issue with step 1. But step 2, oh dear! Never install Python software direct to your system via pip. Many Linux distributions won’t let you do this anyway instructing you that the environment is externally managed and Python packages should be installed with a virtual environment. That’s for the simple reason you’ll pollute your system if you don’t create a virtual environment. By pollute, I mean that you will break your system sooner or later.
AMD are not alone in failing to describe this adequately, probably because there are quite a few ways to create a virtual environment. I used conda to create the virtual environment, activate the environment, and then install via pip.
Next, you need to install the Lemonade server. But AMD’s readme fails to point out the elephant in the room. Here’s a table from Lemonade’s website.
That’s right, the table indicates there isn’t NPU support in Lemonade Server for Linux. There is support for the NPU in Windows though. Handy if you’re still living on the dark side of the moon.
The table on Lemonade’s website is not up-to-date. There is not just CPU support in Linux, as there’s also support in Linux via llama.cpp/Vulkan backend. This means the Radeon 890M (the iGPU in the Minisforum machine) is used for relevant models. And I can confirm this is working very nicely with fast responses.
It’s easy to install Gaia (both CLI and GUI) under Linux. There’s a deb package for Ubuntu which installs with no issues.
The GGUF model is being fully offloaded to the GPU.
You’ll see I had allocated 8GB of RAM to the iGPU (up from the default 2GB of RAM). Obviously that’s insufficient for some models e.g gpt-oss-20b-mxfp4-GGUF which needs about 13GB of GPU RAM. Fortunately with 64GB of system RAM, there’s plenty of scope to increase the amount allocated to the GPU. To use the gpt-oss model I increased the amount of memory allocated to the GPU to 16GB (this requires rebooting and changing the relevant setting in the BIOS).
It’s easy to add models and other types such as vision models. Vision language models (VLMs) are multimodal, generative AI models capable of understanding and processing video, image, and text. Here’s an example.
I have briefly tested the software under Windows (well the Minisforum machine came with a Windows 11 license). And the software under Windows is using the NPU. 74% utilisation is very good for the Minisforum machine.
From only brief testing, it’s pretty clear that Gaia is fantastic open source software. The interface is easy to use, adding models is straighforward, and the Minisforum machine generates responses very quickly, both when using the CPU and iGPU. Let’s hope Linux NPU support is just around the corner!
Website: github.com/amd/gaia
Support:
Developer: AMD
License: MIT License
Gaia is written in Python. Learn Python with our recommended free books and free tutorials.
Complete list of articles in this series:
| Minisforum AI X1 Pro | |
|---|---|
| Introduction | Introduction to the series and interrogation of the machine |
| Benchmarks | Benchmarking the Minisforum AI X1 Pro |
| Power | Testing and comparing the power consumption |
| Jan | ChatGPT without privacy concerns |
| ComfyUI | Generate video, images, 3D, audio with AI |
| AMD Ryzen AI 9 HX 370 Cores | Primary (Zen 5) and Secondary Cores (Zen 5c) |
| Gerbil | Run large language models locally |
| Neural Processing Unit (NPU) | Introduction |
| Gaia | Run LLM Agents |