Dalai bills itself as “the simplest way to run LLaMA on your local machine”.
Given that our Machine Learning in Linux series focuses on apps that make it easy to experiment with machine learning, Dalai looks an interesting project to spotlight.
What’s LLaMA? It’s an acronym for Large Language Model Meta AI, a collection of open and efficient foundation language models designed to help researchers advance their work in this subfield of AI.
Large Languages Models trained on massive amount of text can perform new tasks from textual instructions. They can generate creative text, solve maths problems, answer reading comprehension questions, and much more. Like other large language models, LLaMA works by taking a sequence of words as an input and predicts a next word to recursively generate text.
Dalai needs Python <= 3.10 and Node.js >=18. On our Ubuntu 22.04 LTS test system, we need to replace our version of Node.js as it’s too old. The simplest way to upgrade on Ubuntu is via a PPA (personal package archive). Run the following command:
$ curl -fsSL https://deb.nodesource.com/setup_19.x | sudo -E bash - &&\
sudo apt-get install -y nodejs
We can verify the new version of node.js with the command:
$ node -v
$ sudo apt install npm
We’re now ready to install Dalai and its 7B model (we recommend you start with this model as it’s the smallest).
$ npx dalai llama install 7B
Part of the installation process quantizes the model. On a fairly modern processor the quantization process takes under a minute.
The web UI server is launched with the command:
$ npx dalai serve