Shimmy is a 4.8MB single-binary that provides 100% OpenAI-compatible endpoints for GGUF models.
Read more
Tag: LLM
Minisforum AI X1 Pro: Gerbil – Run Large Language Models Locally
Gerbil is a desktop app to easily run Large Language Models locally. The software is powered by KoboldCpp, a fork of llama.cpp.
Read more
Gerbil – desktop app to easily run Large Language Models locally
Gerbil lets you run LLMs locally powered by KoboldCpp which itself is a highly modified fork of llama.cpp.
Read more
Elelem – versatile LLM client
Elelem is a simple LLM client that connects seamlessly with OpenAI API compatible services.
Read more
Raspberry Pi 5 Desktop Mini PC: Ollama GUI
Did you know that even a Raspberry Pi 5 is able to run LLMs locally? Even better, the developer of Pi-Apps has produced an optimized install script which makes it Raspberry Pi 5 friendly.
Read more
Machine Learning in Linux: Ollama – self-hosted Llama 2
Ollama is software in an early stage of development that lets you run and chat with Llama 2 and other models.
Read more