AI00 RWKV Server is an open source RWKV inference API server with Vulkan acceleration and OpenAI API compatibility.
Read more
Tag: LLM
llmfit – find local large language models
llmfit is a terminal application that helps users find local large language models that are suitable for the hardware they already own.
Read more
Tenere – TUI interface for LLMs
Tenere is a TUI interface for Large Language Models. Tenere supports the following backends: ChatGPT, llama.cpp, and ollama.
Read more
Cherry Studio – desktop client that supports multiple LLM providers
Cherry Studio is a desktop client that supports multiple LLM providers, available on Linux, macOS, and Windows.
Read more
Shimmy – lightweight OpenAI API server
Shimmy is a 4.8MB single-binary that provides 100% OpenAI-compatible endpoints for GGUF models.
Read more
Minisforum AI X1 Pro: Gerbil – Run Large Language Models Locally
Gerbil is a desktop app to easily run Large Language Models locally. The software is powered by KoboldCpp, a fork of llama.cpp.
Read more
Gerbil – desktop app to easily run Large Language Models locally
Gerbil lets you run LLMs locally powered by KoboldCpp which itself is a highly modified fork of llama.cpp.
Read more
Elelem – versatile LLM client
Elelem is a simple LLM client that connects seamlessly with OpenAI API compatible services.
Read more
Raspberry Pi 5 Desktop Mini PC: Ollama GUI
Did you know that even a Raspberry Pi 5 is able to run LLMs locally? Even better, the developer of Pi-Apps has produced an optimized install script which makes it Raspberry Pi 5 friendly.
Read more
Machine Learning in Linux: Ollama – self-hosted Llama 2
Ollama is software in an early stage of development that lets you run and chat with Llama 2 and other models.
Read more