LlamaStash – terminal-based local LLM manager

LlamaStash is a terminal-based manager for running and supervising large language models locally. It combines a text user interface, command-line tools, a background daemon and an API proxy into a single application.

The software is designed to simplify the management of local inference engines without introducing unnecessary layers between applications and the underlying model runtime. It uses llama.cpp as its principal backend, with support for other engines including vLLM, SGLang and configurable OpenAI-compatible servers.

LlamaStash can discover models already stored on a computer, provide information about their requirements and launch them using settings appropriate to the available hardware. Running models can be accessed by compatible applications through a common API endpoint.

The interface offers controls for model selection, configuration and monitoring. Its command-line functionality also makes it useful for scripting and automated development workflows.

This is free and open source software.

Key Features

• Interactive terminal interface for managing local language models and inference processes.
• Automatically discovers models in Hugging Face, Ollama and LM Studio caches.
• Supports GGUF models through llama.cpp, with additional backends for other model formats.
• Hardware-aware model recommendations based on available memory and processing capabilities.
• Automatically estimates suitable context sizes according to model requirements and available memory.
• Supports concurrent model execution with individual process management and health monitoring.
• Background daemon keeps models running independently of the terminal interface.
• Provides OpenAI-compatible API endpoints for connecting external applications.
• Offers compatibility with Anthropic Messages API clients and selected Ollama interfaces.
• Supports configurable inference presets, GPU selection and model-specific launch parameters.
• Built-in Hugging Face model browser with search and download capabilities.
• Includes interactive chat, embedding, reranking and logging interfaces.
• Provides command-line commands and structured JSON output for scripting and automation.
• Supports access controls for exposing inference services to other computers on a local network.
• Cross-platform support – runs on Linux, macOS, and Windows.

Website: github.com/llamastash/llamastash
Support:
Developer: Deepu K Sasidharan
License: MIT License

LlamaStash is written in Rust. Learn Rust with our recommended free books and free tutorials.

Explore our carefully curated directory of recommended free and open source software, covering every major software category. The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form.