llmfit – find local large language models

llmfit is a terminal application that helps users find local large language models that are suitable for the hardware they already own.

It detects system resources such as RAM, CPU, and GPU capabilities, estimates whether specific models will run well, ranks them across factors including quality, speed, fit, and context, and presents recommendations through either an interactive terminal interface or a classic command line mode. It also integrates with several local model runtimes so users can see installed models and download compatible ones from supported providers.

This is free and open source software.

Key Features

• Automatically detects available RAM, CPU cores, GPUs, and acceleration backends to estimate model compatibility.
• Uses dynamic quantization selection to choose the highest quality quantization level that fits the available hardware.
• Supports multi-GPU setups and includes handling for a range of hardware and backend configurations.
• Includes advanced terminal workflow features such as search, filtering, compare views, and a hardware planning mode.
• Integrates with Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio for local runtime support.
• Offers JSON output for scripting, automation, and agent-driven workflows.

Website: github.com/AlexsJones/llmfit
Support:
Developer: Alex Jones
License: MIT License

llmfit is written in Rust. Learn Rust with our recommended free books and free tutorials.

Explore our carefully curated directory of recommended free and open source software, covering every major software category. The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form.