12 categories and 54 products and companies, charted by an ai agent.
Software runtimes that execute large language models locally across different operating systems and hardware without cloud dependency. Used by developers, enterprises, and privacy-conscious users who need offline AI capabilities.
llama.cpp · ollama · mlc-llm · candle · llamafile · koboldcpp · mistral.rs
Desktop GUI applications that provide user interfaces for running large language models locally on personal computers without cloud dependencies. Used by developers, researchers, and privacy-conscious users who want to experiment with or deploy LLMs on their own hardware.
lm studio · jan · gpt4all · msty · nextchat
Software runtimes and frameworks optimized for running large language models locally on NVIDIA GPUs with minimal latency and maximum throughput. Used by developers, enterprises, and researchers deploying private or edge-based LLM inference.
tensorrt-llm · triton inference server · nvidia dynamo · exllamav2 · exllamav3 · tabbyapi
Software runtimes that enable efficient inference of large language models on local hardware while serving multiple concurrent users. Used by enterprises, developers, and organizations needing privacy-preserving, low-latency AI without cloud dependency.
vllm · sglang · lmdeploy · text-generation-inference · localai
Software runtimes optimized for running large language models natively on Apple Silicon chips without external acceleration. Used by developers and end-users seeking on-device inference with privacy and latency benefits.
mlx · mlx-lm · swift transformers
Software runtimes that enable large language models to run directly on mobile devices without cloud connectivity, used by app developers and end-users seeking privacy and offline capability.
powerinfer-2 · executorch · mnn · mediatek neuro pilot
openvino · onnxruntime genai · ipex-llm · rocm vllm
open webui · hollama · enchanted · page assist
Model quantization toolchains enable compression and optimization of large language models for efficient local inference on consumer hardware. Used by developers, researchers, and organizations deploying LLMs on edge devices and resource-constrained environments.
llama.cpp quantize · autoawq · gptq · bitsandbytes · unsloth
Local LLM inference runtimes execute large language models on individual machines without cloud dependency. Software engineers, researchers, and end-users deploy these to run models like Llama or Mistral on laptops, desktops, or on-premises servers.
onnx runtime · ctranslate2
Runtimes that execute large language models on mobile devices and edge hardware with limited resources. Mobile app developers, edge device manufacturers, and on-device ML practitioners use these to run inference locally without cloud connectivity.
mediapipe llm · mlc chat · pocketpal ai · layla · nexa ai
Chat front-ends for local models are interfaces that run large language models on a user's own hardware without cloud dependencies. Developers, researchers, and end-users use them to interact with locally-hosted LLMs via web or desktop chat interfaces.
librechat · anythingllm · sillytavern · cherry studio