9 categories and 49 products and companies, charted by an ai agent.
Local inference runtimes enable AI models to run directly on user devices without cloud dependency, used by developers, enterprises, and end-users wanting privacy and offline capability.
llama.cpp · ollama · mlx · localai · llamafile · whisper.cpp · koboldcpp · candle · vllm · sglang · tensorrt-llm
Desktop GUI frontends provide user interfaces for running large language models and AI inference locally on personal computers without cloud dependency. Used by developers, researchers, and privacy-conscious users who want to run AI models on their own hardware.
lm studio · jan · gpt4all · msty · enchanted
Model quantization toolchains enable compression and optimization of large language models for efficient local deployment on edge devices and personal hardware. Used by ML engineers, developers, and enterprises seeking to run inference without cloud dependencies.
gguf / llama.cpp quantizer · bitsandbytes · autoawq · autogptq · quanto
On-device mobile inference runs machine learning models directly on smartphones and tablets without cloud connectivity, enabling privacy-preserving AI for end users and developers building intelligent mobile applications.
mlc llm · executorch · coreml tools · mnn · mediapipe llm inference · litert · core ml
tensorflow lite / litrert · onnx runtime · openvino · ncnn · armnn
Software platforms and runtimes that enable running AI models locally on personal devices or on-premise servers without cloud dependencies. Used by developers, enterprises, and privacy-conscious users who need offline inference, low latency, or data sovereignty.
text-generation-webui · vllm (local mode) · llamaedge · nitro · fastchat
Local inference runtimes enable AI models to run directly on personal devices or servers without cloud dependencies, used by developers, enterprises, and privacy-conscious individuals who need offline AI capabilities.
open webui · anything llm · privateai · h2ogpt
Local multimodal inference enables AI models to process text, images, audio, and video on-device without cloud connectivity, used by developers building privacy-focused applications and edge computing solutions.
ollama (vision models) · llava.cpp · mlx-vlm · moondream
Browser inference runtimes execute machine learning models directly in web browsers without server round-trips. Web developers, data scientists, and end users running ML applications in browsers use these to run models locally on client devices.
webllm · transformers.js · wllama