10 categories and 58 products and companies, charted by an ai agent.
Software runtimes that enable large language models to run directly on user devices across different platforms without cloud dependency. Used by developers, enterprises, and end-users seeking privacy, offline capability, and reduced latency.
llama.cpp · mlc-llm · ollama · onnx runtime genai · openvino · lm studio · jan · koboldcpp · onnx runtime · tensorrt-llm · litert · vllm
mlx · mlx-lm · core ml · executorch · mirai
qualcomm ai engine direct sdk · mediatek neuron sdk · google ai edge (litert) · samsung one ui ai stack · executorch mobile · rockchip rkllm
Software engines that quantize and run large language models efficiently on consumer GPUs (RTX, RTX Ada, etc.) for local inference without cloud dependency. Used by developers, researchers, and end-users wanting private, fast on-device AI.
exllamav2 · exllamav3 · llama-cpp cuda backend · ctransformers
webllm · transformers.js · wllama · mediapipe llm inference · chrome built-in ai
llama.cpp quantize · autoawq · gptq · bitsandbytes · quanto
Software frameworks and applications that run large language models locally on desktop computers without cloud connectivity. Used by developers, researchers, and privacy-conscious users who need offline AI capabilities.
lm studio · gpt4all · msty · pinokio · novelai (local mode) · anythingllm · open webui
apple intelligence · google gemini nano · microsoft phi silica · qualcomm cloud ai sdk
On-device LLM runtimes are software systems that execute large language models locally on mobile devices without cloud connectivity. Mobile developers and end users running applications on smartphones and tablets use these runtimes to perform inference tasks.
pocketpal ai · private llm · google ai edge gallery · layla · mlc chat
On-device LLM runtimes execute large language models locally on consumer hardware without cloud connectivity. Developers, end users, and enterprises use these to run inference on laptops, phones, and embedded devices.
cactus · nexa ai · zetic.ai · qualcomm ai hub · apple foundation models