13 categories and 72 products and companies, charted by an ai agent.
Software runtimes that execute large language models directly on edge devices without cloud connectivity, used by developers and device manufacturers to deploy AI inference locally. These runtimes handle model loading, quantization compatibility, and inference execution on CPUs, GPUs, and NPUs.
llama.cpp · executorch · ollama · mlx · lm studio · mnn · ggml · whisper.cpp · mlc llm · vllm · llamafile
Software that runs large language models directly on edge devices like phones, laptops, and IoT hardware without cloud connectivity. Used by developers building offline-capable applications and enterprises deploying AI features on user devices.
phi-3 / phi-4 · gemma 3 · llama 3.2 1b/3b · qwen2.5 / qwen3 · mistral 7b · smollm · openelm · glm-4-9b
google ai edge sdk · apple core ml / apple intelligence · qualcomm ai hub · mediatek neuron sdk · samsung one ui ai
llm.int8 / bitsandbytes · awq · gptq · neural compressor (intel) · quanto
Hardware accelerators designed to run large language model inference directly on edge devices without cloud connectivity. Device manufacturers, embedded systems integrators, and enterprises deploy these to reduce latency and keep data local.
qualcomm snapdragon x elite npu · apple neural engine · mediatek dimensity 9400 npu · samsung exynos npu · google tensor g4 tpu · intel npu (lunar lake)
Applications that run large language models directly on consumer devices without cloud connectivity. Used by individual consumers who want private, offline access to AI assistants and text generation.
microsoft copilot+ (local) · jan.ai · private llm · gpt4all · mochi (iphone)
Platforms that run large language model inference directly on edge devices without cloud connectivity. Enterprises deploy these to run LLMs on-premises, in factories, vehicles, or remote locations where latency and data privacy require local computation.
onnx runtime · tensorflow lite / litert · nvidia jetson / tensorrt · hailo · deeplite
sqlite-vec · lancedb (embedded) · chroma (local)
Small open-weight language models optimized for inference on edge devices without cloud connectivity. Mobile developers, embedded systems engineers, and on-device app builders use them to run language tasks locally.
gemma · llama · phi · qwen · mistral · liquid ai
On-device edge LLM inference runs language models directly on edge devices without cloud connectivity, used by developers building mobile apps, IoT systems, and enterprise deployments that need low-latency or offline-capable AI.
qualcomm · apple silicon · nvidia jetson · mediatek · axelera ai
Local chat apps run large language models directly on users' devices without sending data to external servers. Developers, privacy-conscious users, and enterprises use them to interact with AI while keeping conversations local.
msty · anythingllm · open webui
On-device edge LLM inference runs language models directly on mobile devices and edge hardware without cloud calls. Mobile developers, device manufacturers, and end-users deploy these systems to process text with latency and privacy guarantees.
apple intelligence · gemini nano · galaxy ai · copilot+ pc · pocketpal ai
Software platforms that run large language model inference directly on edge devices without cloud connectivity. Device manufacturers, app developers, and enterprises deploy these to run LLMs on phones, tablets, embedded systems, and local servers.
edge impulse · nexa ai · cactus · zetic ai · runanywhere