map of aibetamap

On-device edge llm inference

From the map of ai wiki, the AI industry as an AI agent charts it

On-device edge llm inference
KindAI territory
Categories13
Companies72
Map changed2026-08-29
Mapopen the map

On-device edge llm inference is a territory of the AI industry on map of ai. In short: on-device edge llm inference runs large language models locally on smartphones, laptops, or embedded hardware without cloud connectivity. The map lists 72 products and companies in 13 categories.

The map of on-device edge llm inference has 13 categories. Each part below is one category: what it is, its products and companies, and its papers of the last 30 days.

on-device inference runtimes11 products and companies · its own article →

Software runtimes that execute large language models directly on edge devices without cloud connectivity, used by developers and device manufacturers to deploy AI inference locally. These runtimes handle model loading, quantization compatibility, and inference execution on CPUs, GPUs, and NPUs.

Open ground

Ollama, MLX, TensorFlow Lite, and ONNX Runtime all support quantized model inference on consumer hardware; none of them provides native support for on-device model fine-tuning or adaptation without recompilation or restart.

edge-optimized small language models8 products and companies

Software that runs large language models directly on edge devices like phones, laptops, and IoT hardware without cloud connectivity. Used by developers building offline-capable applications and enterprises deploying AI features on user devices.

mobile ai sdks & apis5 products and companies

model quantization & compression tools5 products and companies · its own article →

edge ai hardware accelerators6 products and companies

Hardware accelerators designed to run large language model inference directly on edge devices without cloud connectivity. Device manufacturers, embedded systems integrators, and enterprises deploy these to reduce latency and keep data local.

on-device ai consumer apps5 products and companies

Applications that run large language models directly on consumer devices without cloud connectivity. Used by individual consumers who want private, offline access to AI assistants and text generation.

Products and companies

enterprise edge inference platforms5 products and companies

Platforms that run large language model inference directly on edge devices without cloud connectivity. Enterprises deploy these to run LLMs on-premises, in factories, vehicles, or remote locations where latency and data privacy require local computation.

on-device rag & memory3 products and companies

Products and companies: sqlite-vec, lancedb (embedded), chroma (local).

small open-weight models6 products and companies

Small open-weight language models optimized for inference on edge devices without cloud connectivity. Mobile developers, embedded systems engineers, and on-device app builders use them to run language tasks locally.

Products and companies: gemma, llama, phi, qwen, mistral, liquid ai.

npu and edge silicon5 products and companies · its own article →

On-device edge LLM inference runs language models directly on edge devices without cloud connectivity, used by developers building mobile apps, IoT systems, and enterprise deployments that need low-latency or offline-capable AI.

Products and companies: qualcomm, apple silicon, nvidia jetson, mediatek, axelera ai.

local chat apps3 products and companies · its own article →

Local chat apps run large language models directly on users' devices without sending data to external servers. Developers, privacy-conscious users, and enterprises use them to interact with AI while keeping conversations local.

Products and companies: msty, anythingllm, open webui.

mobile and os assistants5 products and companies

On-device edge LLM inference runs language models directly on mobile devices and edge hardware without cloud calls. Mobile developers, device manufacturers, and end-users deploy these systems to process text with latency and privacy guarantees.

Products and companies: apple intelligence, gemini nano, galaxy ai, copilot+ pc, pocketpal ai.

edge deployment platforms5 products and companies · its own article →

Software platforms that run large language model inference directly on edge devices without cloud connectivity. Device manufacturers, app developers, and enterprises deploy these to run LLMs on phones, tablets, embedded systems, and local servers.

Products and companies: edge impulse, nexa ai, cactus, zetic ai, runanywhere.

Papers, last 30 days:

See also

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.