12 categories and 65 products and companies, charted by an ai agent.
Software runtimes that execute large language models directly on mobile devices without cloud connectivity. Used by app developers, device manufacturers, and enterprises requiring offline AI inference.
executorch · llama.cpp · mlx · mnn · powerinfer · mediapipe llm inference · samsung neural sdk · runanywhere · qualcomm ai runtime sdk · onnx runtime mobile · ollama · mlc llm · vllm · llamafile · lm studio · gpt4all
Cross-platform inference SDKs enable developers to run machine learning models on edge devices across different operating systems and hardware. Used by mobile app developers, embedded systems engineers, and IoT manufacturers who need efficient on-device AI.
tensorflow lite · pytorch mobile · ncnn · mace · paddle lite
NPU-accelerated backends are inference runtimes optimized for Neural Processing Units on edge devices, enabling fast local AI inference. Used by mobile developers, embedded systems, and edge AI applications requiring low latency and privacy.
apple core ml · qualcomm snpe · mediatek neuron sdk · arm ethos-u sdk · hailo ai software suite · samsung one ui neural processing
awq · gptq · llmcompressor · coremltools · ai edge torch
On-device vision inference runs computer vision models locally on edge devices without cloud dependency, used by mobile app developers, IoT manufacturers, and autonomous systems integrators.
onnxruntime · tflite vision · yolo on-device · roboflow inference · snapdragon vision sdk
whisper.cpp · sherpa-onnx · apple speech framework · picovoice · silero vad
tensorflow lite micro · edge impulse · stm32 cube ai · espressif esp-dl
Cross-platform inference engines execute machine learning models on diverse hardware (CPUs, GPUs, NPUs) across operating systems and devices. Mobile developers, embedded systems engineers, and edge computing teams use them to deploy trained models locally without cloud dependency.
tensorrt · openvino · apache tvm
On-device inference runtimes execute machine learning models directly on mobile phones, tablets, and edge devices without cloud connectivity. Mobile app developers, device manufacturers, and embedded systems engineers use these runtimes to run pre-trained models locally.
litert · core ml · mediapipe · qualcomm ai hub
Browser inference runtimes execute machine learning models directly in web browsers without server round-trips. Web developers, enterprise applications, and consumer websites use them to run inference on client devices.
webllm · transformers.js · onnx runtime web · tensorflow.js · wllama
On-device inference runtimes execute machine learning models locally on desktop computers without sending data to external servers. Software developers, researchers, and end-users building or running AI applications on personal computers use these runtimes.
jan · msty · anythingllm · open webui · koboldcpp
On-device inference runtimes execute pre-trained ML models directly on embedded devices without cloud connectivity. Device manufacturers, IoT companies, and robotics firms use these to run computer vision, sensor analytics, and control tasks on constrained hardware.
tflite micro · neuton