7 categories and 37 products and companies, charted by an ai agent.
On-device LLM runtimes execute large language models directly on edge devices without cloud connectivity. Mobile app developers, embedded systems engineers, and device manufacturers use these to run inference locally on phones, tablets, IoT devices, and specialized hardware.
llama.cpp · ollama · mlc llm · executorch · mediapipe llm inference · mlx · lm studio · llamafile
Cross-platform model runtimes enable inference execution of machine learning models on edge devices across multiple hardware architectures and operating systems. Used by application developers, device manufacturers, and ML engineers to deploy trained models to phones, IoT devices, embedded systems, and local servers without cloud dependency.
onnx runtime · tensorflow lite · apache tvm · ncnn · openvino · coreml
Browser inference runtimes execute machine learning models directly in web browsers without server roundtrips. Web developers and applications use these to run inference on client devices.
webllm · transformers.js · onnx runtime web · wllama
Hardware vendor SDKs that enable machine learning model inference on edge devices (mobile phones, IoT devices, embedded systems). Device manufacturers, app developers, and embedded systems integrators use these to run trained models locally without cloud connectivity.
tensorrt · qualcomm ai hub · hailo · amd ryzen ai · arm kleidiai · coral
Edge inference runtimes execute trained ML models on edge devices without cloud connectivity. ML engineers, embedded systems developers, and IoT teams use them to deploy inference workloads locally on phones, industrial equipment, and IoT hardware.
edge impulse · nota ai · deeplite · latent ai · octoai
Runtimes that execute machine learning models on microcontrollers and embedded devices with limited memory and compute. Firmware engineers, embedded systems developers, and IoT manufacturers use these to run inference locally on hardware like ARM Cortex-M processors.
tflite micro · cmsis-nn · stm32cube.ai · emlearn
Edge inference runtimes execute machine learning models locally on consumer devices without cloud connectivity. Developers and end-users deploy these runtimes to run inference on phones, laptops, and tablets.
jan · gpt4all · msty · apple intelligence