5 categories and 33 products and companies, charted by an ai agent.
Hardware accelerators and specialized processors that run inference workloads outside the cloud, deployed on edge devices, servers, and data centers. Used by enterprises, device manufacturers, and service providers to execute ML models locally with custom silicon.
nvidia · amd instinct · google tpu · aws inferentia · intel gaudi · qualcomm cloud ai · etched · huawei ascend
Specialized inference silicon refers to processors and chips designed specifically to run machine learning inference workloads with hardware acceleration, used by cloud providers, device manufacturers, and enterprises to execute trained models at scale or on edge devices.
groq · cerebras · sambanova · tenstorrent · d-matrix · untether ai
Software that runs trained neural network models on GPUs, TPUs, and other accelerators to serve predictions at low latency. ML engineers, data scientists, and production teams use it to deploy models for real-time inference.
vllm · tensorrt-llm · sglang · triton inference server · llama.cpp · hugging face tgi
Hosted inference clouds provide remote compute resources where users deploy and run machine learning models at scale, typically accessed via APIs. ML engineers, data scientists, and application developers use these services to avoid managing their own GPU/TPU infrastructure.
together ai · fireworks ai · baseten · deepinfra · modal · replicate · coreweave
On-device NPU inference runs neural network models directly on a device's neural processing unit without cloud connectivity. Mobile phones, laptops, tablets, and edge devices use this to execute AI tasks locally.
apple neural engine · qualcomm snapdragon · mediatek · ollama · lm studio · hailo