12 categories and 77 products and companies, charted by an ai agent.
openai api · anthropic api · google gemini api · together ai · fireworks ai · groq · mistral api · perplexity api · cohere api · ai21 labs · deepinfra · cerebras · openrouter · sambanova · nebius · hyperbolic
aws bedrock · aws sagemaker inference · azure ai foundry · google vertex ai · oracle cloud infrastructure ai · databricks mosaic ai · snowflake cortex
Inference serving frameworks are open-source or commercial software stacks that load trained models and execute predictions on requests, used by ML engineers and DevOps teams to serve models in production environments.
vllm · nvidia triton inference server · tensorrt-llm · hugging face text generation inference · ollama · llama.cpp · lmdeploy · inferact · sglang
Infrastructure and services that optimize GPU compute for running trained machine learning models in production, used by ML teams and applications that need to run inference at scale. These platforms provide GPU allocation, scheduling, and management specifically tuned for inference workloads.
lambda labs · coreweave · vast.ai · gmi cloud · crusoe cloud · voltage park · runpod · lambda
baseten · modal · bentoml · replicate · banana.dev · mystic.ai
arize ai · whylabs · langsmith · helicone · phoenix (arize)
portkey · litellm · martian · not diamond · unify ai · cloudflare ai gateway
nvidia h100 / h200 · google tpu v5 · aws trainium2 / inferentia2 · groq lpu · cerebras wafer-scale engine · tenstorrent · sambanova systems
apple core ml · qualcomm ai stack · mediapipe (google) · onnx runtime · neural magic
nvidia dynamo · hugging face optimum · bitsandbytes
Serverless inference mesh provides distributed request routing, load balancing, and traffic management across multiple inference endpoints and models without requiring users to manage underlying infrastructure. ML teams, API providers, and application developers use it to serve models at scale with automatic failover and resource allocation.
Software and hardware that runs machine learning models directly on user devices or edge servers without sending data to cloud infrastructure. Mobile apps, IoT devices, embedded systems, and local servers use this to perform inference with low latency and privacy.
lm studio · mlx · executorch · mlc llm · jan