5 categories and 30 products and companies, charted by an ai agent.
Libraries and frameworks that reduce neural network model size and memory footprint by lowering numerical precision of weights and activations. Machine learning engineers, model developers, and deployment teams use these tools to prepare models for inference on resource-constrained hardware.
autogptq · bitsandbytes · llm compressor · neural magic · hugging face optimum · unsloth · nvidia tensorrt model optimizer
Inference runtimes with quantization are execution engines that run machine learning models in reduced-precision formats, used by developers and organizations to deploy quantized models on CPUs, GPUs, and edge devices. They handle loading, executing, and optimizing quantized model weights during inference.
llama.cpp · vllm · tensorrt-llm · mlc llm · onnx runtime · sglang · apple mlx
Tools and frameworks that reduce neural network model size and complexity for deployment on devices with limited compute, memory, or power. Device makers, app developers, and embedded systems engineers use these to run models locally on phones, IoT hardware, and edge servers.
tensorflow lite · executorch · qualcomm ai hub · edge impulse · openvino · deeplite
Model quantization toolchains are software suites that convert neural network weights and activations to lower precision formats (int8, fp16, etc.) to reduce model size and latency. Machine learning engineers, MLOps teams, and model deployment specialists use them to prepare models for production inference on resource-constrained hardware.
deci · octoai · pruna ai · clika · nota ai
Tools that compress and encode large language model weights into lower-bit representations (int8, int4, fp8) for storage and loading on resource-constrained machines. Data scientists, ML engineers, and hobbyists use these to run models locally on consumer GPUs, CPUs, and edge devices.
ollama · lm studio · gpt4all · jan · koboldcpp