13 categories and 68 products and companies, charted by an ai agent.
Parallelism frameworks enable distributed training of large machine learning models across multiple GPUs, TPUs, or nodes. Data scientists and ML engineers use them to scale model training and reduce training time.
pytorch · deepspeed · megatron-lm · jax · fairscale · colossal-ai · nanotron · torchtitan · tensorflow · lightning · hugging face accelerate
Pipeline parallelism runtimes enable distributed training of large models by splitting computation stages across multiple devices, reducing memory per device and increasing throughput. Used by ML researchers and engineers training very large language models and vision models.
gpipe · pipedream · alpa · veScale
deepspeed-moe · tutel · megablocks · switch transformer
nccl · gloo · mpi · rccl · aws ofi-nccl
Distributed training orchestration manages the coordination and execution of machine learning model training across multiple GPUs, TPUs, or nodes. Used by ML engineers, data scientists, and organizations training large-scale deep learning models.
ray train · horovod · pytorch lightning · accelerate · bagua
Managed training cloud services provide infrastructure and tooling to simplify distributed machine learning model training across multiple machines and GPUs. Data scientists and ML teams use these platforms to reduce complexity in scaling training workloads.
amazon sagemaker · google vertex ai · azure ml · lambda cloud · coreweave · together ai · modal · foundry · azure machine learning · databricks mosaic ai
xla · torch.compile · openxla · easydist
flower · openfedllm · petals · prime intellect
trl · openrlhf · verl · nemo-aligner
Cluster orchestration and compute scheduling manages resource allocation, job placement, and workload distribution across multiple machines for distributed training. Machine learning engineers, infrastructure teams, and training platforms use it to run parallel model training jobs across GPU and TPU clusters.
ray · anyscale · kubeflow · slurm · skypilot · volcano · run:ai
LLM fine-tuning frameworks are software libraries and platforms that enable distributed training and optimization of large language models across multiple GPUs or TPUs. ML engineers, researchers, and organizations use them to adapt pre-trained models to specific tasks or domains.
axolotl · torchtune · unsloth · nvidia nemo
Federated learning frameworks enable model training across distributed data sources where raw data remains local and only model updates are shared. Data scientists, healthcare institutions, financial firms, and mobile device networks use these systems to train models on sensitive or geographically dispersed data.
nvidia flare · openfl · fedml
Decentralized training networks enable multiple independent nodes to collaboratively train machine learning models without a central coordinator, used by researchers and organizations building resilient training infrastructure across geographically distributed or heterogeneous compute.
nous research · gensyn · pluralis research