dstack

dstack is an open-source container orchestration platform designed for AI/ML workflows. It provides a unified control plane for ML teams to simplify the end-to-end process of developing, training, fine-tuning, and deploying generative AI models, reduce the complexity of managing underlying infrastructure (such as Kubernetes), and optimize GPU resource costs.

dstack supports multi-cloud (e.g., AWS, GCP, Azure), on-premise server clusters, and existing Kubernetes environments. On the hardware side, it natively supports NVIDIA, AMD, TPU, Intel Gaudi and other leading AI accelerators.

The basics are to install Git, Docker, and Docker Compose. After deploying the dstack server and CLI tools, you enable resources by configuring them (e.g., Fleet) in a config file. For on-prem clusters, you only need Docker and SSH keys to manage.

Fleet (resource pool) is a core concept in dstack that defines and manages a group of compute resources (such as number of nodes, GPU types and quantities). It supports on-demand resource creation and automatic release of idle resources after tasks complete to control costs, and is a key component for efficient GPU orchestration.

dstack achieves cost savings through unified resource orchestration and intelligent scheduling, delivering GPU resources on demand and maximizing utilization to avoid idle capacity. It claims to help teams reduce infrastructure costs by 3x to 7x.

dstack is designed for AI/ML teams, whether startups or large enterprises. It offers a range of deployment options from open-source self-hosted to hosted services (dstack Sky), meeting the needs of individual developers or small teams for experimentation as well as enterprise-grade, large-scale production deployments.