This article was firstly released at Container Cube on December 24th, 2020, refer to锐天投资基于Volcano的大规模分布式离线计算平台的应用实践

This article was firstly released at Container Cube on October 27th, 2020, refer to HPC on Volcano：容器在气象行业HPC高性能计算场景的应用

Kubernetes has become the de facto standard for cloud native application orchestration and management. An increasing number of applications are being reconstructed or built to employ Kubernetes. High performance computing (HPC) is a popular distributed computing mode and is widely used in many fields. For users who have deployed HPC applications and are eager to containerize and manage their applications using Kubernetes, Volcano, CNCF's first distributed scheduling system for batch computing, is a good choice. Volcano supports multiple types of computing frameworks, such as Spark, TensorFlow, and Message Passing Interface (MPI). This article uses a traditional HPC application, the Weather Research and Forecasting (WRF) model, as an example to describe how Volcano works for HPC applications.

This article was firstly released at Container Cube on May 27th, 2021, refer to 小红书基于Volcano的大规模离线与在线推荐模型训练实践

This article was firstly released at Container Cube on December 24th, 2020, refer to Volcano在中科类脑云OS中的落地实践

Introduction to the Leinao cloud AI platform​

The Leinao cloud AI platform includes an AI development platform, public service platform, AI visualized operations platform, and an AI community.

• The AI development platform provides end-to-end technical support and solutions for AI researchers in a diverse range of business scenarios.

• The public service platform provides news and insight into AI for institutions and for the general public.

• The AI visualized operations platform helps managers make better informed decisions.

• The AI community is where AI developers and companies gather, exchange ideas, and improve their skills.

This article was firstly released at Container Cube on November 6th, 2019, refer to 百度飞桨(PaddlePaddle)分布式训练在Volcano系统上的实践

PaddlePaddle is a deep learning framework open-sourced by Baidu in September 2016. It aims to provide a secure, easy-to-use, and scalable deep learning platform.

This article was firstly released at Container Cube on January 5th, 2021, refer to 基于Volcano的锐天离线高性能计算最佳实践

The easiest way to deploy Volcano is using Helm charts.

Preparation​

Clone the repository to a local path:

# mkdir -p $GOPATH/src/volcano.sh/
# cd $GOPATH/src/volcano.sh/
# git clone https://github.com/volcano-sh/volcano.git

1. Volcano Images​

Official images are available on DockerHub. You can also build local images by running the following command:

cd $GOPATH/src/volcano.sh/volcano
make images
## Verify your images.
# docker images
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
volcanosh/vk-admission     latest              a83338506638        8 seconds ago       41.4MB
volcanosh/vk-scheduler     latest              faa3c2a25ac3        9 seconds ago       49.6MB
volcanosh/vk-controllers   latest              7b11606ebfb8        10 seconds ago      44.2MB

NOTE: Ensure that the images are correctly loaded to your Kubernetes cluster. For example, if you are using kind luster, run the kind load docker-image <image-name>:<tag> command for each image.

2. Helm Charts​

Who is Using kube-batch?​

This document describes how to run kube-batch as a batch scheduler for Kubernetes. To get the complete code, go to master.