VMAgent is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks. It is developed by the Multi-Agent Artificial Intelligence Lab (MAIL) in East China Normal University and Algorithm Innovation Lab in HUAWEI Cloud. VMAgent is constructed based on one month real VM scheduling dataset called Huawei-East-1 from HUAWEI Cloud and it contains multiple practicle VM scheduling scenarios (such as Fading, Rcovering, etc). These scenarios also correspond to the challanges in the RL. Exploiting the design of RL methods in these secenarios help both the RL and VM scheduling communities. To emphasis, more details about VMAgent can be found in our paper VMAgent: Scheduling Simulator for Reinforcement Learning. Our another paper Learning to Schedule Multi-NUMA Virtual Machines via Reinforcement Learning has employed this VMAgent simultor to design RL-based VM scheduling algorithms.

Key Components of VMAgent:

• SchedGym (Simulator): it provides many practical scenarios and flexible configurations to define custom scenarios.
• SchedAgent (Algorithms): it provides many popular RL methods as the baselines.
• SchedVis (Visulization): it provides the visualization of schedlueing dynamics on many metrics.

Scenarios and Baselines

The VMAgent provides multiple practical scenarios:

Scenario	Allow-Deletion	Allow-Expansion	Server Num
Fading	False	False	Small
Recovering	True	False	Small
Expanding	True	True	Small
Recovering-L	True	False	Large

Researchers can also flexibly customized their scenarios in the vmagent/config/ folder.

Besides, we provides many baselines for quick startups. It includes FirstFit, BestFit, DQN, PPO, A2C and SAC. More baselines is coming.

Installation

git clone https://github.com/mail-ecnu/VMAgent.git
cd VMAgent
conda env create -f conda_env.yml
conda activate VMAgent-dev
python3 setup.py develop

To use runx module and .cv2 module, you may need to do next instructs:

pip install runx
sudo apt update
sudo apt install libgl1-mesa-glx

Quick Examples

In this quick example, we show how to train a dqn agent in a fading scenario. For more examples and the configurations' concrete definitions, we refer readers to our docs.

config/fading.yaml:

N: 5
cpu: 40 
mem: 90
allow_release: False

config/algs/dqn.yaml:

mac: 'vectormac'
learner: 'q_learner'
agent: 'DQNAgent'

Then

python vmagent/train.py --env=fading --alg=dqn

It provides the first VM scheudling simulator based on the one month east china data in HUAWEI Cloud. It includes three scenarios in practical cloud: Recovering, Fading and Expansion. Our video is at video. Some demonstrations are listed:

Docs

For more information of our VMAgent, we refer the readers to the document. It describes the detail of SchedGym, SchedAgent and SchedVis.

Data

We collect one month scheduling data in east china region of huawei cloud. The format and the stastical analysis of the data are presented in the docs. one month east china data in huawei cloud.

Visualization

For visualization, see the schedvis directory in detail.

References

• Junjie Sheng, Shengliang Cai, Haochuan Cui, Wenhao Li, Yun Hua, Bo Jin, Wenli Zhou, Yiqiu Hu, Lei Zhu, Qian Peng, Hongyuan Zha and Xiangfeng Wang, VMAgent: Scheduling Simulator for Reinforcement Learning. IJCAI Demos, 2022.
• Junjie Sheng, Yiqiu Hu, Wenli Zhou, Lei Zhu, Bo Jin, Jun Wang and Xiangfeng Wang, Learning to Schedule Multi-NUMA Virtual Machines via Reinforcement Learning, Pattern Recognition, 121, 2021, pp.108254.

License

Licensed under the MIT License.