Skip to content
/ D-PSGD Public
forked from srQ-cpc/D-PSGD

Algorithm: Decentralized Parallel Stochastic Gradient Descent

Notifications You must be signed in to change notification settings

innerop/D-PSGD

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 

Repository files navigation

D-PSGD

Algorithm: Decentralized Parallel Stochastic Gradient Descent

Requirements

  • Install PyTorch (pytorch.org)
  • GPU clusters with OpenMPI to communicate

Training

A 20-layer ResNet model and Cifar10 dataset are choosed for evaluation. Use the code bellow to start a training process on 1 coordinator node and 4 training nodes.

mpirun -n 5 --hostfile hosts python PSGD.py --epochs 160 --lr 0.5

Results

Top1 accuracy rate on test dataset is shown below. cifar

About

Algorithm: Decentralized Parallel Stochastic Gradient Descent

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%