deep-learning sam optimizer pytorch ranger loss-functions lookahead nero adabound learning-rate-scheduling radam diffgrad gradient-centralization adamp adabelief madgrad adamd adan adai ademamix

transformer wgan-gp pytorch-implementation madgrad

sophia cifar10 lion second-order-optimization adamw super-convergence weight-decay sharpness-aware-minimization madgrad large-batch-optimization lion-optimizer