Skip to content
This repository has been archived by the owner on Aug 26, 2022. It is now read-only.

v2.0.0
Compare
Choose a tag to compare
@hyunwoongko hyunwoongko released this 14 Feb 18:46
· 713 commits to master since this release
v2.0.0
0582b8a
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Official release of OSLO 2.0.0 🎉🎉

This version of OSLO provides the following features:

  • Tensor model parallelism
  • Efficient activation checkpointing
  • Kernel fusion

We plan to add the pipeline model parallelism and the ZeRO optimization in the next versions.


New feature: Kernel Fusion

{
  "kernel_fusion": {
    "enable": "bool",
    "memory_efficient_fusion": "bool",
    "custom_cuda_kernels": "list"
  }
}

For more information, please check the kernel fusion tutorial

Assets 2
Loading