NNFusion v0.3 (JAN~FEB) Release Plan

[AlisaChen98]

This is the draft plan for the upcoming iteration in Jan ~ Feb, it's a 6 week's iteration.

Release Plan for V0.3

• Release Manager: @wenxcs
• Feature Freeze Date: 2021.03.04 TBD
• Code Freeze Date: 5.18 with fixes for [BUG] BlockFusion bug in model training after PR#242 #266, [BUG] kernel fusion pass check failure in broadcast fusion #265, [BUG] Possible bug in nnfusion::kernels::cuda::DepthwiseConv2dNative kernel emitter #264, [BUG] Shape inference of generic_op_define/DepthwiseConv2dNative does not consider filter_shape and padding #263
• Test Verification/Bug Bash: 5.18 - 5.19
• Release Date: 5.20
• Release Note:

---

Annotation Legend:

Mark	Description
🏃‍♂️	work in progress
👪	work under review
✋	blocked task

Feature

Mechanism

• P0 - Custom op support | @jlxue @wenxcs @mzmssg | Testing @wenxcs

  • C++ | 3 days
  • JSON | 2 days
  • TF Protobuf
  • ONNX
  • doc @jlxue

• P0 - Support training | Testing @xysmlx

  • bert training on real dateset ｜2 days｜@mzmssg
  • LSTM Pytorch @xysmlx
  • doc @mzmssg

• P0 - 🏃‍♂️ C++ support HLSL | @jlxue | | Testing @mzmssg

  • port wrapper lib to XDev ｜2 days｜ @jlxue
  • C++ codegen for HLSL｜1 day｜ @xiayuqing0622

Refactor/Improvement

• P0 - Bert Related Optimization | 14～18days｜@xiayuqing0622 | Testing @yuanjie
  • LayerNorm fusion | 2~3 days
  • Embedding layer fusion| 2~3 days
  • Skip layer norm fusion| 2~3 days
  • Attention fusion| 2~3 days
  • Gelu fusion| 2~3 days
  • Matmul add fusion| 2~3 days

Bug Fix

• model debug

• check grad number & Unsortedsegmentsum op: @yuanjie

• Antares translate IR bugs: (check with Yuxiao): @jlxue @weicui
  • padding for AvgPool [BUG] Antares IR problem in AvgPool operator #172
  • DepthwiseConv2d [BUG] Antares IR problem in DepthwiseConv2d operator #192 (asymmetric padding)
• Generate valid tensor name when the given name is invalid [BUG] Tensor name might not be legal variable name #163 @xysmlx
• onnx op_bridge check [BUG] onnx op_bridge check fails for some ops even when these ops are not used by the input model  #218 @mzmssg

---

Deferred

• offline inference | @wenxcs

  • bert hugging face | 5 days
    • autotunning for new hardware
    • support apex
  • nnstream: streaming offline inference interface| 10 days
  • 🏃‍♂️ general sub-graph substitution | @Niupple
  • matching and substituting algorithms | 4 days
  • python rule editing interface | 2 days

• support HLSL end-to-end python interface | @mzmssg

• support low-precision & mixed-precision | @Niupple

  • train model with pure half precision | 3 days
  • fp32 master copy of weights | 3 days
  • manual/auto loss scaling factor | 3 days

• Control-flow support ｜@xysmlx

  • control-flow op define: loop, if｜@xysmlx
  • onnx frontend for control-flow ops｜@xysmlx
  • codegen graph pass whose input is a graph and output is a function｜@xysmlx
  • codegen for control-flow ops ｜@xysmlx
  • refactor codegen & other graph passes ｜@xysmlx @xiayuqing0622

• sub-graph substitution | @wenxcs @Niupple sub-graph substitution  #127

• Test strategy ｜ @wenxcs

• support pip install ｜10 days | @mzmssg

  • refine python code structure
  • pip install python sources
  • pip install nnfusion cli

---

Stretch Goals

• docker image support low version of cuda driver | @wenxcs
• Control-flow support ｜@xysmlx
  • python integration
• new profiler: support different profiling information, e.g., time, kernel time provided by nvprof | @xysmlx
  • Base profiler | 5 days
  • Update existing profilers | 1 day
  • CUDA profiler: kernel time, num of __syncthreads(), num of active thread blocks, shared memory, grid/block size | 2 days
  • Kernel profiling pass | 1-2 days
• Profiling cache DB | @xysmlx
  • DB schema and interfaces
  • Integrate with profiler
• BlockFusion | @xysmlx
  • BlockFusion active thread block check [ENHANCEMENT] Active block check in -fblockfusion_level=2 #50
  • Process __shared__ and __syncthreads() with profiler to support any BlockCudaEmitter::get_kernel_cache_entry() function
• Generate identifier to identify unique op configs | @xysmlx

[nnfbot]

Thanks for the report @AlisaChen98! I will look into it ASAP! (I'm a bot).

[AlisaChen98]

Our main target of this release:

1. Custom op support
2. Offline inference(Take Bert as a typical example)
3. Other fix and enhancements

[wenxcs]

NNFusion v0.3版本发布

特性更新

• 支持在真实数据集上端到端的BERT模型训练（ONNX格式）
• 增加一系列针对Transformer模型的算子融合优化
• 提供扩展自定义算子的C++和JSON接口
• 支持新的HLSL代码生成

其他更新

• 更新相关文件
• 修复bug