2020 Oct ~ Nov release plan

[scarlett2018]

This plan captures the work in Nov 2020. This is a 4 week iteration major focus on system features and improvements, and we will ship it at the end of Nov.

Release Manager

@suiguoxin

Endgame

Feature freeze: TBD
Code freeze: 11.16
Scrum demo date: TBD
Release date & retrospective date: 11.30

INT bed deploy date: 11.17
Bug bash: #5087

---

Planned Items

multi-cluster

• P0 - Hybrid or multi-cluster management (SHAIIC) Multi Cluster Load Balance  #4929 @hzy46 ETA: 11.9 demo: 11.5

Autoscaler

• P2 doc refine @abuccts Update docs for Cluster Autoscaler on AKS Engine #5057 ETA: 11/6 in review

Log Collection

• P1 Log collection PAI logging pipeline enhance #4992
  • API @Binyang2014
  • Front-end @AmberMsy
    • update API logic for Stdout / Stderr / Sdtout+Stderr button ETA: 11/11
    • update Browse log folder ETA 11/16

Https related issues

• P2 doc ETA: 11/11 @vvfreesoul

Marketplace

• Data -- @debuggy
• P0 Move NFS to Azure Blob as backend -- ETA: Nov. 5
• P0 Upload Job output to Azure Blob ETA: 11.9
• P0 Download data from azure blob to local -- ETA: Nov. 9
• P1 Use Azure storage SDK for privacy -- ETA: 11.16
• P1 Refactor data use logic after change storage to blob ETA: 11.13
• Service Deployment -- @yiyione
• P0 Start Local Rest Server -- ETA: Nov. 2
• P0 Deployed Rest Server in PAI -- ETA: Nov. 5
• P0 Start database and save items into it ETA: 11.9
• P0 Register in PAI pylon PR: [Marketplace] Deploy scripts and config #5066 -- ETA: Nov. 10 PR review
• P0 Create marketplace item -- ETA: Nov. 16

Web Portal Experience

• P0 After click stop button successfully, the button is still enabled until the job becomes Stopping Job stop button no feedback after click successfully #5023 @debuggy ETA: 11/16 Disable stop button immediately after click #5079 in review

Admin Experience

• P1 Alert rules update: [Monitor] Alert System Refine #4810 @suiguoxin alert handler stop-job notice not clear to end user #5021 ETA: 11/10
• P2 Deployment issues Deploy Issues #5001 @suiguoxin ETA: 11/15 deployment process refine #5077 Deployment process refine #5085

Bug Fix

• Admin auth check issue in APIs: getGroupUserList, createGroup, creatUser, updateUserGroupList @suiguoxin update users's own profile: support AAD mode; update extension #5032 use checkAdmin middleware #5051
• updateUserGroupList not work @suiguoxin fix updateGroupUserList api issue #5121
• hived config issue caused by k8s coreDNS deployment CoreDNS pod will affect resource calculation for hived scheduler #5056 @Binyang2014 @hzy46 Set coreDNS requests to zero #5071
• Forever running job caused by out of sync between api server and database Forever running job caused by out of sync between api server and database #5065 DRI handbook doc @hzy46
  • Updated DRI handbook (in the Etcd Maintainence page)

Defer:

Web Portal Experience

• P2 Job detail, task list : sort, filter, search, export csv @yiyione
• P1 Job detail page cannot be searched by browser Job detail page cannot be searched by browser #4917 @debuggy defer can be solved with sort/filter
• P3 weportal overlap issue webportal home page responsive layout #5086

Admin Experience

• P2 Group management & VC request management for user and admin @yiyione Group & VC request management for user and admin #4949 ETA: defer

VS Code

• P3 bug fix @yiyione ETA: defer

HiveD scheduler

• P2 Cell as sku in hived scheduler. @abuccts (backbone support (config) for "Cell", submission form supports for Cell)
  ETA: design: in progress

• P2 Python 2 Deprecation Upgrade Python 2 to Python 3 #5042

Release Tracking

• Release Process: refine archived oneNote doc as github wiki @yiyione
• Deployment: doc for release branch deployment @yiyione

Rest Server Performance

• Rest-Server Job API Perf Tests and Improvements Rest-Server Job API Perf Tests and Improvements #5027

Web Portal Experience

• P2 Favorite jobs (SHAIIC) @suiguoxin

Submission Related Backlog

• P1 The defaulting should only be done after submission phase The defaulting should only be done after submission phase #4576 @debuggy ETA: TODO after 11/6
• P2 Submit job ui should show protocol prerequisites parts 2020 Oct. ~ Nov. Release Plan openpaimarketplace#73 @debuggy @yiyione @TobeyQin