2021 Feb Endgame Plan

[hzy46]

release plan #5253
release manager @hzy46
test period: TBD

New Feature

Feature | Feature Owner | Test Owner(s) | Test case

• Job submission page new UI | @debuggy | @yiyione | New Job Submission openpaimarketplace#206
• Job protocol update: add prerequistes | @hzy46 | @suiguoxin @Binyang2014 | test cases
• Marketplace 2021 Feb. Release Test Plan openpaimarketplace#225
• Dockerhub cache | @debuggy | @Starmys | test case TBD
• GPU Utilization Statistics | @suiguoxin | @Binyang2014 |
  • Test cases:
    • set alert schedule in services-configuration.yaml->alert-manager->cluster-utilization->schedule
    • check the email report
• save SSH publish keys on user profile page | @Lijiaoa @suiguoxin | @hzy46
  • Test cases:
    • in webportal, check add / delete /view ssh public key functions
    • submit job with ssh enabled, add some secret key/value (to test that they have no conflict)
    • kubectl get secrets | grep usercred, a corresponding secrets should be created in default namespace
    • both pre-saved key & new key should work
• Log experience | @Binyang2014 | @hzy46 |
• Reduce ansible logs when deploy Reduce ansible logs when deploy #5305 | @Starmys @suiguoxin | @hzy46

Bug Fixes

• DB controller | @hzy46 | @debuggy |
  • Tolerant to wrong framework spec make DBController tolerant to wrong framework request #5284
  • Remove sensitive fields in db Remove sensitive fields when the job is completed and deleted #5289
  • Fix memory leak Fix database controller memory leak #5309
• Set correct launchTime in rest-server fix get jobs api launchedTime field issue #5307 | @suiguoxin | @Binyang2014
  Test cases:
• call API https://int.openpai.org/rest-server/api/v2/jobs/{frameworkName}
• check jobStatus->appLaunchedTime filed
• check job in different status (waiting/ running / completed)

Regression Test

(as usual)

Remaining Work Items

Found Issues

• Database may use unmounted host path Database may use unmounted host path #5341
• K8s API server's cert need renew each year K8s API server's cert need renew each year #5334
• Fix wrong nvidia gpu card number Fix wrong nvidia gpu card number  #5353
• Fix get-pip.py compatibility problem with Python3.5 Fix get-pip.py compatibility problem with Python3.5 #5356

Agile CI and nightly-build

P1 Agile CI and split heavy tests with nightly-deployment #5173 @yiyione

• Setup the nightly-deployment and test
• Build & publish nightly tag image
• Add build failure alert to contributor
• Replace current Jenkins CI

Document

• fault document @mydmdm
• update add&remove node doc @suiguoxin
• update aad doc @suiguoxin
• document for new features
  • docker image cache
  • save user's SSH keys
  • GPU Utilization Statistics
  • Job protocol update: add prerequistes