Paddle Continuous Evaluation Baselines

Howtos

Contribute

Run pre-commit run -a before your PR, this will help to format code automatically

Add New Evaluation Task

Reference mnist task, the following files are required by CE framework:

run.xsh , a script to start this evaluation execution
- this script can be any bash script, just place #!/bin/bash or #/bin/xonsh to the head if it is written in the bash or xonsh language
continuous_evaluation.py to include all the KPIs this task tracks
latest_kpis directory, include all the baseline files

PR and Add to Service

PR to fast branch, and run ce-kpi-fast-test test on teamcity,
if passed, PR from fast to master branch.

Add new KPI to track

Reference the interface kpi.py, there are two basic KPIs:

LessWorseKpi
GreaterWorseKpi