Home

Test Working Group

• Test Working Group
  • Welcome
  • TWG Representatives
  • TWG Charter
  • Motivation
  • Scope
  • Test prerequisite for Code Commit
  • Type of test beds
  • SIT test bed charter
  • SIT Testbed SLA

Welcome

Welcome to test working group for DENT OS project. The DENT TWG was formed in January 2021 to help DENT community build and maintain test content.

Please join the TWG mailing list. TWG meets every 2 weeks. Please register for TWG Zoom meeting.

TWG Representatives

Company	Representatives
Amazon	Sandeep Nagaraja, Chetan Murthy, Taskin Ucpinar
Keysight	Manodipto Ghose, Mircea Dan Gheorge

TWG Charter

In line with the goals and motivations for DENT OS project and under the guidance of DENT technical steering committe (TSC) the DENT TWG's charter will include the following:

• Define processes that ensures DENT OS maintains the following:
  • Quality of code, automation runs and physical condition of labs
  • Ability to choose the right lab and deploy OS for testing.
  • A vendor neutral environment for testing.
  • Support inter-op testing.
• Continuous test process and infrastructure improvement.
• Build and maintain community test bed (subject to resource and support availability from members)
• Long term vision: 5 years down the line (TSC to determine and TWG can add to charter)

Motivation

As we move along the process of test development and maintenance for DENT project the high level motivation for this are as listed below.

• Provide a single productionable branch for opensource NOSes
• Support active co-development on community branch(es)
• Establish rules and processes to avoid instabilities
• Minimize instabilities caused by incoming PRs

Scope

DENT TWG will focus on two lines of test execution to make sure quality standards are met at each phase.

• Functional tests in community and member labs with the below goals:
  • Setting Measurable Common Goals
  • Common Development and Test Environment
  • Strict Code Commit Standards
• System integration test in community and optionally at member labs with below goals:
  • Test plans for Inter-op and system
  • Test automation for system
  • Review board
  • Use case coverage (retail)

Test prerequisite for Code Commit

Type of test beds

SIT test bed charter

• Setup is dedicated to run system test regression fully automated as part of CI/CD pipeline. Intention is to maintain 100% setup utilization.
• As number of branches, builds and test cases increase, and it is not possible to run every build only best builds will be selected for SIT (feature test results may be used)
• Debugging:
  • In case of failures, logs will be collected and made available automatically for debugging. Issue owner will be determined in TWG triage meeting.
  • In case the required info for a bug is not available, enhancements will be made to collect the required info.
  • If a bug cannot be reproduced on feature test setup, or vendor SIT setup, and logs are not sufficient, on case-by-case basis access to setup may be provided for limited time.
• Neutrality:
  • Setup not to be used to gain competitive intelligence on participant members.
  • Only SIT cases defined in the GitHub testing repo can be executed. 
  • Customer demos to be requested by DENT TWG or DENT marketing. It must be scheduled ahead of time (Minimum 2days) and involve Keysight lab personnel to facilitate.

SIT Testbed SLA

• 24 x 7 up time on SIT automated test bed and availability of test results/logs
• Daily 4 hours of test bed access for debugging during first 6 months of operation – Mon to Fri
• 4 hours a week resource from Keysight to support lab infrastructure for first 6 months of operation.
• Failure triaging to be owned by the community. TWG will facilitate regular triage meetings

Visit the Dent Project website at dent.dev