This project was created to increase the quality of Mattermost products by having highly effective, well-thought and well-maintained living test cases through high collaboration and based on industry-accepted best practices.
- Create or update test cases.
- Mark test cases as tested by you.
- Enhance GitHub and Zephyr Scale integration.
- Improve the web application for test cases.
- Assist with the Open Test Initiative.
This project is actively maintained and owned by the QA Core Team.
Phases:
- Open test data and work with test organization.
- Open up test contributions from core staff using pull request flow and GitHub approval.
- Open up test contributions to the community, especially from the QA Community.
- Build web applications for ease of viewing, creation, editing and reporting.
- Iterate and improve the Open Test Initiative.
If you're interested in shaping this project, please reach out to our community channel. Say hi, and let us know what you think!
-
/data
folder: Main folder of the repo where written test cases live primarily. Test cases can be found at/data/test-cases
.Note: We're still in the process of restructuring the folder at
/data/test-cases
. The primary goal is to align with our Multi-Product Architecture. The rough structure would beProduct > Feature > Client > Test Cases
. -
/src
folder: Scripts used for integrating with our third-party Test Management System. These scripts retrieve or sync information.
There are three ways to get started in contributing to this project.
- It is highly recommended to for it provides a fully initialized and readily available environment. After opening it, make and submit changes using your forked repository like this:
# Add your forked repository as remote git remote add fork https://github.com/<username>/mattermost-test-management.git # Create branch git checkout -b <branch_name> # Once ready, push the current branch to your fork git push fork # Finally, create a PR directly from GitHub
Note: In Gitpod, if you forget to do the above, pushing a change or branch to the origin will result in a permission error, like this one:
remote: Permission to mattermost/mattermost-test-management.git denied to <github_handlename>. fatal: unable to access 'https://github.com/mattermost/mattermost-test-management.git/': The requested URL returned error: 403
- You can also work on your local machine by checking out this repo. Make sure you have Deno installed on your machine to run scripts in the /src folder. See the Deno installation docs for more information.
- If you're contributing test cases only where they are written in Markdown files, you may do so directly via this GitHub repo. See the GitHub docs on creating new files and submitting pull requests.
- View test cases at
/data/test-cases
. Each test case is written in a Markdown file with two main sections:Frontmatter
: for the metadata of a test case where each field is grouped into the categories "required", "optional", and "do not change".Content
: for the description and steps information.
- When creating a test case:
- Copy the basic template at data/TEST_CASE_TEMPLATE.md.
- Update the required information in the
Frontmatter
section, such as:name
- the name of the test casestatus
- with the default value ofApproved
priority
- with the default value ofNormal
folder
- the name of a folder. It should follow the exact name of an existing folder (e.g.Channels
fordata/test-cases/channels
) or a slug equivalent of a new folder (e.g.2236 Shiny Feature
fordata/test-cases/channels/2236-shiny-feature
).Note:
2236
is in the format ofYYWW
(shortened year and week) and refers to when a feature is introduced.authors
- the author's GitHub handle (e.g.@original
)team_ownership
- (array) the team name; who owns a feature or test case.priority_p1_to_p4
- has the default value ofP2 - Core Functions (Do core functions work?)
- Update/populate the content section fields, such as:
Objective
(optional)Precondition
(optional)Step
(array, required)Description
(required)Test Data
(optional)Expected
(optional)
- When updating a test case:
- In addition to the same process when creating a new test case, the submitter should append his/her GitHub handle to
authors
field, comma-separated (e.g.authors: @original, @updater
). - To indicate that a test case has been tested by a contributor, the submitter should append his/her GitHub handle and date tested (mm-dd-yyy) to optional
tested_by_contributor
field, comma-separated (e.g.tested_by_contributor: "@qa_contributor|08-20-2023, @qa_contributor|10-15-2023"
).
- In addition to the same process when creating a new test case, the submitter should append his/her GitHub handle to
- Once changes have been made, run
deno task validate
and see if all changes are valid. Fix errors that get flagged in the terminal. - Always run
deno task check
to ensure all formatting is correct and there are no lint errors. - Create a branch and submit changes as a pull request(s).
Note: It's highly recommended to follow Conventional Commits for the
title
of the pull request.
- Find test cases at
/data/test-cases
. - Update a test case by adding your GitHub handle and the date (mm-dd-yyyy) when you tested it:
- tested_by_contributor: null
+ tested_by_contributor: "@qa_contributor|08-20-2023"
- or use commas for multiple testing contributions:
- tested_by_contributor: "@qa_contributor|08-20-2023"
// from same contributor
+ tested_by_contributor: "@qa_contributor|08-20-2023,@qa_contributor|10-15-2023"
// from different contributor
+ tested_by_contributor: "@qa_contributor|08-20-2023,@another_qa_contributor|10-15-2023"
- Checkout this repo and make sure that
Deno
is installed on your local machine or . - Scripts can be found in the
/src
folder and are all written in Typescript. - Integrating Zephyr Scale and Jira requires access tokens. You may set these via environment variables or through an
.env
config file with values forZEPHYR_TOKEN
andJIRA_PAT
. See these docs on how to get the access tokens: - See a list of tasks that are available at deno.jsonc. You can also list these tasks using the
deno task
command at the root of the repository.- To start up Silver Bullet (SB), run
deno task silverbullet
, then navigate to http://localhost:3333. - To start the test management site, navigate into the
www
folder (cd www
), rundeno task start
, then navigate to http://localhost:8000/.- The search bar in the left-hand-side (LHS) of the Test Cases page requires Elasticsearch to be running and accessible at http://localhost:9200. You can start this by entering the command
docker compose up
in thewww
folder.
- The search bar in the left-hand-side (LHS) of the Test Cases page requires Elasticsearch to be running and accessible at http://localhost:9200. You can start this by entering the command
- To start up Silver Bullet (SB), run
- Always run
deno task check
to ensure all formatting is correct and there are no lint errors. - Create a branch and submit changes as a pull request(s).
Note: It's highly recommended to follow Conventional Commits for the
title
of the pull request.
- GitHub Actions runs codespell over your
data/
folder and will point out any common spelling mistakes in the Pull Request. - Locally there is .pre-commit-config.yml. You can configure as explained here, precommit
- If you want to skip precommit locally, export PRE_COMMIT_ALLOW_NO_CONFIG=1, before running git commit.
Licensed under Apache License 2.0. Read the full text here.