Development process

We will manage development tasks via the github issue tracker. GitHub issues have built-in Open and Closed states, and can be assigned to team members. For workflow management, they support the notion of tagging issues. We have set up the tracker with pre-defined labels corresponding to the workflow described below.

For each distinct development task, we will create an issue. In particular, each distinct task in the functional spec will have an issue. True bug reports are thus just a special category of issue.

Note that we are using our tracker for three distinct purposes:

1. Bug Report Processing -- users will report bugs to us, and we will use tags to indicate how the bug will be handled and it's current status. The labels that accomplish this are
   • Accepted (we will fix this in the next release if not sooner)
   • Address Later (we will fix this, but perhaps in a later release)
   • Cannot Reproduce (won't do anything further without more info)
   • Duplicate (won't do anything further)
   • Expected Behavior (not an actual bug, so won't do anything further)
   • Fixed (the issue is now fixed in the development code)
2. Development Workflow management - we will use these tags to indicate and track where in our development workflow issues are. The labels that accomplish this are:
   • Investigate (bug reported but it is not clear what is causing it yet)
   • Ready For Development (investigation is complete, and a solution designed)
   • In Development
   • Ready for Review (a testing branch has been prepared for review and testing)
   • Ready for Release (the issue has passed testing, and is ready to merge to master)
3. Quality assurance management - we will use tags to indicate whether the bug requires testcases and give the status of the development of unit tests. The labels that accomplish this are:
   • Testcase Wanted: a reduced testcase demonstrating the issue is expected.
   • Unit Test Wanted: a reduced testcase is available but we want to convert it into an automated test.
   • In Testsuite: an automated test has been written and is integrated in the testsuite.
   • Do Not Write Automated Test: the issue won’t be tested automatically because it is hard to write a test.

Bug Report Processing

As bugs come in, we will look at each and investigate. As soon as reasonable, we will mark it with one of the statuses Accepted | Address Later | Cannot Reproduce | Duplicate. Cannot Reproduce and Duplicate bugs are closed immediately. Address Later issues stay open, but nothing further is done is the short term.

Accepted bugs go into the Development workflow.

Development Workflow

This is an overview over the process outside our Source control policies which follow this branching model.)

For each Accepted bug, the developer identifies the problem and/or outlines a design to address the issue. Once a design outline is complete, the developer marks the issue as Ready for Development and notifies the team. For simple issues, the developer may be able to outline a design in the issue, and setting the status is mostly a heads up to avoid conflicts. For more substantial issues, the developer is responsible for leading a discussion on mathjax-dev, writing design pages here, and obtaining consensus on the implementation plan. In these cases, setting the status to Ready for Development is a "last call" and denotes the end of the design phase.

When development begins, the issue is moved to In Development.

When development is complete, the developer merges the working branch into the develop branch (see the Source control policies), again notifies the team (via a pull request) and tags the issue as Ready for Review. At least one senior project developer has to review and sign off on all new work. The code is also tested in the develop branch. If problems arise, they may spawn new issues, or merely send the original issue back to In Development. Ideally, at least one automated test should be written to check the issue and in that case the QA flag is changed to In Testsuite. If that's too difficult or not necessary, the QA flag may be changed to Do Not Write Automated Test and the issue is then only tested manually.

Once the issue is tested, it goes to Ready For Release. Issues in this state are complete, but have not been merged into the develop or release line for which they are destined yet. For now, Davide will be responsible for making all merges of Ready For Release issues into the MathJax repo.

Once an issue has been merged, it will get the Fixed tag, and Closed.

Quality assurance management

When an issue is reported, it may contain more or less accurate description of the bug and how it happens. Sometimes, the reporter has provided a testcase, that is a code snippet or test page demonstrating the bug. For convenience, we often want to isolate the bug and make such a testcase as small as possible. For that purpose, we mark the issue as Testcase Wanted.

Once we have a reduced testcase, it is generally possible to convert it into one or more unit tests. When it is the case, we mark the issue as Unit Test Wanted. The conversion may be more or less straightforward, depending on the kind of testcase we have. For example, an issue which involves a complex scenario to be reproduced may require a few lines of code to simulate each user action. At the opposite, issues related to layout or LaTeX conversion can generally be written using the reftest paradigm i.e. using only a reference and test page.

Finally, when tests are written and included in the testsuite, we mark the issue as In Testsuite. In theory, this means that we have unit tests covering all aspects of the bug and hence the issue remains in the In Testsuite status forever. If we can not write a unit test for a given issue we use the tag Do Not Write Automated Test.