You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are using a very different meaning for the term "regression testing" than is usually meant.
Regression testing is when a programmer runs the unit test suite (or a subset) after every small change in code. This is usually done on the programmers computer (i.e. with "make test"), but can be accomplished with GitHub continuous integration, if the programmer commits changes frequently, and pays attention to the results of the CI run on each commit.
The name comes from the agile term "regression", which is when a programmer working on the code accidentally breaks code that was already working. It is intended to be done frequently, because the cost of fixing a bug goes up dramatically over time.
What we are doing is not regression testing for two reasons:
1 - we are not testing any results for correctness, but only that they have not changed.
2 - the tests are not being run by the programmer as the work is done, but instead done at the end, when work is submitted as a PR. Finding bugs months after they were introduced is not what anyone else means by "regression testing."
Let us re-term this "consistency checking." All we are checking is that we did not change any results. We are not checking that the results are actually correct.
When we have unit tests, and the programmers are using them properly, then we will have regression testing. Let's not claim to have something that we don't have.
The text was updated successfully, but these errors were encountered:
I believe this has been changed everywhere it needs to change. Moving forward, please stop calling these "regression" tests, because that is claiming something we don't yet have - comprehensive unit testing.
One day, when we have comprehensive unit testing, we will have regression testing.
We are using a very different meaning for the term "regression testing" than is usually meant.
Regression testing is when a programmer runs the unit test suite (or a subset) after every small change in code. This is usually done on the programmers computer (i.e. with "make test"), but can be accomplished with GitHub continuous integration, if the programmer commits changes frequently, and pays attention to the results of the CI run on each commit.
The name comes from the agile term "regression", which is when a programmer working on the code accidentally breaks code that was already working. It is intended to be done frequently, because the cost of fixing a bug goes up dramatically over time.
For more on regression testing: https://en.wikipedia.org/wiki/Regression_testing
What we are doing is not regression testing for two reasons:
1 - we are not testing any results for correctness, but only that they have not changed.
2 - the tests are not being run by the programmer as the work is done, but instead done at the end, when work is submitted as a PR. Finding bugs months after they were introduced is not what anyone else means by "regression testing."
Let us re-term this "consistency checking." All we are checking is that we did not change any results. We are not checking that the results are actually correct.
When we have unit tests, and the programmers are using them properly, then we will have regression testing. Let's not claim to have something that we don't have.
The text was updated successfully, but these errors were encountered: