- Start Date: 2019-06-07
- RFC PR: #25
- Authors: Toru Nagashima (@mysticatea)
This RFC improves RuleTester
to check more mistakes.
RuleTester
overlooks some mistakes.
- Using non-standard properties of AST (typescript-eslint/typescript-eslint#405).
Especially,node.start
andnode.end
exist in ASTespree
made, but it's not standardized and some custom parsers don't make those properties. Butnode.loc
hasstart
/end
properties, so it's hard to detectnode.start
/node.end
with static analysis. Therefore,RuleTester
should detect those. - Untested autofix.
If people forgot to writeoutput
property in test cases,RuleTester
doesn't test autofix silently. errors
property with a number (found in eslint/eslint#11798).
errors
property with a number ignores syntax errors in test code. We overlooked the mistake of tests/lib/rules/complexity.js#L84 due to this. The numbererrors
property cannot check the reported error was the expected error.- Typo property names in
errors
property with objects.
eslint/eslint#12096.
- Disallowing
node.start
andnode.end
- Ensuring to test autofix
- Changing the
errors
property of a number to fail on syntax errors - Changing the
errors
property of objects to fail on unknown properties
RuleTester
fails test cases if a rule implementation used node.start
or node.end
in the test case.
- In
RuleTester
, it registers an internal custom parser that wrapsespree
or the parser ofitem.parser
toLinter
object. - The internal custom parser fixes the AST that the original parser returned, as like test-parser.js.
RuleTester
fails test cases if a rule implementation fixed code but output
property was not defined in the test case.
- If
output
property didn't exist but the rule fixed the code,RuleTester
fails the test case as "The rule fixed the code. Please add 'output' property." It's implemented around lib/rule-tester/rule-tester.js#L594.
RuleTester
fails test cases always if the code
has a syntax error.
RuleTester
fails test cases if any item of errors
has unknown properties.
RuleTester should be updated.
output
("optional" → "required if the rule fixes code")
This change may enforce plugin owners to fix their tests.
This is a breaking change that can break existing tests.
But the breaking cases may indicate that the rule was not tested enough.
- About "Disallowing
node.start
andnode.end
", we can standardize those properties. But it's a breaking change for custom parser owners. On the other hand, usingnode.start
andnode.end
breaks the rule if users used custom parsers, so the impact of this disallowing is limited.