Cut down test262 suite size in half

[szegedi]

We could probably cut down the test suite size of test262 by a factor of 2: for every test case, there's both a …-s.js and a …gs.js file. Their logic seems identical, so running both seems unnecessary. The -s ones use the "assert" function while the gs ones basically inline it (so they run even when "assert" is not defined). We should stick with one set instead of running both.

[sainaen]

Hm, your “factor of 2” might be a bit too optimistic. :)

Right now I see 119 pairs of *-s.js and *gs.js tests (out of 29693 total *.js files in test directory) and we run only 7 of them. Here’s how I counted:

$ find test262/test -iname '*.js' -type f | sed 's/gs\.js$//;s/-s\.js$//' | sort | uniq -c | grep -v ' 1 ' | wc -l
  119

$ ./gradlew -Dtest.single=Test262SuiteTest test
$ grep 'test262Case\[' buildGradle/**/org.mozilla.javascript.tests.Test262SuiteTest.html | sed 's/gs\.js,//;s/-s\.js,//' | sort | uniq -c | grep -v ' 1 ' | wc -l
  21 # we run each test with opt={-1,0,9}, so 21/3 = 7

But also,

• from what I understand, there is a slight difference: gs ones execute test in the top-level script context and seem to always expect SyntaxError during the “parse phase,” while -s run through eval(). For example, see language/literals/numeric/7.8.3-2gs.js vs language/literals/numeric/7.8.3-2-s.js
• not all -s have gs pair and vice versa. There’s language/types/reference/8.7.2-8-s.js but no language/types/reference/8.7.2-8gs.js, built-ins/Function/15.3.5.4_2-42gs.js but no built-ins/Function/15.3.5.4_2-42-s.js
• tests with es5id in the filename are being gradually renamed/removed from the upstream test262 repo; their current master has 112 pairs — that’s −7 in five months

[szegedi]

In this case, we should leave everything as-is, it's not worth spending time on it. On the other hand, thank you for your time looking into it and making a thorough analysis!